| Commit message (Collapse) | Author | Age | Files | Lines |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Move ResourceLoader classes to their own namespace. Strip the
"ResourceLoader" prefix from all except ResourceLoader itself.
Move the tests by analogy.
I used a namespace alias "RL" in some callers since RL\Module is less
ambiguous at the call site than just "Module".
I did not address DependencyStore which continues to have a non-standard
location and namespace.
Revert of a241d83e0a6dabedf.
Bug: T308718
Change-Id: Id08a220e1d6085e2b33f3f6c9d0e3935a4204659
|
|
|
|
|
|
|
|
|
|
|
|
| |
This reverts commit e08ea8ccb9932f9924a613056afcb2d01c8c7b39.
Reason for revert: Breaks Phan in extensions, and as far as I’m aware,
this change isn’t urgently needed for anything, so the simplest fix is
to revert it again for now. After PHP 7.4 it should be safer to try this
again (we hopefully won’t need the two “hack” classes by then).
Bug: T308443
Change-Id: Iff3318cbf97a67f821f78e60da62a583f63e389e
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Move ResourceLoader classes to their own namespace. Strip the
"ResourceLoader" prefix from all except ResourceLoader and
ResourceLoaderContext.
Move the tests by analogy.
I used a namespace alias "RL" in some callers since RL\Module is less
ambiguous at the call site than just "Module".
I did not address DependencyStore which continues to have a non-standard
location and namespace.
Change-Id: I92998ae6a82e0b935c13e02a183e7c324fa410a3
|
|
|
|
|
|
|
|
| |
The global function wfWikiID() is deprecated since 1.35 and it's usages
should be replaced with WikiMap::getCurrentWikiId().
Bug: T298059
Change-Id: I22d96b7aec17323d15a9bc401d4511ad2ee14165
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
addGoodLinkObj() has many optional arguments, but omitting them actually
means corrupting the cache.
Nearly all existing callers are in tests.
So LinkCacheTestTrait::addGoodLinkObject() was created only
for testing. It is better to have this method in the
trait, because building the row directly in each test
would make these tests brittle against schema changes.
The only usage in WMF production code was in WikiPage and has been
fixed.
Bug: T284955
Change-Id: I03a2bd9ed64fcc0281ee29a286c8db395a9e03d9
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
composer:
* mediawiki/mediawiki-codesniffer: 36.0.0 → 37.0.0
The following sniffs now pass and were enabled:
* Generic.ControlStructures.InlineControlStructure
* MediaWiki.PHPUnit.AssertCount.NotUsed
npm:
* svgo: 2.3.0 → 2.3.1
* https://npmjs.com/advisories/1754 (CVE-2021-33587)
Change-Id: I2a9bbee2fecbf7259876d335f565ece4b3622426
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
There is a fallback in Module->getConfig() to the global services
container. This is not meant to be used in practice, but there were
two places where this was missing: WebInstallerOutput, and various
PHPUnit tests.
* Add missing setConfig() to WebInstallerOutput.
* Add missing setConfig() to various tests. Most tests were already
doing this correctly and using the standard mock from
ResourceLoaderTestCase. Upon switching the last few tests as well,
I uncovered various errors due to the mock missing some settings
that the tested code uses, so these have been added now to
ResourceLoaderTestCase.
Bug: T277728
Change-Id: I44f16ec4e00423fb6f641e58fffc1d40e4571f01
|
|
|
|
|
|
| |
Bug: T278459
Depends-On: I9bc371fd299e7a5401ebc1b8d7456d9a4ecde90d
Change-Id: Ib3a0591766f873492ec5a24721e7840f4e1a6407
|
|
|
|
|
|
|
|
|
|
|
|
| |
Ended up using
grep -Prl '\->setMethods\(' . | xargs sed -r -i 's/setMethods\(/onlyMethods\(/g'
special-casing setMethods( null ) -> onlyMethods( [] )
and then manual fix of failing test (from PS2 onwards).
Bug: T278010
Change-Id: I012dca7ae774bb430c1c44d50991ba0b633353f1
|
|
|
|
|
|
|
| |
Result of a new sniff I25a17fb22b6b669e817317a0f45051ae9c608208
Bug: T274036
Change-Id: I695873737167a75f0d94901fa40383a33984ca55
|
|
|
|
| |
Change-Id: I636be48eb9f713680abac35d46091f7b49374696
|
|
|
|
|
|
| |
Bug: T192167
Depends-On: I581e54278ac5da3f4e399e33f2c7ad468bae6b43
Change-Id: I3a21fb55db76bac51afdd399cf40ed0760e4f343
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
* Remove redundant getContent() cases that were all testing the same.
The redirect logic should indeed be tested, but exists in getContentObj(),
not getContent(). This test was also mocking getContentObj() thus not
actually testing what the case claims to test.
Fortunately, the right test already exists (testGetContentForRedirects),
so this is redundant.
* Add actual coverage of successful outcomes for getContent (previously
they were all error/null cases), with JS content, and with CSS content.
* Fix broken test case for "Bad content model". This was not working because
it mocked out getContentObj, thus it wasn't actually testing "bad content model",
but rather pointlessly duplicated the previous test case.
Fix it by actually making it use a WikitextContent object, which makes it
test the branch that handles incompatible content models.
Change-Id: I59af5318e536c730755352e9be8f995df1f56a86
|
|
|
|
| |
Change-Id: If004cc4ff3835091c01a4df84006be3f6fa173b2
|
|
|
|
|
|
|
|
|
|
|
|
| |
* Remove redundant any() calls.
* Use willReturn() instead of will(returnValue()).
* Use yield for providers and add missing test case descriptions.
* Use createMock() instead of wfGetDB() where the DB isn't needed.
* For provideIsKnownEmpty, re-order the cases and add a few extra
cases, and document why they behave the way they do.
Change-Id: Iba42325a55bb3dfc50a8d2af46e1ddba8dda885a
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
This was done as a "clever" shortcut to make sure tests a little
but shorter, but also made them less consistent with normal code.
Remove this in favour of 'class' or 'factory' options as needed.
Also remove a bunch of unneeded register() calls.
The tests cover everything affected by this change.
Side fix - isFileModule should reject modules with 'factory'
the same way it rejected raw objects and non-FileModule 'class'
cases already. This is now covered by tests as well.
Bug: T222637
Change-Id: I3996317dbcd780cc6e0f82c84e769c08a3fc42bb
|
|
|
|
| |
Change-Id: I78b3315f26ab91b6b443f5b028a635552f82f5a3
|
|
|
|
|
|
|
|
|
|
|
|
| |
Also do so in various other test classes.
Follows-up 170c49d61cb8. Fixes Travis CI regression:
> 1) MediaWiki\Tests\Revision\MutableRevisionRecordTest::testSetGetPageId
> Failed asserting that 2 is identical to 0.
> tests/phpunit/includes/Revision/MutableRevisionRecordTest.php:129
Change-Id: I41c8bda8e620ebe7608a393d81f3b0f13af68ba7
|
|
|
|
|
|
|
|
|
| |
Added spaces around .
Removed empty return statement which are not required
Removed return after phpunit markTestIncomplete,
which is throwing to exit the test, no need for a return
Change-Id: I2c80b965ee52ba09949e70ea9e7adfc58a1d89ce
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
There has long been a hack for previewing edits to user JS/CSS, where
OutputPage would pass an 'excludepage' parameter to
ResourceLoaderUserModule to tell it not to load one particular page and
would instead embed that page statically. That's nice, but there are
other places where we could use the same thing.
This patch generalizes it:
* DerivativeResourceLoaderContext may now contain a callback for mapping
titles to replacement Content objects.
* ResourceLoaderWikiModule::getContent() uses the overrides, and
requests embedding when they're used. All subclasses in Gerrit should
pick it up automatically.
* OutputPage gains methods for callers to add to the override mapping,
which it passes on to RL. It loses a bunch of the special casing it
had for the 'user' and 'user.styles' modules.
* EditPage sets the overrides on OutputPage when doing the preview, as
does ApiParse for prop=headhtml. TemplateSandbox does too in I83fa0856.
* OutputPage::userCanPreview() gets less specific to editing user CSS
and JS, since RL now handles the embedding based on the actual
modules' dependencies and EditPage only requests it on preview.
ApiParse also gets a new hook to support TemplateSandbox's API
integration (used in I83fa0856).
Bug: T112474
Change-Id: Ib9d2ce42931c1de8372e231314a1f672d7e2ac0e
|
|
|
|
|
|
|
|
|
|
| |
WikiModule::isKnownEmpty
If a module itself is empty, it must consider any dependencies
it has before bailing out as empty.
Bug: T191596
Change-Id: I2b45b948a6f78060e53513d3b4b77f48d7bf4a6b
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
* Document the structure of the in-process $titleInfo cache.
Specifically, specify that it is not the value from getTitleInfo(),
but rather a container for zero or more versions of such values.
The reason this is fragmented is because ResourceLoaderContext
is a parameter to most methods and as such, makes everything
variable. Tracked as T99107.
* Make various bits easier to understand by consistently refering
to the container keys as "batchKey", and referring to the internal
keys as "titleKey".
* Centralise title key logic by moving to private method.
* Replace the internal creation of titleKey to be based on LinkTarget
with plain namespace IDs and db keys, instead of invoking the
expensive getPrefixedTitle function which involves quite a lot
of overhead (TitleCodec, GenderCache, Database, Language,
LocalisationCache, ..).
Change-Id: I701e5156ef7815a0e36caefae5871524eff3f688
|
|
|
|
|
|
|
| |
This helps to find renamed or misspelled classes earlier.
Phan will check the class names
Change-Id: Ie541a7baae10ab6f5c13f95ac2ff6598b8f8950c
|
|
|
|
|
|
| |
Preparation change for updating mediawiki code sniffer to 0.8.0
Change-Id: Ib0b3fe4afea9096ffa3a1347b4f7e07d3398b0b2
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Replaces \TestingAccessWrapper (defined in core) with
\Wikimedia\TestingAccessWrapper (defined in the composer package
wikimedia/testing-access-wrapper).
See https://gerrit.wikimedia.org/r/#/q/topic:librarize-testing-access-wrapper
for downstream patches.
The core version of the class is kept around for a while to avoid
circular dependency problems.
Bug: T163434
Change-Id: I52cc257e593da3d6c3b01a909e554a950225aec8
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
preloadTitleInfo:
* Add missing case for empty $moduleNames.
* Add missing case for invalid page names.
getContent:
* Add missing case for bad title
* Add missing case for dead redirect.
* Add missing case for no content found.
Change-Id: I44dde13cb0db19d91c4ff15a5abefd17353cad90
|
|
|
|
| |
Change-Id: If7e8a8ff574661fd827de8bcec11d2c39a687300
|
|
|
|
|
| |
Bug: T108653
Change-Id: I9a321fc5728f3bf9df950c060c92e4148cf3ef93
|
|
|
|
|
|
|
|
|
|
| |
This is one of the top three DB queries showing up in xenon
reverse flamegraph profiling.
It works via a per-wiki check key that is bumped whenever
someone changes a .js or .css page on that wiki.
Change-Id: I73f419558864ba3403b4601a098f6aaf84a3e7c1
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Follows-up 6f8dc27ca2 and dbd11e04aa. You'd expect a bug in preloading
to fallback to fetching it on-demand but due to a title format mismatch
the preload method did use the same title format for the cache key, but
not the same title format for discovering the results. As such, it set
the right cache key to an empty array.
* Make relevant methods testable and mockable.
* Add regression test.
* Change call to array_interect_key to use the same format as
fetchTitleInfo(): Normalised title keys from Title::getPrefixedText.
Previously, the intersect used the declared titles from getPages() which
are not localised - causing an empty array to be returned from the
intersect on wikis where the namespace name is localised.
Bug: T145673
Change-Id: Ibe788157724d73c727b9e2127b6828db32ca9420
|
|
|
|
|
|
|
|
|
|
|
| |
While rev_sha1 was preferred to rev_id (page_latest) to allow
client-cache to be re-used in case of a bad change and revert,
this is no longer possible since we recently added support for
explicit purging by considering page_touched.
As such, use of rev_sha1 is no longer useful.
Change-Id: Iddb65305ca4655098fdea9fcf736fd4046035dd7
|
|
|
|
|
|
|
|
|
|
| |
No longer needed per doing Ic137cb494ba23 in a different way.
This may be useful to revisit, but for now preferring to keep
simplicity and removing this unused option.
This reverts commit 9e217bf42d73ef2dfcdecfe3a753cf8d702a18fd.
Change-Id: I9c0c316a3b58a3d0a3d3282dd74c7fa4eef8e378
|
|
|
|
|
|
|
| |
To be used for the 'site' module. This will make it easier to split up
'site' into 'site' and 'site.styles' (per T92459).
Change-Id: Iaac3e458d5107e4c10c2826bd64608d5c47e8b87
|
|
|
|
|
|
|
|
|
|
| |
Per wikitech-l consensus:
https://lists.wikimedia.org/pipermail/wikitech-l/2016-February/084821.html
Notes:
* Disabled CallTimePassByReference due to false positives (T127163)
Change-Id: I2c8ce713ce6600a0bb7bf67537c87044c7a45c4b
|
|
|
|
| |
Change-Id: I89d53c5051e5ee4bd8624df8ee2b25993090a7df
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Wiki modules are special due to their isKnownEmpty implementation and support
for foreign databases. MediaWiki doesn't have convenient ways of making
Revision objects for remote wikis. As such, wiki modules will keep using meta
data to generate the hash.
However minimise needless cache invalidation by refining the implementation.
Impact:
* Remove use of getMsgBlobMtime(). This module doesn't support getMessages().
* In the title info, use the revision content sha1 and size for tracking.
The page_touched previously used updates too often. It's updated both on edits
for various types of purges. Using the rev_sha1 means old versions return
when the content is the same. Regardless of how the content changed via
revert or actual edits resulting in the same contnet.
* Change in-process cache to be keyed by page list instead of entire
ResourceLoaderContext.
Because of this, getTitleInfo() was previously performing its batch query
twice on the same page. Once for only=styles (top) and only=scripts (bottom).
Both operate on the full getPages() set but had different context keys.
Clean up:
* Better document the support for foreign databases.
* Move Title construction to getContent to reduce duplication.
* Remove use of getDefinitionMtime(). That method is a no-op since the switch
to version hashing.
* Remove remaining use of mtime in getModifiedTime(). This is now covered by
hashing the title info in getDefinitionSummary().
Also refactor the code to be more readable. No intended change in behaviour.
Bug: T98087
Change-Id: Id46740db04c0c42bc5ca87d1487230a32feb34df
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Modules now track their version via getVersionHash() instead of getModifiedTime().
== Background ==
While some resources have observeable timestamps (e.g. files stored on disk),
many other resources do not. E.g. config variables, and module definitions.
For static file modules, one can e.g. revert one of more files in a module to a
previous version and not affect the max timestamp.
Wiki modules include pages only if they exist. The user module supports common.js
and skin.js. By default neither exists. If a user has both, and then the
less-recently modified one is deleted, the max-timestamp remains unchanged.
For client-side caching, batch requests use "Math.max" on the relevant timestamps.
Again, if a module changes but another module is more recent (e.g. out-of-order
deployment, or out-of-order discovery), the change would not result in a cache miss.
More scenarios can be found in the associated Phabricator tasks.
== Version hash ==
Previously we virtually mapped these variables to a timestamp by storing the current
time alongside a hash of the value in ObjectCache. Considering the number of
possible request contexts (wikis * modules * users * skins * languages) this doesn't
work well. It results in needless cache invalidation when the first time observation
is purged due to LRU algorithms. It also has other minor bugs leading to fewer
cache hits.
All modules automatically get the benefits of version hashing with this change.
The old getDefinitionMtime() and getHashMtime() have been replaced with dummies
that return 1. These functions are often called from getModifiedTime() in subclasses.
For backward-compatibility, their respective values (definition summary and hash)
are now included in getVersionHash directly.
As examples, the following modules have been updated to use getVersionHash directly.
Other modules still work fine and can be updated later.
* ResourceLoaderFileModule
* ResourceLoaderEditToolbarModule
* ResourceLoaderStartUpModule
* ResourceLoaderWikiModule
The presence of hashes in place of timestamps increases the startup module size on
a default MediaWiki install from 4.4k to 5.8k (after gzip and minification).
== ETag ==
Since timestamps are no longer tracked, we need a different way to implement caching
for cache proxies (e.g. Varnish) and web browsers. Previously we used the
Last-Modified header (in combination with Cache-Control and Expires).
Instead of Last-Modified (and If-Modified-Since), we use ETag (and If-None-Match).
Entity tags (new in HTTP/1.1) are much stricter than Last-Modified by default.
They instruct browsers to allow usage of partial Range requests. Since our responses
are dynamically generated, we need to use the Weak version of ETag.
While this sounds bad, it's no different than Last-Modified. As reassured by
RFC 2616 <http://www.w3.org/Protocols/rfc2616/rfc2616-sec13.html#sec13.3.3> the
specified behaviour behind Last-Modified follows the same "Weak" caching logic as
Entity tags. It's just that entity tags are capable of a stricter mode (whereas
Last-Modified is inherently weak).
== File cache ==
If $wgUseFileCache is enabled, ResourceLoader uses ResourceFileCache to cache
load.php responses. While the blind TTL handling (during the allowed expiry period)
is still maxage/timestamp based, tryRespondNotModified() now requires the caller to
know the expected ETag.
For this to work, the FileCache handling had to be moved from the top of
ResoureLoader::respond() to after the expected ETag is computed.
This also allows us to remove the duplicate tryRespondNotModified() handling since
that's is already handled by ResourceLoader::respond() meanwhile.
== Misc ==
* Remove redundant modifiedTime cache in ResourceLoaderFileModule.
* Change bugzilla references to Phabricator.
* Centralised inclusion of wgCacheEpoch using getDefinitionSummary. Previously this
logic was duplicated in each place the modified timestamp was used.
* It's easy to forget calling the parent class in getDefinitionSummary().
Previously this method only tracked 'class' by default. As such, various
extensions hardcoded that one value instead of calling the parent and extending
the array. To better prevent this in the future, getVersionHash() now asserts
that the '_cacheEpoch' property made it through.
* tests: Don't use getDefinitionSummary() as an API.
Fix ResourceLoaderWikiModuleTest to call getPages properly.
* In tests, the default timestamp used to be 1388534400000 (which is the unix time
of 20140101000000; the unit tests' CacheEpoch). The new version hash of these
modules is "XyCC+PSK", which is the base64 encoded prefix of the SHA1 digest of:
'{"_class":"ResourceLoaderTestModule","_cacheEpoch":"20140101000000"}'
* Add sha1.js library for client-side hash generation.
Compared various different implementations for code size (after minfication/gzip),
and speed (when used for short hexidecimal strings).
https://jsperf.com/sha1-implementations
- CryptoJS <https://code.google.com/p/crypto-js/#SHA-1> (min+gzip: 2.5k)
http://crypto-js.googlecode.com/svn/tags/3.1.2/build/rollups/sha1.js
Chrome: 45k, Firefox: 89k, Safari: 92k
- jsSHA <https://github.com/Caligatio/jsSHA>
https://github.com/Caligatio/jsSHA/blob/3c1d4f2e/src/sha1.js (min+gzip: 1.8k)
Chrome: 65k, Firefox: 53k, Safari: 69k
- phpjs-sha1 <https://github.com/kvz/phpjs> (RL min+gzip: 0.8k)
https://github.com/kvz/phpjs/blob/1eaab15d/functions/strings/sha1.js
Chrome: 200k, Firefox: 280k, Safari: 78k
Modern browsers implement the HTML5 Crypto API. However, this API is asynchronous,
only enabled when on HTTPS in Chromium, and is quite low-level. It requires boilerplate
code to actually use with TextEncoder, ArrayBuffer and Uint32Array. Due this being
needed in the module loader, we'd have to load the fallback regardless. Considering
this is not used in a critical path for performance, it's not worth shipping two
implementations for this optimisation.
May also resolve:
* T44094
* T90411
* T94810
Bug: T94074
Change-Id: Ibb292d2416839327d1807a66c78fd96dac0637d0
|
|
|
|
|
|
|
|
|
|
|
|
| |
constructor
This makes it easier for subclasses to use ResourceLoaderWikiModule. Currently
many subclasses of this simply need to override the getPages() method.
UserModule and SiteModule keep their getPages override due to the set of pages
being dependent on context.
Change-Id: I388531398671afacfec36c6c5746d72267b5bdac
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Follows-up b36d883.
By far most data providers are static (and PHPUnit expects them
to be static and calls them that way).
Most of these classes already had their data providers static
but additional commits sloppily introduced non-static ones.
* ResourceLoaderWikiModuleTest, 8968d8787f.
* TitleTest, 545f1d3a73.
Odd unused method 'dataTestIsValidMoveOperation' was introduced
in 550b878e63.
* GlobalVarConfigTest, a3e18c3670.
Change-Id: I5da99f7cd3da68c550ae507ffe1f725d31e7666f
|
|
In most cases, we just check whether the pages exist before saying
the module is not empty to avoid generating cached HTML without
the appropriate <script> or <link> tags.
However, for modules in the 'user' group, normal users cannot
delete their personal JavaScript/CSS pages, causing needless
extra requests, even though we know the pages are empty.
ResourceLoader::isKnownEmpty() now checks the page_len field
for modules in the 'user' group to check that there is
some actual content.
Bug: 68488
Change-Id: I0570f62887fd4642fd60367ae0b51d7dc19488ca
|