| Commit message (Collapse) | Author | Age | Files | Lines |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Update a few remnant references inside the library while at it.
* WANObjectCache: Didn't use ATTR/QOS directly, and getQoS test
already implemnented as BagOStuff const.
Update getQoS() docs to match.
* WANObjectCache: Already documented ERR_ as BagOStuff::ERR_.
And Codesearch confirms no use of `WANObjectCache::(ERR|ATTR|QOS)`.
* MemcachedClient: Switch from referencing one to the other.
The class is not directly used outside core, only via
BagOStuff.
* Implicitly remove unused ATTR_EMULATION by not carrying over.
Follows-up:
* e8275758fe (I20fde9fa5c) Split IExpiringStore from BagOStuff,
and re-use in WANObjectCache.
* 74be3a0150 (I4377fc3f53) Move ERR_ from BagOStuff to IExpiringStore.
* 69950da666 (Ia862c5111a) Replace IExpiringStore with StorageAwareness.
* 59b002b866 (I9885f53f00) Remove StorageAwareness::QOS_LOCALITY_.
* ec90b543ab (I8dec3f73fa) Remove StorageAwareness::QOS_EMULATION_SQL.
* 62bdd78817 (I5649a29310) Adopt ERR_ in MemcachedClient.
* e5a3e36bd1 (I836735b1fe) Mark StorageAwareness as internal.
Bug: T353529
Bug: T364652
Change-Id: I1dfde995e29d5264611cf3500d61fe4d8631a7d7
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
I assume these are all either auto-generated by an IDE or the
language-level type declarations have been added later. In any case
the comments don't add any new information to what the code already
says. This is just extra clutter that makes the code harder to read,
I would argue.
There are many, many more comments like this. In this patch I
intentionally focus on the most trivial 1-line comments.
Change-Id: Ia294bf4ce0d8a77036842fe25884bc175c2b0e7d
|
|
|
|
| |
Change-Id: I27b9a19ab952ede1267921bd042af0fe1c89e228
|
|
|
|
|
| |
Bug: T353458
Change-Id: I3b736346550953e3b2977c14dc3eb10edc07cf97
|
|
|
|
|
|
|
|
|
|
| |
A single test function can have multiple @dataProviders. No problem.
A @dataProvider can also be used by multiple test functions. No
problem. No need to duplicate such large pieces of code when it's
identical anyway.
Change-Id: I5aea36304ec2d1666ff2334ba883df07a70c15c5
|
|
|
|
|
|
|
|
|
| |
And deprecated aliases for the the no namespaced classes.
ReplicatedBagOStuff that already is deprecated isn't moved.
Bug: T353458
Change-Id: Ie01962517e5b53e59b9721e9996d4f1ea95abb51
|
|
|
|
| |
Change-Id: I9c64a631b0b4e8e4fef8a72ee0f749d35f918052
|
|
|
|
| |
Change-Id: I5629f91387f2ac453ee4341bfe4bba310bd52f03
|
|
|
|
|
| |
Bug: T353180
Change-Id: Iac8e6fab0f6a95ba3534ab0ce40b91d0967f74c0
|
|
|
|
|
|
| |
Most noisily, this enables MediaWiki.Arrays.OneSpaceInlineArray.
Change-Id: I8ab11399c67ce7e3ab1b6249b591452774393428
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
When getWithSetCallback() is called with check keys, if the keys are
missing, a check key is inserted with the current time, as if
touchCheckKey() were called. This causes cache misses for
HOLDOFF_TTL = 11 seconds. This seems unnecessary since in the case of
an empty cache, there is no expectation of replication delay.
However, it's reasonable for it to be a cache miss when the check key is
missing, and a cache hit subsequently, so we do need to add a purge
value.
So, in getWithSetCallback(), set the holdoff to zero when inserting a
purge value.
Also, use a holdoff of zero when initialising a missing touch key in
getCheckKeyTime().
Bug: T344191
Change-Id: Ib3ae4b963816e5b090e87e4cb93624afefbf8058
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
== Background
Most of this was introduced in commit 5c335f9d77 (I1eb897c2cea3f5b7).
The original motivation was:
* Ensure wrappers like MultiWriteBagOStuff naturally do the right
thing. In practice, makeKey() results are interchangeable, with
the most contrained one (Memcached) also generally used as the first
tier. However, this is not intuitive and may change in the future.
To make it more intuitive, the default implemention became known
as "generic", with proxyCall() responsible for decoding these,
and then re-encoding them with makeKey() from the respective
underlying BagOStuff. This meant that MultiWriteBag would no longer
use the result of the Memcached-formatted cache key and pass it
to SqlBagOStuff.
* Allow extraction of the key group from a given key cache,
for use in statistics.
Both motivations remains valid and addressed after this refactor.
== Change
* Remove boilerplate and indirection around makeKey from a dozen
classes. E.g. copy-paste stubs for makeKey, makeKeyInternal, and
convertGenericKey.
Instead, let BagOStuff::makeKey and ::makeKeyInternal hold the
defaults. I believe this makes the logic easier to find, understand,
and refer to.
The three non-default implementations (Memcached, WinCache, Sql)
now naturally reflect what they are in terms of business logic,
they are a method override.
Introduce a single boolean requireConvertGenericKey() to let the
three non-default implementations signal their need to convert
keys before use.
* Further improve internal consistently of BagOStuff::makeKeyInternal.
The logic of genericKeyFromComponents() was moved up into
BagOStuff::makeKeyInternal. As a result of caling this directly
from BagOStuff::makeKey(), this code now sees $keyspace and $components
as separate arguments. To keep the behaviour the same, we would
have to either unshift $keyspace into $components, or duplicate
the strtr() call to escape it.
Instead, excempt keyspace from escaping. This matches how the most
commonly used BagOStuff implementations (MemcachedBag, and SqlBag)
already worked for 10+ years, thus this does not introduce any new
responsibility on callers. In particular, keyspace (not key group)
is set by MediaWiki core in service wiring to the wiki ID, and so
is not the concern of individual callers anyway.
* Docs: Explain in proxyCall() why this indirection and complexity
exists. It lets wrapping classes decode and re-encode keys.
* Docs: Explain the cross-wiki and local-wiki semantics of makeKey
and makeKeyGlobal, and centralise this and other important docs
about this method in the place with the most eye balls where it is
most likely seen and discovered, namely BagOStuff::makeKey.
Remove partial docs from other places in favour of references to this one.
Previously, there was no particular reason to follow `@see IStoreKeyEncoder`
much less to know that it holds critical that communicate the
responsibility to limit the key group to 48 chars.
* Docs: Consistently refer to the first component as the "key group",
thus unifying what was known as "key class", "collection",
"key collection name", or "collection name".
The term "key group" seems to be what is used by developers in
conversations for this concept, matching WMF on-boarding docs and
WMF's Grafana dashboard for WANObjectCache.
Change-Id: I6b3167cac824d8bd8773bc66c386f41e4d380021
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
This was introduced for use by WANObjectCache, however, this never
worked in practice.
The feature only influences determineKeyPrefixForStats(), which in
turn is only called by MediumSpecificBagOStuff::updateOpStats, which
in turn is exclusively called within BagOStuff subclasses that aren't
used with WANObjectCache (RESTBag, RedisBag, SqlBag).
The next commit simplifies BagOStuff::makeKey, which can make a bigger
impact if this feature doesn't need to be supported.
During drafting of the next commit, I found that the feature doesn't
appear to work well in practice, because getCollectionFromSisterKey
did not account for mcrouter-style route prefixes, as inserted by
makeSisterKey().
Change-Id: Iecea959c88abbd7a0a17b92c1c4d854602748280
|
|
|
|
|
|
|
|
| |
When the value of the associated array is not used, there is no need to
use array_keys and create internal a copy of the array, just omit the
value with $_ to tell static analyzer that the var is unused.
Change-Id: I5c6d667e98c0167c0573c683f0d617bc1d4ca619
|
|
|
|
|
|
|
|
|
| |
Initally used a new sniff with autofix (T333745),
but some provide are defined non-static in TestBase class
and need more work to make them static in a compatible way
Bug: T332865
Change-Id: I889d33424f0c01fb26f2d86f8d4fc3de3e568843
|
|
|
|
|
|
|
| |
QOS_DURABILITY_RDBMS can be checked instead
Bug: T279977
Change-Id: I42a00275995dd855c60c8f45a01d2da11bab7029
|
|
|
|
|
|
|
|
|
|
|
|
| |
* Triple backslash in regex should really be quadruple backslash
* Using the returned value of a void method
* Immediately overwritten array keys
* Duplicate array keys
* Foreach variable reuse
* sprintf() with too many params
* Incorrect reference usage
Change-Id: I3c649b543c9561a1614058c50f3847f663ff04df
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
I noticed this in CI during development for Ia893ddb36427eb5e9bff,
however, it is reproducible on latest master as well. Locally, it
happens about 1/30 times:
```
$ php phpunit.php --filter testLockTSESlow unit/includes/libs/objectcache/
Using PHP 8.1.13
PHPUnit 9.5.27 by Sebastian Bergmann and contributors.
There was 1 failure:
1) WANObjectCacheTest::testLockTSESlow with data set #0 (array('hash_tag'), '{')
Callback not used (mutex not acquired, stale value used)
Failed asserting that 3 is identical to 2.
/w/tests/phpunit/unit/includes/libs/objectcache/WANObjectCacheTest.php:1311
/w/tests/phpunit/MediaWikiUnitTestCase.php:11
```
* Remove randomisation from test. The HashBag is empty at this point,
and a dedicated instance is created for each test case. Remove doubt
over conflicts or other uncertainty, and make it easier to debug at
the same time.
* Increase mocked time gap from 2s to 32s similar to what the
previous assertion in the same test does already. 2s does not actually
go past the reduced logical TTL which is 30s at this point.
It seems to still pass on master in most cases, but it makes sense
that it can fail. I could not reproduce failures after this change.
Change-Id: I6e491ae3c4f41413e7a952aa0796a787d559ac48
|
|
|
|
|
|
|
|
|
| |
Loosen up the code coverage a bit so that we can more clearly identity
what logic truly isn't excercised by tests. It also restores some
gaps that were left behind by various refactors when code moved around
or out into various private methods.
Change-Id: I4e65f92ea1eb3f0e781ada4fb2ced0f8ef9d3cc4
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
This avoids the latency of waiting for a response.
Change yieldStampedeLock() to use delete() instead of changeTTL() since
the PECL memcached client does not support WRITE_BACKGROUND with TOUCH.
Also, simply relayVolatilePurges() into relayVolatilePurge() and tweak
some related documentation for WANObjectCache
Bug: T302623
Change-Id: I16f011b730f34332aa54a48c579de3c6606dafda
|
|
|
|
|
|
|
|
| |
This allows for large string value storage similar to that
of direct BagOStuff callers.
Bug: T309126
Change-Id: I22d2d8bb520ef2ea6ccd28016d34db36fe27c3df
|
|
|
|
|
|
|
|
|
|
| |
Make set() and getWithSetCallback() wrap setMainValue(), avoiding some
duplicate option array resolution logic.
Track set() calls via statsd, now that it can be distinguished from
getWithSetCallback() invocations.
Change-Id: Ia1e7b244b1db3b76314d718b3aea9e6e708235c0
|
|
|
|
|
|
|
|
|
|
| |
Introduced in 2017 with I7f14b9ca2533032 (2e5eb693) but remains
unused at WMF, and disabled by default.
Follows-up I62107789fa (9e49260fc958) which added reap to LinkCache
test cases in 2021.
Change-Id: I0654c29a671467dd6b366f462d1c09b90a273413
|
|
|
|
|
|
|
|
| |
Introduced in PHP 7.1. Because it's shorter and looks nice.
I used regex replacement.
Change-Id: I0555e199d126cd44501f859cb4589f8bd49694da
|
|
|
|
|
|
|
|
|
|
|
| |
Follows-up Ia550ef7fe3 (4d3549ad71) which added reap to NameTableStore
in 2018, which was then (mostly) removed again with I63e61e1ab (2c7e4adcea)
for T198561, except one call was left behind in reloadMap(). I've
changed this to match the existing delete call in acquireId().
To be removed in I0654c29a671467dd6.
Change-Id: I7541c9c6d1ef70d552944aca94cbc4d0f7ec0d5b
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
And also assertNotRegExp -> assertDoesNotMatchRegularExpression. The
methods were renamed in PHPUnit 9.
Done automatically with:
grep -rl assertRegExp tests/ | xargs sed -r -i "s/>assertRegExp\(/>assertMatchesRegularExpression\(/"
grep -rl assertNotRegExp tests/ | xargs sed -r -i "s/>assertNotRegExp\(/>assertDoesNotMatchRegularExpression\(/"
Split out from Ifdba0f9e98eb6bce4590b7eb73170c51a697d7c6 so that it
remains smaller and easier to review.
Also make a test use MediaWikiUnitTestCase (it's already in the unit/
dir) so that it can access the forward-compat method.
Bug: T243600
Change-Id: Ifa279d5f201d7abeebece292141ebface8278046
|
|
|
|
|
| |
Bug: T201491
Change-Id: I4fc428c77bd2f094d194bcd5aaa3636449ef2658
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
It took me a good half hour to understand what these tests are doing,
and another half hour to build up a modicum of confidence that the
current behaviour is the indeed the desired behaviour.
* Remove random strings.
* Remove indirection and computed expected values as much as possible
in favour of inlined literals that explicitly expect outcomes.
* Remove confusing second argument from the data provider
getMultiWithUnionSetCallback_provider, which wasn't used, and
add labels to it.
* Remove some of the existing inline comments.
* Add some inline comments with what I've understood.
* Fix bug in "C: Repeat case with recently created check keys",
which claimed to be observing that caches were ignored due to the
check key being too new, when in fact it was a cache miss simply
because the key didn't exist yet upto this point. Make it actually
repeating by adding a second call afterwards to observe the
stated behaviour.
Bug: T303092
Change-Id: I874a20ecd49464f7b0e6a0b579c000dc0e804c0a
|
|
|
|
|
|
|
|
| |
The examples and comments were misleading with regard to the handling
of missing entities.
Bug: T303092
Change-Id: Ib672fd18d1270c35253683eb54b48be056e6dc3c
|
|
|
|
| |
Change-Id: I76feaa559e2e55ca11f49df3df63534a34127f2c
|
|
|
|
|
|
|
|
|
|
| |
Tweak testLockTSESlow() tests to reduce mt_rand() dependency.
Also, rename last few $now variables to $mockWallClock and use
integral floats to make it easier to find raw microtime() values
via debugging statements.
Change-Id: Ie2b125e42d3ac37104f5b87b75a9a81558956518
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Handle edge case of fast callbacks that happen to run just before
the transaction age reaches MAX_READ_LAG. Such callbacks will no
longer be considered "slow".
Distinguish slow and fast regenerations for the case when TTL
mitigation is triggered by the combination of snapshot and
replication lag.
Reduce the cases where TTL_UNCACHEABLE is used as the mitigation
TTL, to avoid operational risk.
Simplify the mitigation TTL code by reducing needless variation
between the "lockTSE" and non-"lockTSE" cases.
Track the key and time of the last 10 get() calls that returned
either a stale value or no value at all. Use this this estimate
the walltime in set(), if it was not provided.
Bug: T285070
Change-Id: Ie6c88fbf7dbfc831d67bfb6cde082c123555a8cf
|
|
|
|
|
|
|
| |
In prep for the next commit and to confirm that it passes
both before and after.
Change-Id: I4d203843a8cb2486584df1601fec589389110f73
|
|
|
|
| |
Change-Id: Iad4d1dc4727828f9e0120c8bf99dc245a29ee14d
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
… including PHPDoc tags like `@return <type> $variableName`.
A return value doesn't have a variable name. I can see that
some people do this intentionally, repeating the variable
name that was used in the final `return $var;` at the end
of a method. This can indeed be helpful. I leave a lot of
these untouched and removed them only when it's obviously
wrong, or does not provide any additional information in
addition to what the code already says.
Change-Id: Ia18cd9f25ef658b08ad25b97a744897e2a8deffc
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
It is not entirely meaningless. It might be an indicator that
the number of calls to a method is intentionally unlimited.
This is similar to e.g. an @inheritDoc PHPDoc comment that
marks a method as being "intentionally undocumented".
However, what's the meaning of being "intentionally
unconstrained"? Let's just not have any constraint then.
I feel all these ->expects( $this->any() ) bloat the test
code so much that it's never worth it.
Change-Id: I9925e7706bd03e1666f6eb0b284cb42b0dd3be23
|
|
|
|
|
|
|
| |
It's shorter, the more canonical form, and easier to read
because of this.
Change-Id: I9c68a49c1f31bddfac91a74a6592fe3545ca00a7
|
|
|
|
|
|
|
|
|
|
|
|
| |
Ended up using
grep -Prl '\->setMethods\(' . | xargs sed -r -i 's/setMethods\(/onlyMethods\(/g'
special-casing setMethods( null ) -> onlyMethods( [] )
and then manual fix of failing test (from PS2 onwards).
Bug: T278010
Change-Id: I012dca7ae774bb430c1c44d50991ba0b633353f1
|
|
|
|
|
|
|
|
|
| |
Also remove the unused 'region' option.
Added release notes, and updated the notes for coalesceKeys as well
to reside in the configuration section instead.
Change-Id: I219ea67416c6d121ce1488d99dab9a3a089f27d9
|
|
|
|
|
|
|
|
|
|
|
|
| |
Renamed several methods for clarity and updated various comments
Use explicit number_format() instead of casting floats to strings
when making purge values
Fix LocalRepoTest mocking of WANObjectCache::makeGlobalKey()
Bug: T264604
Change-Id: Ib03b97fcab8a34d38d3a8a12da6e28e12feef4ad
|
|
|
|
|
|
|
|
|
|
| |
* Split makePurgeValue() into one for "check" keys and one for
tombstones.
* Simplify tombstones by ommitting the zeroed-out holdoff component.
* Remove unused and deprecated HOLDOFF_NONE constant.
* Clarify various comments.
Change-Id: If6cc34538b1ea30764836b6870ba8648424ce9d7
|
|
|
|
|
|
|
|
| |
Key coalescing hints are now unconditionally enabled, using
the mcrouter style by default. This can be changed by setting
"coalesceScheme" (e.g. in case of experimental dynomite use).
Change-Id: I4866ed66d42d7d7da2b3edc2f99b8ba35fe3d511
|
|\ |
|
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| | |
This reverts commit 4e94fa8812d229bd93e4329647a6f45c88bacc67.
The extra serialization call here is not needed since storing
a value naturally results in serialization and thus exception
throwing already (hence the fatal error at T273242).
Performing serialization twice comes with a significant cost,
which has in past led to notable regressions.
Adding a try-catch makes sense, but this can be placed at a
higher level instead, as is already done by Ibfcbf0eceb4b7a36
which was already merged and back-ported, making this obsolete.
Other use cases:
- Continuous integration where HashBag is used, which doesn't
need serialize and thus doesn't cover the Closure issue.
Cost of serialize was removed there for performance reasons,
but if we want to add it back, let's do so there in the natural
place where it once was (or in MediumBag, which might be preferable
so that we can use that same code path to gradually switch to JSON
encoding and warn/throw for non-jsonic values).
- Regular getWithSet calls (rather than the async refresh from which
the fatal is currently seen). For regular getWithSet calls we already
get a useful stack trace that points directly to the line of code
where the value and cache key are computed, so additional logging
does not seem that valuable. But for consistency we could move the
try-catch from Ibfcbf0eceb4b7a36e33a8 a little lower to cover all
fetchOrRegenerate() calls, not just those from scheduleAsyncRefresh.
Bug: T273242
Change-Id: Ic649016dde7b58323a028ce6d4fe617ef62a1ff8
|
|\| |
|
| |
| |
| |
| |
| |
| |
| | |
Alternative to Ia91d9f8533d4cc58564473ee9a375ba6be90eb64
Bug: T273242
Change-Id: I76a6667a1fcdc35ff972ebcdf1ad0b95ef07e015
|
|/
|
|
|
|
|
| |
Result of a new sniff I25a17fb22b6b669e817317a0f45051ae9c608208
Bug: T274036
Change-Id: I695873737167a75f0d94901fa40383a33984ca55
|
|
|
|
|
| |
Bug: T274036
Change-Id: I911d3041cebe417d162934223b46ea295c6d20e3
|
|
|
|
| |
Change-Id: Idb413be4b8cba8611afdc022af59810ce1a4531e
|
|\ |
|