aboutsummaryrefslogtreecommitdiffstats
path: root/includes/objectcache
Commit message (Collapse)AuthorAgeFilesLines
* objectcache: Re-number array keys in SqlBagOStuffAmir Sarabadani2025-03-181-1/+1
| | | | | | | | | | | | | | | | | Currently, if only one server has responded with the key, we take the first value of the array but the key here is actually the shard index of the server and can be something like this [ 5 => $value ]. This results in an null value being used and causing exceptions down the line. We can simply renumber it to [ 0 => $value ] by calling array_values() Tested in mwdebug2001 and this fixes the issue CI tests didn't catch this because we test with a couple of sections only so shard index ends up being zero anyway. Bug: T389169 Change-Id: I91a6eab88f28504ef14c8aa3f39bc923ba13e8a7
* objectcache: Introduce dataRedundancy to SqlBagOStuffAmir Sarabadani2025-03-111-37/+92
| | | | | | | | | | | | | | | | | | | | | | | | | | | | This forces SqlBagOStuff to instead of sharding keys, write to n out of m servers instead. It also reads from those servers as well and in case of incosistency, picks the value with the highest exptime. This is mostly for mainstash and allows us to provide stronger consistency guarantees while allowing for a section to be depooled and put to maintenance. It basically implements the logic already used by NoSQL database systems such as Cassandra (There are two types to solve conflict, quorum or timestamp, Cassandra is using quorum while we are using timestamp). There will be some edge cases that it might still pick the wrong value: - if TTL is set to INDEF - if the TTL gets shortened for various reasons. - If we go with two clusters, value is set, one gets depooled, a new value is set, the depooled one gets pooled and the other depooled and then read happens. But all of these are extremely rare edge cases and we should be fine. This also means if data redundancy is set, locking means all sections will be locked and removal means all sections must allow the unblock. Otherwise, the lock will be kept. Bug: T383327 Change-Id: I80da12396858ee4fc58ae257f6c154b3050df696
* Replace call_user_func with dynamic function callUmherirrender2025-02-131-1/+1
| | | | | | | | Use modern php syntax to call a callable. Reduce the stack trace to improve performance and better IDE and static analyzer support Change-Id: I9ef131032a662a3b8db69aa7079dbd51f88f575a
* Merge "BagOStuff: Update makeKeyInternal to accept/handle null"jenkins-bot2025-01-281-5/+9
|\
| * BagOStuff: Update makeKeyInternal to accept/handle nullReedy2025-01-281-5/+9
| | | | | | | | | | | | | | | | Minor doc fixes Bug: T384858 Follows-Up: I358a0356a55dc5f1349c240c41ec406de85942ee Change-Id: Ie62624793d3b2956b821517bb0d31d19e2d5e7c2
* | objectcache: Require a TracerInterface in ObjectCacheFactoryMáté Szabó2025-01-281-3/+3
|/ | | | | | | | | | | | | | | | | | Why: - Ief4258ef6d46fb237837907c6ecb96bb3afe6d81 has implemented tracing support for ObjectCacheFactory via an injected TracerInterface instance. - This new parameter is typehinted as TelemetryHeadersInterface, which TracerInterface extends, but in practice always receives a TracerInterface implementation. What: - Make it require TracerInterface. - Add a test. Bug: T340552 Change-Id: I653a48ab9d904816ae062896454e7c3193e94fa4
* ObjectCacheFactory: use Tracer telemetryChris Danis2025-01-271-10/+6
| | | | | Bug: T340552 Change-Id: Ief4258ef6d46fb237837907c6ecb96bb3afe6d81
* Remove 2-line PHPDocs that just repeat the types from the codethiemowmde2025-01-171-2/+0
| | | | | | | | | | | | | | | | Same as Ia294bf4 did for 1-line comments. This patch removes slightly more complex 2-line PHPDoc comments that don't add any new information to the code, but literally repeat what the code already says. They say "don't document the code, code the documentation", and we are doing this more and more. We just tend to forget to remove the obsolete comments. Note I'm also removing a line of text in a few cases when it's very short and literally says the same as the method name. Again, such comments add zero new information. Change-Id: I01535404bab458c6c47e48e5456403b7a64198ed
* objectcache: Automatically fallback to the second db in lineAmir Sarabadani2025-01-061-12/+35
| | | | | | | | | | | | | | | | | | | | | | ...when the first attempt causes db error. We currently have this problem that if a parsercache host goes down, until it's replaced or fixed, all 1/nth pageviews trigger a reparse (unless cached in memcached). With a rather simple change, if db error is returned, simply fallback to the second database in line, that way in case a pc host goes down, we will have a graceful degradation and move the load to the rest of hosts automatically. It's similar to ring designs. This helps us re-architecture parsercache in favor of getting rid of idle spares because we wouldn't need an immediate fix in case of issues. It also can be used to make x2 much more robust (we can set up x4 and they would automatically fail over in case of issues). Bug: T373037 Change-Id: I95424f453396fbcdd399e588a4b6638ecb572441
* Use explicit nullable type on parameter argumentsUmherirrender2024-10-161-3/+3
| | | | | | | | | | | Implicitly marking parameter $... as nullable is deprecated in php8.4, the explicit nullable type must be used instead Created with autofix from Ide15839e98a6229c22584d1c1c88c690982e1d7a Break one long line in SpecialPage.php Bug: T376276 Change-Id: I807257b2ba1ab2744ab74d9572c9c3d3ac2a968e
* Merge "objectcache: Add support for sister keys in SqlBagOStuff"jenkins-bot2024-10-121-0/+5
|\
| * objectcache: Add support for sister keys in SqlBagOStuffAmir Sarabadani2024-10-091-0/+5
| | | | | | | | | | | | | | | | | | | | | | | | This would allow us to force parsercache idhash and idoptions keys next to each other so when e.g. depooling 1 of 4 hosts in a parser cache cluster, the amount of cache misses due to rehashing will be ~25% instead of ~%50. This is similar to the implementation in WANCache and the same structure has been used to make it consistent. Bug: T373037 Change-Id: I920fe76e45298aeee6acf725324a5e1ed2b57a37
* | Remove meaningless @var documentation from constantsthiemowmde2024-10-091-1/+0
|/ | | | | | | | | A constant is not a variable. The type is hard-coded via the value and can never change. While the extra @var probably doesn't hurt much, it's redundant and error-prone and can't provide any additional information. Change-Id: Iee1f36a1905d9b9c6b26d0684b7848571f0c1733
* Merge "objectcache,profiler,externalstore: Simpler mt_rand() for 1 in N chance"jenkins-bot2024-09-251-2/+2
|\
| * objectcache,profiler,externalstore: Simpler mt_rand() for 1 in N chanceTimo Tijhof2024-09-251-2/+2
| | | | | | | | | | | | | | Based on https://gerrit.wikimedia.org/r/1057269, as confirmed by https://3v4l.org/XW30U. Change-Id: I5dc2e140a213bf59f48e39c333280f9662979964
* | Merge "objectcache: Add regression test for MultiWrite dependency injection"jenkins-bot2024-09-241-1/+1
|\ \ | |/ |/|
| * objectcache: Add regression test for MultiWrite dependency injectionTimo Tijhof2024-09-231-1/+1
| | | | | | | | | | | | | | | | | | Follows-up 4e596f5112 (I59266726ad), which fixed bug T318272, but did not add a regression test for it. Bug: T318272 Bug: T327158 Change-Id: Ia8af6671887d3914fdc761d8e5d10fd33fb40f88
* | Remove unchecked exception annotationsAdam Wight2024-09-171-1/+0
|/ | | | | | | | | | | | | | | Callers should not catch an unchecked exception, so it doesn't belong in a function signature. Unchecked exceptions indicate a coding error, which by definition the code will not be able to handle correctly. If any of these exceptions were supposed to be in response to an edge case, user input, or initial conditions, then they should be changed to a runtime error. If the exception class cannot be changed, then the annotation should include a comment explaining its purpose and prognosis. Bug: T240672 Change-Id: I2e640b9737cb68090a8e1cb70067d1b74037d647
* objectcache: Remove WinCache supportMáté Szabó2024-09-051-3/+3
| | | | | | | | | | | | WinCache is an APCu equivalent for use with Microsoft IIS, but in recent years has been unmaintained and lacks support for PHP 8 and newer.[1] So, remove support for it as MediaWiki will be raising the minimum supported PHP version to 8.1. [1] https://www.php.net/manual/en/install.windows.recommended.php Bug: T365691 Change-Id: I4d2dc01a9119bb1f858132f0146b894750c1e86d
* objectcache: Remove ReplicatedBagOStuff, deprecated since 1.42Ebrahim Byagowi2024-09-041-5/+1
| | | | | Bug: T352481 Change-Id: I4e1ee5680b7ba0207dfe30a1208db35eca07e218
* objectcache: make SqlBagOStuff correctly ignore DBO_DEFAULTAaron Schulz2024-08-071-3/+3
| | | | | | | | Generally, configuration has 0 or DBO_DEFAULT, not DBO_TRX. Also, clearing DBO_TRX does not stop it from getting set again due to DBO_DEFAULT in DatabaseFactory::create(). Change-Id: Ie958475c9706199bd4f474f55565c8a9a32f1291
* objectcache: Update tests that are really for ObjectCacheFactoryDerick Alangi2024-07-241-1/+1
| | | | | | | | | | Most of the logic in ObjectCache.php has been refactored to a proper factory in ObjectCacheFactory and exposed as a service. These tests now are really for ObjectCacheFactory, hence the rename. See: I3179a387486377c6a575d173 (d372626b97e83ec1f82b88e75fbaadf). Change-Id: I7c2c20091490065ec93cccdb8cdb311b398d21ba
* Add namespace to the root classes of ObjectCacheEbrahim Byagowi2024-07-103-0/+10
| | | | | | | | | And deprecated aliases for the the no namespaced classes. ReplicatedBagOStuff that already is deprecated isn't moved. Bug: T353458 Change-Id: Ie01962517e5b53e59b9721e9996d4f1ea95abb51
* Use namespaced classes (1)Umherirrender2024-06-161-0/+1
| | | | | | | Changes to the use statements done automatically via script Addition of missing use statement done manually Change-Id: Ic4d4dd61de5ab896fb6173eb579c81f164a1e4a3
* rdbms: Remove ILoadBalancer::getWriterIndex()Amir Sarabadani2024-06-031-1/+2
| | | | | | | | It doesn't need to have its own method, We can just use the constant instead. Bug: T363839 Change-Id: Iaec5a8e88dc3e5ae4eaf1f24aebf4c5d73f4b350
* Use RawSQLValue for some SET clauses in upsertUmherirrender2024-05-291-3/+4
| | | | | | | | Need a check for RawSQLValue in Database::upsert Also check for all int keys, as this must not be an real list starting as index 0 Change-Id: If525243154072ebe81b5ecce2da92e5ccf773ab9
* objectcache: Remove "for b/c" from internal ObjectCacheFactory commentTimo Tijhof2024-05-231-1/+1
| | | | | | | | | Follows-up 7475063bfde7. We don't usually keep back-compat between a subject and its own tests. It looks like the tests don't have an alternative way to set or test this, so perhaps this is not for back-compat but rather for internal usage in tests. Change-Id: I4d212433326592589a45d71a081c4d01377e689b
* objectcache: Complete refactor of `ObjectCache.php`Derick Alangi2024-05-212-137/+148
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | This patch completes the rest of the ObjectCache refactor and migrates methods to the appropriate class while deprecating them in `ObjectCache.php`. It also moves the `_LocalClusterCache` internal service logic into ObjectCacheFactory and calls that instead making sure that wiring code stays wiring code and let the class do the heavy lifting. `::makeLocalServerCache()` is retained as a static method in the ObjectCacheFactory class because it's called early in Setup.php before the services container is available (so it needs to be stand- alone). To add, we also converts all global variables that were used in the `ObjectCache.php` class into the config schema approach and retrieves them using ServiceOptions injected in service wiring. NOTE: MediaWikiIntegrationTestCase::setMainCache() was slightly rewritten to take care of service reset which throws away the cache object preserved by setInstanceForTesting() after service reset. Instead, we preserve the object via MainConfigNames::ObjectCaches setting with a factory closure which returns the correct cache object. As a nice side effect of the above, the setInstanceForTesting() method was removed entirely. As a follow-up to this patch, I would like to remove the internal _LocalClusterCache service in a stand-alone patch. Bug: T363770 Change-Id: Ia2b689243980dbac37ee3bcfcbdf0683f9e1779b
* rdbms: Remove IDatabase::getTopologyBasedServerId()Amir Sarabadani2024-05-141-1/+1
| | | | | | | | Looking at all implementations, it only returns null or getServerId(), which we already rely on in the only caller SqlBagOStuff. Bug: T363839 Change-Id: I680e82d6d36548cd6bc351ab1d1fba48a827cbf3
* Merge "fix: use objectcachefactory methods instead of deprecated objectcache ↵jenkins-bot2024-05-051-1/+1
|\ | | | | | | methods"
| * fix: use objectcachefactory methods instead of deprecated objectcache methodsIrina Balaban2024-05-051-1/+1
| | | | | | | | | | Bug: T363770 Change-Id: Ie732f6925ec2b1316a60bebbe3c27f963c9dacb1
* | Formally deprecate code marked with @deprecatedJames D. Forrester2024-05-031-2/+2
|/ | | | | | | | | | | | | | | Some of these have been marked in-code as deprecated for a long while, but haven't ever been announced in the RELEASE-NOTES (and later, HISTORY) file, so let's mark them up now so we can get the ball rolling at least. Per Gergo, the AuthManager one was 'born deprecated' and should only have been used by humans also reading the deprecation notice given in the code, and indeed no uses are known to code search, so also emit deprecation warnings there immediately; others will have to wait until uses have been migrated. Change-Id: I0c1c71d8f4293623039302da35d58d2a24367e97
* objectcache: Inject DBLoadBalancerFactory into ObjectCacheFactoryDerick Alangi2024-04-182-48/+52
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | This patch does the follow with a few gotchas: * Properly inject LBFactory service into ObjectCacheFactory to be used where needed. * doc: Pull in relevant documentation from ObjectCache class into the ObjectCacheFactory class and also update docs in the OCF class to be more accurate with the code. * This patch also resolves an issue that caused an infinite loop in SqlBagOStuff making connections to the DB not to be reused within a request there by crashing the application (when the index.php entry-point) is accessed directly and every cache type is set to CACHE_ANYTHING. Meaning in LocalSettings, only `$wgMainCacheType` is set and its relatives (ParserCache, MessageCache, etc) aren't set. NOTES ===== -> A circular dependency would occur in OCF when injecting LBFactory: DBLoadBalancerFactory->DBLoadBalancerFactoryConfigBuilder ->LocalServerObjectCache->ObjectCacheFactory->DBLoadBalancerFactory directly, so in order to resolve this, we'll inject a closure instead and call that to retrieve the service when needed. The solution above is already used in other services today in the code. As an example, you can see SignatureValidatorFactory. -> In MainConfigSchema.php, the CACHE_ANYTHING key got removed in https://gerrit.wikimedia.org/r/c/mediawiki/core/+/955771 and this is a change in behavior. So we need to recompute the value of CACHE_ANYTHING's ID via service wiring for DB operations. Test plan ========= This patch is fairly straight forward, all it does is do some DI of a service into OCF (via a callable). No change in behavior should be expected in your local wiki. So if your local wiki is still running smoothly when you hit the `/index.php` entry-point directly and other cases, then everything should be working fine. Bug: T362686 Change-Id: I305ef0c377a023236b8ed9a101762813f32e6cd0
* objectcache: Restore default keyspace for LocalServerCache serviceDerick Alangi2024-03-282-10/+9
| | | | | | | | | | | | | | | * Fix main makeLocalServerCache() call in ObjectCacheFactory::newFromId to include a default keyspace, since wgCachePrefix is false by default (including at WMF). * Idem for ExtensionRegistry. * Dependency inject the domain ID so that service wiring does the correct thing when doing cross-wiki operations. This is a followup on: I3179a387486377c6a575d173f39f82870c49c321. Bug: T358346 Bug: T361177 Change-Id: Ibbb250465529810b8593f90bbb8330af0a2c3dbd
* Merge "objectcache: remove deprecated QOS_EMULATION_SQL constant"jenkins-bot2024-03-281-1/+0
|\
| * objectcache: remove deprecated QOS_EMULATION_SQL constantAaron Schulz2024-03-271-1/+0
| | | | | | | | Change-Id: I8dec3f73faec44a00ef12614039c323dd07b695f
* | objectcache: Fix typo in getLocalServerInstance deprecation noticeTimo Tijhof2024-03-271-3/+1
|/ | | | | Bug: T358346 Change-Id: I16a5f588707b2e79ed5340ec8f0aec96195aa577
* objectcache: Introduce `ObjectCacheFactory` MW serviceDerick Alangi2024-03-192-173/+331
| | | | | | | | | | | | | | | | | | | | | ObjectCache is already doing a lot of factory pattern logic like creating instances of the various BagOStuff, this should really be the responsibility of the factory servicet. This patch introduces a proper factory (ObjectCacheFactory) to handle the responsibility of creating various instances of BagOStuff. Since `newFromParams()` is a static function that gets passed in configuration of $wgObjectCaches, that can stay that way (to keep supporting how we do this in prod today). Technical Breaking Change: `ObjectCache::makeLocalServerCache()` now has a parameter and requires it but there are no callers of this method outside MW core hence it is safe to change (and this patch update all callers) to work correctly. Cache prefix is gotten from global state because sometimes at this stage, the services container is not available. Bug: T358346 Change-Id: I3179a387486377c6a575d173f39f82870c49c321
* objectcache: Migrate BagOStuff from StatsdD to StatsLibDerick Alangi2024-02-271-1/+1
| | | | | | | | | | | | | | | | | | | | | | | This patch migrates BagOStuff metrics capture from StatsdD to StatsLib. `updateOpStats()` definition significantly changed with this migration thereby affecting the changes in the test case. What we tried to do in the test is to assert that the metrics match the StatsLib equivalent ones which is the new format we're using and that the values corresponding to the samples are also in the range of values written to the cache. Notes ===== * And the cache key also uses underscore (_) instead of (.) because of StatsLib as well. * The most important thing which is tested is that the keys inserted into the cache are all there and correspond to their values. The way we do this is observe the metrics stream after `->flush()`. Bug: T356062 Change-Id: Ia6c14746de5bddeaca7917c76f1c9d1e100ae2b2
* objectcache: Drop support for `$wgObjectCaches['db-replicated']`Derick Alangi2023-12-081-4/+0
| | | | | | | | | | | | | | | This was introduced but never really used outside of core[1]. The only place that used it in core was MainStash setting which under the hood will use CACHE_DB (SqlBagOStuff). This patch removes the "db-replicated" key in $wgObjectCaches without deprecation because it was never really used in the first place and had a replacement already when it got released, see: T352481. [1] https://codesearch.wmcloud.org/search/?q=ReplicatedBagOStuff&files=&excludeFiles=&repos= Bug: T352481 Change-Id: I8e19ee262a64b00742bb9203b2a2610ec0cc39fa
* Namespace remaining files under includes/deferredJames D. Forrester2023-11-221-0/+1
| | | | | Bug: T166010 Change-Id: Ibd40734b96fd2900e3ce12239d09becfb4150059
* Improve uses of ->where( array_merge( … ) ) in query buildersBartosz Dziewoński2023-10-261-6/+2
| | | | | | | Replace ->where( array_merge( a, b ) ) with ->where( a )->andWhere( b ). It's shorter and I find it easier to read. Change-Id: I94fef6219b5611659f7a09fd3a555aba001f5339
* Mass migrate simple cases to use expression builderAmir Sarabadani2023-10-261-4/+4
| | | | | | | | | | | | | | | Done via '([A-Za-z_\.]+) ?(=|!=|<|<=|>|>=) ?' . (\$db(?:r|w|))->addQuotes\( (.+?) \) to: $3->expr\( '$1', '$2', $4 \) And '([A-Za-z_\.]+) IS NULL OR ([A-Za-z_\.]+) ?(=|!=|<|<=|>|>=) ?' . (\$db(?:r|w|))->addQuotes\( (.+?) \) to: $4->expr( '$1', '=', null )->or\( '$2', '$3', $5 \) Bug: T210206 Change-Id: I109bf2a712bdefa9e074f775b1bee41ac5b9d665
* Replace single-value $db->buildComparison() with $db->expr()Bartosz Dziewoński2023-10-221-1/+1
| | | | | | | | | | Find: ->buildComparison\( ('..?'), \[(\s*)([^\],]+) => ([^\],]+)(\s*)\] \) Replace with: ->expr($2$3, $1, $4$5) Change-Id: I2cfc3070c2a08fc3888ad48a995f7d79198cc336
* objectcache: Deprecate unused SqlBagOStuff::deleteAll()Derick Alangi2023-09-271-0/+4
| | | | | | | | | This method is no longer used anywhere. This looks like it was never used since its introduction in r3516 (0c2fba0ac401938729a14c3bda5e793a96295139). Change-Id: I4318ac4ffded2b0b5f7ed2b1bb7aeba84d198da7
* objectcache: Deprecate unused `SqlBagOStuff::expireAll()`Derick Alangi2023-09-251-0/+4
| | | | | | | | | | In c860f99a6ea9a4fe6d6bcec693f2cacf74f883b9, the last usage of the method was removed. We can soft and hard deprecate it at once, did not find any known usage per code search. See: https://codesearch.wmcloud.org/search/?q=expireAll%5C%28&files=&excludeFiles=&repos= Change-Id: I0e4aefec984d16087a72a72381e31cbc72d529c9
* Merge "http: MultiHttpClient supports TelemetryHeadersInterface"jenkins-bot2023-09-121-0/+4
|\
| * http: MultiHttpClient supports TelemetryHeadersInterfacePiotr Miazga2023-09-121-0/+4
| | | | | | | | | | | | | | | | | | | | | | | | | | | | Introduce a new interface Wikimedia/Http/TelemetryHeadersInterface that provides telemetry information that could be attached to HTTP Requests. MultiHttpClient is expecting `telemetry` option of TelemetryHeadersInterface type. The MediaWiki/Http/Telemetry implements the interface, therefore ObjectCache can inject it to RESTBagOStuff, that further injects it to MultiHttpClient. Bug: T344926 Change-Id: I59a3f1048c403fe2e4ef0c74353dfe74ff9ca893
* | objectcache: Roll up small once-used private methods to the callerAmir Sarabadani2023-09-121-80/+58
| | | | | | | | | | | | Small clean up Change-Id: I5816e4e7f8652809a3b8c500d8c798bf1312bb33
* | Merge "rdbms: Move DebugDumpSql knowledge from SqlBagOStuff to DatabaseFactory"jenkins-bot2023-09-111-10/+3
|\ \
generated by cgit v1.2.3 (git 2.25.1) at 2025-04-26 08:01:46 +0000