aboutsummaryrefslogtreecommitdiffstats
path: root/includes/objectcache
Commit message (Collapse)AuthorAgeFilesLines
* Merge "SqlBagOStuff: Fix modtoken comparison"jenkins-bot2022-08-191-1/+2
|\
| * SqlBagOStuff: Fix modtoken comparisonTim Starling2022-08-191-1/+2
| | | | | | | | | | | | | | | | | | | | | | | | String offsets in MariaDB are 1-based, except in "Oracle compatible" mode. SUBSTR(modtoken,0,13) was always the empty string and so the modtoken comparison was always true. I was able to reproduce a failure to reach consistency using ring replication. Add regression test. Bug: T315271 Change-Id: I74e54e8aba44505dd04426c12d91a9ea0de17f22
* | SqlBagOStuff: Migrate from IDatabase::select to SelectQueryBuilderDerick Alangi2022-08-181-36/+46
|/ | | | | Bug: T311866 Change-Id: I0101a0d5f4008bfd79e823248bb625e15f7135bd
* Merge "SqlBagOStuff: use cancelAtomic()"jenkins-bot2022-08-171-10/+17
|\
| * SqlBagOStuff: use cancelAtomic()Tim Starling2022-08-171-10/+17
| | | | | | | | | | | | | | | | | | | | | | | | | | | | * When an exception is thrown from modifyTableSpecificBlobsForIncrInit(), call cancelAtomic() on the connection. Otherwise a subsequent call to LBFactory::commitPrimaryChanges() will throw a user-visible error, violating the goal of graceful failure. * After getting a DBConnRef, call ensureConnection() on it. Otherwise the try/catch blocks around getConnection() have no purpose since connections are always lazy-initialised. Bug: T315274 Change-Id: I8a5bc90bb340f34bd4a88872a1a2e72612ac91f4
* | objectcache: Add trace to SqlBagOStuff DBError loggingTimo Tijhof2022-08-171-4/+4
|/ | | | | Bug: T315274 Change-Id: I7551f2d41238f2dd064e1049723b22805b495516
* objectcache: Remove unused WRITE_SYNC flagTimo Tijhof2022-08-021-39/+0
| | | | | | Bug: T270225 Depends-On: I7e72c1180b7ba9e479ade62ab3dd3139d7bd5bb0 Change-Id: I9f59ff35bbb806d7b1375739001d1cf458f366a8
* PHPUnit: introduce setMainCachedaniel2022-07-071-1/+1
| | | | | | | | | | | | The main object cache is disabled during testing. Some integration tests need it though. This provides a clean way to enable it, to replace the hacks that were used so far. Note that we may want to enable the main cache during testing soon. When that happens, this method is still useful to disable the cache in certain tests, and to set a specific cache instance. Change-Id: I04ae1bf1b6b2c8f6310acd2edf89459d01a9c870
* objectcache: Optimise SqlBagOStuff::incrWithInit with WRITE_BACKGROUNDTim Starling2022-07-041-1/+43
| | | | | | Bug: T310662 Bug: T261744 Change-Id: I9549722ff6f0c4d62d1bcbe8de55f51758157ec4
* objectcache: Deprecate BagOStuff::addBusyCallback() and reduce to a stubTim Starling2022-05-271-10/+1
| | | | | | | | | | It's not called by anything since it was removed from ChronologyProtector, and it is not needed for production. Pass the timeout directly to LoadBalancer::waitForAll() instead of using a WaitConditionLoop, as it was before e99bb1b7dcc4c52a74b Change-Id: I5a1903e6e9ac145c06f77c690a0f6a487b4aa43f
* objectcache: Simplify SqlBagOStuff class configurationTim Starling2022-05-272-170/+113
| | | | | | | | | | | | | | | | | | | | | | | | | Substantive changes: * Remove the parameters globalKeyLB and localKeyLB, not needed in production, probably not used anywhere. * Add a "cluster" parameter, for production. * Don't call markServerDown() in LoadBalancer mode. Style changes: * Document config parameters separately from constructor parameters, to clarify the role of the wiring normalisation. * Inject a load balancer from ObjectCache to SqlBagOStuff, using a closure to defer construction. * Clarify the split between LoadBalancer mode and server array mode by introducing a useLB property. Use it instead of special shard index values and array counts. * multiPrimaryModeType wasn't needed anywhere and wasn't available in the constructor, so it just became a boolean multiPrimaryMode. * Add a test covering most of the server array (!useLB) branches. Bug: T212129 Change-Id: Ib097082efb40b514a29a42286dc362f2e3743ee4
* Merge "objectcache: reduce function_exists() calls in SqlBagOStuff"jenkins-bot2022-05-131-2/+7
|\
| * objectcache: reduce function_exists() calls in SqlBagOStuffAaron Schulz2022-05-051-2/+7
| | | | | | | | Change-Id: I221c5bf35277f0aaf458a3343ac583d332e204d2
* | Fix many typos in commentsMatěj Suchánek2022-05-101-4/+4
|/ | | | | | Found using IntelliJ's "Typo" code inspection. Change-Id: I746220ebe6e1e39f6cb503390ec9053e6518cf16
* docs: Fix 'the the' typoKlein Muçi2022-05-051-5/+5
| | | | | Bug: T201491 Change-Id: Idd776953dcf7dadc136e57d586c09b24363005a6
* Use MainConfigNames instead of string literals, #3Aryeh Gregor2022-04-261-5/+6
| | | | | | | | | | | | | This edition brought to you by: grep -ERIn $(grep -o "'[A-Za-z0-9_]*'" includes/MainConfigNames.php | tr "\n" '|' | sed 's/|$/\n/') includes/ I only corrected a fraction of the results provided by that command. I'm submitting the partial patch now so it doesn't bitrot. Bug: T305805 Change-Id: If1918c0b3d88cdf90403921e4310740e206d6962
* objectcache: remove "multiPrimaryMode" DB type assertionAaron Schulz2022-04-211-8/+2
| | | | | | | | | | | | Since no mysql-specific queries are required, allow sqlite and Postgres setups to use "multiPrimaryMode" for easier testing. Avoid "CASE types text and integer cannot be matched" Postgres error by making dbEncodeSerialValue() cast the result to a string so that addQuotes() applies in buildUpsertSetForOverwrite(). Bug: T212129 Change-Id: Ic84a804c272a7070779c8d41d3a33003852d0839
* objectcache: fix comments for SqlBagOStuff::deleteServerObjectsExpiringBefore()Aaron Schulz2022-04-141-1/+2
| | | | Change-Id: I7eef82820187317aece1f53601598ddf9293281f
* objectcache: Simplify docs of SqlBagOStuff 'purgePeriod' optionTimo Tijhof2022-04-131-2/+2
| | | | | | | | | Parsing "the reciprocal of the probability of purging on any write" is fine in isolation, but requires too much effort when trying to undertand the larger picture at the same time. Bug: T212129 Change-Id: I6f21727c05b4fc0449de5d6acec14e49ed1991ef
* Merge "installer: make 'db-replicated' an alias for CACHE_DB for sqlite"jenkins-bot2022-04-051-1/+3
|\
| * installer: make 'db-replicated' an alias for CACHE_DB for sqliteAaron Schulz2022-04-051-1/+3
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | By default, CACHE_DB and db-replicated are similar, with the main difference being that db-replicated will prefer to read from a local replica server rather than the primary server. Setups using sqlite do not have use for replication logic, since everything uses local database files. To reduce contention, SqliteInstaller makes CACHE_DB use a different database file (rather than a table in the main wiki database file). This change makes 'db-replicated' reuse the same cache instance. Tweak ObjectCache::newFromParams() to make ObjectFactory usable. Also set $wgResourceLoaderUseObjectCacheForDeps to true to avoid similar contention in SqlModuleDependencyStore. Bug: T303225 Change-Id: Ia644f35698b69fe340fc1bcae8899c7e332f332d
* | objectcache: make "multiPrimaryMode" work with LB-based SqlBagOStuffAaron Schulz2022-04-051-1/+7
|/ | | | | | | This avoids the previous database type exception in the constructor. Bug: T212129 Change-Id: I43c6a86883f4122f7395b022bdc6ed7a0cf88121
* docs: Document how SqlBagOStuff::createTables() worksTimo Tijhof2022-03-251-4/+14
| | | | | Bug: T272512 Change-Id: I8f1fc754f096543b677f50a357fb430de8ec6c7c
* http,objectcache: Use local-unaware floats in sprintf() callsAaron Schulz2022-03-241-2/+2
| | | | | Bug: T303628 Change-Id: I7d92bc9dd2bc30711572abff36a0e1545c88479b
* phan: Disable null_casts_as_any_type settingUmherirrender2022-03-211-0/+3
| | | | | | | | | | | Make phan stricter about null types by setting null_casts_as_any_type to false (the default in mediawiki-phan-config) Remaining false positive issues are suppressed. The suppression and the setting change can only be done together Bug: T242536 Bug: T301991 Change-Id: I0f295382b96fb3be8037a01c10487d9d591e7e01
* objectcache: add makeTimestampedModificationToken() type hintAaron Schulz2022-03-191-2/+2
| | | | | | | | All callers pass in floats and this is the expected type, used to fill $microseconds. Bug: T303628 Change-Id: I6c6176b4cc308646d86a5b9d2afdc91aadda4a0d
* cache: Fix various scalar and null types to match documentationUmherirrender2022-03-141-13/+14
| | | | | | | | Adding float as type when INF is used as argument Found by phan strict checks Change-Id: I7e19cd0d9b09bf685e4e8387c900f303b669ad54
* Merge "Use updated ObjectFactory namespace"jenkins-bot2022-03-121-1/+1
|\
| * Use updated ObjectFactory namespaceAlexander Vorwerk2022-03-091-1/+1
| | | | | | | | | | | | | | Depends-On: I99c5e5664d2401c36a9890f148eba7c25e6e8324 Depends-On: I48ab818b2965da14af15ef370aa83ad9455badd9 Depends-On: I018371e4b77911e56152ca7b2df734afc73f58a5 Change-Id: I04ebdb52102f6191d49a9cc70b1f98308299e72f
* | Merge "Fix various documentation related to null types (part II)"jenkins-bot2022-03-091-1/+1
|\ \ | |/ |/|
| * Fix various documentation related to null types (part II)Umherirrender2022-03-081-1/+1
| | | | | | | | | | | | | | | | | | The functions returning null or the class property is set explict null. Some function should not accept null or return null. Found by phan strict checks Change-Id: Ie50f23249282cdb18caa332f562a3945a58d86ff
* | SQLBagOStuff: Fix dbdomain argument in getConnectionViaLoadBalancerUmherirrender2022-03-091-2/+3
| | | | | | | | | | | | | | | | | | getMaintenanceConnectionRef second parameter is the group, not the domain Break another line to make the argument order more visible Follow-Up: 618b61a4c629b46ba7767eb39aca61eeea59f2f8 Change-Id: Id6dd0c2f8e5aded759fe403f8435ee7251a46b8c
* | Fix various documentation related to falseUmherirrender2022-03-031-1/+1
|/ | | | | | | | | Some function already document "False on failure", but does not document it on the used type Found by phan strict checks Change-Id: I12eb8bbc99179833ee3e42c1a7d1dc1443682ca6
* Add explicit casts between scalar typesUmherirrender2022-03-011-4/+4
| | | | | | | | | | | | | | | * Some functions accept only string, cast ints and floats to string * After preg_matches or explode() casts numbers to int to do maths * Cast unix timestamps to int to do maths * Cast return values from timestamp format function to int * Cast bitwise operator to bool when needed as bool * php internal functions like floor/round/ceil documented to return float, most cases the result is used as int, added casts Found by phan strict checks Change-Id: Icb2de32107f43817acc45fe296fb77acf65c1786
* Avoid references to DefaultSettings.daniel2022-02-221-2/+2
| | | | | | | | | | | | | | This replaces references to DefaultSettings with references to config-schema.yaml where appropriate. NOTE: this does not yet change Setup.php. DefaultSettings.php remains intact and is still being used. NOTE: this does not remove usages in the installer, see I5d8843a1062fbf for that. Bug: T300129 Change-Id: Ie6152cf510c3be61bc22167ca6d90dfc28910a45
* objectcache: deprecate BagOStuff incr() and decr() methodsAaron Schulz2022-02-181-5/+4
| | | | | | | | | | | | | | | | | | | | All callers in Wikimedia gerrit repos have since been migrated to incrWithInit(), which is usually what is desired and is easier to implement in various backends. Newer memcached versions of the memcached protocol even support auto-initializing increments (including an initial TTL) via the 'ma' command. The incr()/decr() methods currently bloat the interface and subclasses (often with slow and buggy implementations). These methods are also hard to implement in a multi-DC store (either extremely slow or racey) even though callers might assume they handle high concurrency. Also, force each MediumSpecificBagOStuff subclass to implement incrWithInit() instead of having some rely on the base method In mcc.php, add incrWithInit() and simplify parameters, making it possible to test non-memcached backends. Change-Id: I53c9c2c839a1e71d5c104913fea0680c30d11108
* objectcache: add "globalKeyLbDomain" option to use with "globalKeyLB"Aaron Schulz2022-01-311-5/+21
| | | | | | | | | | | | Previously, "globalKeyLB" was generally useless for sharing keys since the LoadBalancer would still use the default domain. This does not effect configuration using "server"/"servers". Follow-up to 6a8943d8c53b. Bug: T212129 Change-Id: I5b20c9d3f153650d6e03581fd523cba4961f664f
* Fix typos in comments (S)Siddharth VP2022-01-091-2/+2
| | | | Change-Id: Id86769ca81ff65707edde520585aa9ea117123e1
* objectcache: Change variable name from master to primaryUmherirrender2021-10-141-4/+4
| | | | Change-Id: I3574ea348f32ccd289a0b4158fe93b5cb48ddffa
* Cleanup mixed space/tab line indentUmherirrender2021-09-041-1/+1
| | | | Change-Id: I833052a656b1ce419c0929f6f0514f2a33c2c4cc
* objectcache: simplify BagOStuff::ATTR_* flags and set them for backendsAaron Schulz2021-08-311-3/+1
| | | | | | | | * Remove unused ATTR_SYNCWRITES/QOS_SYNCWRITES_* constants * Remove unused ATTR_LOCALITY/QOS_LOCALITY_* constants Bug: T279977 Change-Id: I5ada228fbd504f8e94833df8a8414ed8da0bce4b
* Merge "objectcache: call silenceTransactionProfiler() in ↵jenkins-bot2021-08-281-1/+3
|\ | | | | | | SqlBagOStuff::fetchBlobs()"
| * objectcache: call silenceTransactionProfiler() in SqlBagOStuff::fetchBlobs()Aaron Schulz2021-08-281-1/+3
| | | | | | | | | | | | | | | | | | This avoids any warnings about DB_PRIMARY connections. Folllows-up 719bfa89774cc1e, which handled the writes, but we still emitted warnings about connections. Change-Id: I1f82af996ba179890d159e4ca307424b46ff6bfa
* | objectcache: cleanup config injection and method names in SqlBagOStuffAaron Schulz2021-08-282-14/+20
|/ | | | | | | | | | | | | Dependency inject the write batch size into SqlBagOStuff. Clean up SqlBagOStuff method names and comments. Renamed getServerIndexByTag() to getShardServerIndexForTag() and getServerShardIndexes() to getShardServerIndexes(). Since each of these servers manages one of the shards, it makes sense to call them "shard servers". Change-Id: I9af1f64602dabe85288937ab52c08470138f4c91
* objectcache: make use of new `modtoken` field in SqlBagOStuff (ii)Aaron Schulz2021-08-201-377/+1059
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Add a multi-primary mode option that supports MySQL DB setups that use circular replication with STATEMENT formatted binlogs. The `modtoken` column is only used when multi-primary mode is explicitly enabled in configuration. The column is used by write queries to determine the "winning" version of keys, with the goal of approximating "Last-Write-Wins" eventual consistency. Writes with different timestamps can be handled by picking the one with the highest timestamp as the "winner". Writes with the same timestamp, from different primary DBs, can be handled by picking the one from the primary DB with the highest server_id. Writes with the same token timestamp from the same primary DB can be handled by picking the last write to appear in the binlog. The delete() operation uses tombstones in multi-primary mode, since there must be a key version to actually compare with the versions from other operations. Also: * Remove "LOCK IN SHARE MODE" that was made obsolete by the CONN_TRX_AUTOCOMMIT flag. For the SQLite transaction case, it is serializable anyway. * Simplified handleWriteError() to match handleReadError() and merged them into handleDBError(). Changes from d56a686f83eb7d19a: * Keep using replace() for set() in non-multi-primary mode. * Fix old encoding issues when incrementing keys with postgres. Bug: T274174 Change-Id: I25b1278586f99fdace8f2a081f6f460f734e5d83
* Revert "objectcache: make use of new `modtoken` field in SqlBagOStuff"Ladsgroup2021-08-171-1018/+377
| | | | | | | | | This reverts commit d56a686f83eb7d19abe9aced0af1985f25025901. Reason for revert: T288998#7287717 Bug: T288998 Change-Id: Ia05b00a345335fe8bdd6041323c3abfa79ded169
* objectcache: make use of new `modtoken` field in SqlBagOStuffAaron Schulz2021-08-051-377/+1018
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | Add a multi-primary mode option that supports MySQL DB setups that use circular replication with STATEMENT formatted binlogs. The `modtoken` column is only used when multi-primary mode is explicitly enabled in configuration. The column is used by write queries to determine the "winning" version of keys, with the goal of approximating "Last-Write-Wins" eventual consistency. Writes with different timestamps can be handled by picking the one with the highest timestamp as the "winner". Writes with the same timestamp, from different primary DBs, can be handled by picking the one from the primary DB with the highest server_id. Writes with the same token timestamp from the same primary DB can be handled by picking the last write to appear in the binlog. The delete() operation uses tombstones in multi-primary mode, since there must be a key version to actually compare with the versions from other operations. Also: * Remove "LOCK IN SHARE MODE" that was made obsolete by the CONN_TRX_AUTOCOMMIT flag. For the SQLite transaction case, it is serializable anyway. * Simplified handleWriteError() to match handleReadError() and merged them into handleDBError(). Bug: T274174 Change-Id: Icc5eff9a032dd3403b5718058f20e38f8ea84af5
* ILoadBalancer and IDatabase: Rename getMasterPos to getPrimaryPosJames D. Forrester2021-08-021-1/+1
| | | | | Bug: T282894 Change-Id: I30a600335af735a13a6ad2d3df56a5f17e05bfab
* build: Updating dependencieslibraryupgrader2021-07-221-1/+1
| | | | | | | | | | | | | | composer: * mediawiki/mediawiki-codesniffer: 36.0.0 → 37.0.0 The following sniffs now pass and were enabled: * Generic.ControlStructures.InlineControlStructure * MediaWiki.PHPUnit.AssertCount.NotUsed npm: * svgo: 2.3.0 → 2.3.1 * https://npmjs.com/advisories/1754 (CVE-2021-33587) Change-Id: I2a9bbee2fecbf7259876d335f565ece4b3622426
* objectcache: make BagOStuff::lock/unlock easier to overrideAaron Schulz2021-07-191-52/+21
| | | | | | | | | | | | | | | Avoid the duplication of the reentrant lock logic in subclasses. Move "lock already expired" logic from getScopedLock() to unlock() so that it applies more generally. Warn when unlock() is called on a key that is not even locked as well. Also: * Rename lock() $expiry argument to $exptime for consistency * Fix return types for serialize()/unserialize() Bug: T274174 Change-Id: I211536e616cf7f1cc60181c378bbf9b35ffa40a4
generated by cgit v1.2.3 (git 2.25.1) at 2025-04-22 17:49:49 +0000