aboutsummaryrefslogtreecommitdiffstats
path: root/maintenance/purgeParserCache.php
Commit message (Collapse)AuthorAgeFilesLines
* add `use MediaWiki\Maintenance\Maintenance` to some maintenance classesNovem Linguae2024-12-051-0/+1
| | | | | | | | | this patch changes P–Z. still need to do subfolders in one file, I also deleted an errant line break that was disconnecting the class docblock from the class. Change-Id: I3b60c2a5817f12c47490e78e36cc18bfeeb98af9
* maintenance: Add missing documentation to class propertiesUmherirrender2024-09-131-0/+2
| | | | | | | | | | Add doc-typehints to class properties found by the PropertyDocumentation sniff to improve the documentation. Once the sniff is enabled it avoids that new code is missing type declarations. This is focused on documentation and does not change code. Change-Id: I7dec01892a987a87b1b79374a1c28f97d055e8fa
* Exclude boilerplate maintenance code from code coverage reportsDreamy Jazz2024-08-271-0/+4
| | | | | | | | | | | | | | | | | | | | | | | | | | | Why: * Maintenance scripts in core have bolierplate code that is added before and after the class to allow directly running the maintenance script. * Running the maintenance script directly has been deprecated since 1.40, so this boilerplate code is only to support a now deprecated method of running maintenance scripts. * This code cannot also be marked as covered, due to PHPUnit not recognising code coverage for files. * Therefore, it is best to ignore this boilerplate code in code coverage reports as it cannot be marked as covered and also is for deprecated code. What: * Wrap the boilerplate code (requiring Maintenance.php and then later defining the maintenance script class and running if the maintenance script was called directly) with @codeCoverageIgnore comments. * Some files use a different boilerplate code, however, these should also be marked as ignored for coverage for the same reason that coverage is not properly reported for files. Bug: T371167 Change-Id: I32f5c6362dfb354149a48ce9c28da9a7fc494f7c
* maintenance: Begin using `Maintenance::getServiceContainer()`Derick Alangi2023-09-041-2/+1
| | | | | | | | | | | Maintenance class provides a method for getting a fresh reference of the MW services container instance. Let's make use of these in maintenance scripts now that we have it. NOTE: There are still some static methods like in refreshLinks.php that makes use of services that we can't use this method for now. Change-Id: Idba744057577896fc97c9ecf4724db27542bf01c
* ParserCache: Improve docs for cache type and purgeParserCache.phpTimo Tijhof2023-01-301-5/+6
| | | | Change-Id: I3301ff90a135bd0103c1dfc7e86f1dd1ba245a5a
* Maintenance scripts: replace most globals with ConfigFerran Tufan2022-07-121-3/+4
| | | | | Bug: T72673 Change-Id: Idc46e88756b1aa20f5dccbe7ab3e661b2b102964
* Remove unreachable code after exception/die/exitUmherirrender2021-09-031-2/+0
| | | | Change-Id: Ia81903fb2a8157625fbe63115097eccc1161b5db
* purgeParserCache.php: Implement --tag for purging one server onlyTimo Tijhof2021-06-241-1/+14
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Large wiki farms may have to purge servers concurrently (instead of one at a time) in order to keep up with new writes and delete expired rows faster than new rows are written. The parameter for this uses server tags for three reasons: * Maintenance risk and complexity. This requires the least information about MW configuration to be hardcoded in the scheduled maintenance cronjob (compared to: server indexes which are a runtime concept, or specific hostnames/IP/ tableprefixes which may change and should not require coordinating changes elsewhere). * Operational convenience. By using server tags, the parameters don't have to vary between data centers. * Code complexity. The current code for obtaining connections is based on server indexes, which are easy to mapped at runtime from server tags. Other ways of identifying shard like hostnames are either an awkward fit (as they don't uniquely identify a shard per-se, with multiple instances on the same hardware at WMF), or require SqlBagOStuff to store and maintain more information about connections than it currently has readily available. Bug: T282761 Change-Id: I618bc1e8ca3008a4dd54778ac24aa5948f27c52e
* purgeParserCache.php: Print stats for time and iterationsTimo Tijhof2021-06-231-3/+18
| | | | | | | | | | | This emperical data will help confirm and gain confidence in our understanding of how much time is spent in the delete operations under various conditions, and by extent whether it'll be worth converting the sleep duration logic to an interval that includes the time spent in the delete operation itself. Bug: T282761 Change-Id: I0406d9c0807bb9722623f3ec61f1e559dbe14aa9
* purgeParserCache.php: Remove carriage return printer, minor cleanupTimo Tijhof2021-05-131-13/+32
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | * There was a carriage return, as part of a self-overwriting progress bar. While nice for humans running the script by hand, this made the script difficult to understand in WMF production where the logs go to syslog/journalctl, which doesn't like \r and thus showed only `[2KB blob]` instead of the actual output. This can bypassed with `journalctl -u … --all`, but that still leaves one with a sea of misaligned progress bars, concatenated as one long "line". It seems most of our maintenance scripts nowadays just keep printing lines to notify of progress made, so change the script to that, by just printing a boring "... % done\n" line (at most) every 0.1%. * Change hardcoded batch size of 100 to use $wgUpdateRowsPerQuery. This is generally a no-op as this configuration defaults to 100 in DefaultSettings.php, and is not overridden in WMF production. * Add a "--dry-run" option to allow for last-minute verification of the age and date calculation which is rather non-trivial given that we don't index creation time and assume $wgParserCacheExpireTime is constant and unchanged, which in practice, when this script is considered for running, tends not to be the case, making it hard to be sure exactly what it will do. Minor stuff: * Make the ORDER BY clause explicitly use ascending order. * Try to retroactively document why we use a --msleep parameter instead of waiting for replication. * Revert some of the variable name changes of I723e6377c26750ff9 which imho made the code harder to understand. * Flip order of $overallRatio additions to match the order of $tablesDoneRatio (e.g. high level + current thing). * Remove use of Language->timeanddate() in favour of native date formatting, mainly to include seconds and timezone, but also because its simpler not to bring in all of Language and LocalisationCache into the script. * Pass unix time directly to SqlBagOStuff::deleteObjectsExpiringBefore, as other callers do, instead of TS_MW-formatting first. There is also additional casting to TS_UNIX within the method. Before: > Deleting objects expiring before 21:43, 7 May 2021 > [**************************************************] 100.00% > Done After: > Deleting objects expiring before Fri, 07 May 2021 21:43:52 GMT > ... 50.0% done > ... 100.0% done > Done Bug: T280605 Bug: T282761 Change-Id: I563886be0b3aeb187c274c0de4549374e0ff18f0
* Safer autoloading with respect to file-scope codeTim Starling2021-01-111-1/+1
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Many files were in the autoloader despite having potentially harmful file-scope code. * Exclude all CommandLineInc maintenance scripts from the autoloader. * Introduce "NO_AUTOLOAD" tag which excludes the file containing it from the autoloader. Use it on CommandLineInc.php and a few suspicious-looking files without classes in case they are refactored to add classes in the future. * Add a test which parses all non-PSR4 class files and confirms that they do not contain dangerous file-scope code. It's slow (15s) but its results were enlightening. * Several maintenance scripts define constants in the file scope, intending to modify the behaviour of MediaWiki. Either move the define() to a later setup function, or protect with NO_AUTOLOAD. * Use require_once consistently with Maintenance.php and doMaintenance.php, per the original convention which is supposed to allow one maintenance script to use the class of another maintenance script. Using require breaks autoloading of these maintenance class files. * When Maintenance.php is included, check if MediaWiki has already started, and if so, return early. Revert the fix for T250003 which is incompatible with this safety measure. Hopefully it was superseded by splitting out the class file. * In runScript.php add a redundant PHP_SAPI check since it does some things in file-scope code before any other check will be run. * Change the if(false) class_alias(...) to something more hackish and more compatible with the new test. * Some site-related scripts found Maintenance.php in a non-standard way. Use the standard way. * fileOpPerfTest.php called error_reporting(). Probably debugging code left in; removed. * Moved mediawiki.compress.7z registration from the class file to the caller. Change-Id: I1b1be90343a5ab678df6f1b1bdd03319dcf6537f
* Fix various MediaWiki.WhiteSpace.SpaceBeforeSingleLineComment.NewLineCommentReedy2020-05-211-1/+2
| | | | Change-Id: I50c7c93f1534e966224f98a835ca01f93eb9416d
* Remove Language::factory and getParentLanguage useAryeh Gregor2019-10-271-1/+1
| | | | Change-Id: I11f8801ef47ec1a1f63d840116e69667e6f3ae3c
* Set public for override of Maintenance functionsUmherirrender2019-10-091-2/+2
| | | | | | | | | | | | The following function are set to public in the parent class and cannot have another visibility in subclasses Maintenance::__construct Maintenance::execute Maintenance::getDbType Maintenance::validateParamsAndArgs Maintenance::setDB Change-Id: I0cd6514642d479aca20f1221bf673b0713c21631
* Use ::class to resolve class names in maintenance scriptsUmherirrender2018-01-231-1/+1
| | | | | | | This helps to find renamed or misspelled classes earlier. Phan will check the class names Change-Id: I1d4567f47f93eb1436cb98558388e48d35258666
* Maintenance: add fatalError() methodBryan Davis2017-11-211-2/+2
| | | | | | | | Deprecate the second argument to Maintenance::error() in favor of a new Maintenance::fatalError() method. This is intended to make it easier to review flow control in maintenance scripts. Change-Id: I75699008638f7e99b11210c7bb9e2e131fca7c9e
* Turn ParserCache into a service, deprecate $parserMemcKunal Mehta2017-07-051-1/+3
| | | | | | | | | | | | | | | | | | | | | | ParserCache is already a singleton, making it a good candidate for a service. $parserMemc is an odd global (it lacks the "wg" prefix) and is ripe for deprecation. The following are now deprecated: * $parserMemc global * ParserCache::singleton() * wfGetParserCacheStorage() A ParserCache::getCacheStorage() method was added for cases where direct access to the underlying BagOStuff object is necessary. Usage of $parserMemc will emit deprecation warnings through the DeprecatedGlobal class mechanism. All usage in core was migrated. Also take this opportunity to inject the $wgCacheEpoch global value into ParserCache. This will require an update to the FlaggedRevs extension. Change-Id: I2ac7afff0d8522214329248c3d1cdccd0f72bbd4
* Add --msleep option to purgeParserCache.phpAaron Schulz2016-11-101-8/+18
| | | | | Bug: T150124 Change-Id: I22be4ab67b6f9b0adb5cb62dd54b1139ee29739a
* Convert all array() syntax to []Kunal Mehta2016-02-171-1/+1
| | | | | | | | | | Per wikitech-l consensus: https://lists.wikimedia.org/pipermail/wikitech-l/2016-February/084821.html Notes: * Disabled CallTimePassByReference due to false positives (T127163) Change-Id: I2c8ce713ce6600a0bb7bf67537c87044c7a45c4b
* Update formatting in maintenance/ (4/4)Siebrand Mazeland2014-04-231-2/+3
| | | | Change-Id: I6b58d014a4bfd6600e4e6f80188fdcfce18482ca
* phpcs: More require/include is not a functionTimo Tijhof2013-05-211-1/+1
| | | | | | | | | | | | | Follows-up I1343872de7, Ia533aedf63 and I2df2f80b81. Also updated usage in text in documentation and the installer LocalSettingsGenerator. Most of them were handled by this regex: - find: (require|include|require_once|include_once)\s*\(\s*(.+?)\s*\)\s*;$ - replace: $1 $2; Change-Id: I6b38aad9a5149c9c43ce18bd8edbab14b8ce43fa
* phpcs: Fix WhiteSpace.LanguageConstructSpacing warningsTimo Tijhof2013-05-091-1/+1
| | | | | | | | | | | | | | | | | | | | | | | | Squiz.WhiteSpace.LanguageConstructSpacing: Language constructs must be followed by a single space; expected "require_once expression" but found "require_once(expression)" It is a keyword (e.g. like `new`, `return` and `print`). As such the parentheses don't make sense. Per our code conventions, we use a space after keywords like these. We appeared to have an unwritten exception for `require` that doesn't make sense. About 60% of require/include usage was missing the space and/or had superfluous parentheses. It is as silly as print("foo") or return("foo"), it works because keywords have no significance for whitespace between it and the expression that follows, and since experessions can be wrapped in parentheses for clarity (e.g. when doing string concatenation or mathematical operations) the parenthesis before and after basiclaly just ignored. Change-Id: I2df2f80b8123714bea7e0771bf94b51ad5bb4b87
* Use output/error maintenance functionsReedy2013-03-081-8/+6
| | | | Change-Id: I9f9ebee9e577ab3cc7a042e5280161d2b2bad57a
* Clean up: Declare variables with public instead of varMarius Hoch2012-09-141-1/+1
| | | | | | | | | | | | Variables in classes should be declared using public $foo instead of var $foo for various reasons. As we require PHP 5.3 we don't have to take care about that PHP4 left over, but can get rid of it in favour of the more clear and better readable public. See also: http://php.net/manual/en/language.oop5.visibility.php (Divided into several commits to keep reviewable) Change-Id: Ic723d0347ab2e3c78bc0097345c68bbee3dc035a
* Use __DIR__ instead of dirname( __FILE__ )jeroendedauw2012-08-271-1/+1
| | | | | | We can now do this since we finally switched to PHP 5.3 for MW 1.20 and get rid of the silly dirname(__FILE__) stuff :) Change-Id: Id9b2c9cd2e678197aa81c78adced5d1d31ff57b1
* Improve documentation of maintenance scripts.Alexandre Emsenhuber2012-08-051-5/+10
| | | | Change-Id: I557f85e8526a3e4b48107fbf299ff39f6af1ac12
* Added missing GPLv2 headers in some places.Alexandre Emsenhuber2012-06-141-1/+19
| | | | | | Also made file/class documentation more consistent. Change-Id: I8ee0861bfc2e9ff6511ad61175f1f1273739816c
* Fix doc for maintenance/Antoine Musso2012-02-081-0/+4
| | | | Notes: http://mediawiki.org/wiki/Special:Code/MediaWiki/110941
* purgeParserCache.php improvements:Tim Starling2011-12-021-3/+19
| | | | | | | | | * Do queries in batches of 100 rows instead of all at once. Note that if there are a lot of objects with the exact same expiry time, it might take multiple runs of the script to delete them all. * Progress meter. Notes: http://mediawiki.org/wiki/Special:Code/MediaWiki/104949
* * Added a script to reduce disk space on a MySQL parser cache setup such as ↵Tim Starling2011-09-091-0/+43
the one at Wikimedia at the moment, by removing all objects which expire before a given time. * Fixed unintentional shortcut evaluation in MultiWriteBagOStuff::doWrite(). It would have caused writes to be skipped on the second cache if the first cache failed, now writes should be attempted on all caches. Notes: http://mediawiki.org/wiki/Special:Code/MediaWiki/96645
generated by cgit v1.2.3 (git 2.25.1) at 2025-04-21 08:15:39 +0000