aboutsummaryrefslogtreecommitdiffstats
path: root/maintenance/refreshImageMetadata.php
Commit message (Collapse)AuthorAgeFilesLines
* Replace a number of uses of class aliases with the real thingsJames D. Forrester2025-03-261-0/+2
| | | | | | | Found via disabling phan's alias support for a run (and ignoring wgLang's hard-coded state of being a \Language, alas). Change-Id: I4753bcd84d72d6de111fc7ffc5841fa417bf7333
* add `use MediaWiki\Maintenance\Maintenance` to some maintenance classesNovem Linguae2024-12-051-0/+1
| | | | | | | | | this patch changes P–Z. still need to do subfolders in one file, I also deleted an errant line break that was disconnecting the class docblock from the class. Change-Id: I3b60c2a5817f12c47490e78e36cc18bfeeb98af9
* Exclude boilerplate maintenance code from code coverage reportsDreamy Jazz2024-08-271-0/+4
| | | | | | | | | | | | | | | | | | | | | | | | | | | Why: * Maintenance scripts in core have bolierplate code that is added before and after the class to allow directly running the maintenance script. * Running the maintenance script directly has been deprecated since 1.40, so this boilerplate code is only to support a now deprecated method of running maintenance scripts. * This code cannot also be marked as covered, due to PHPUnit not recognising code coverage for files. * Therefore, it is best to ignore this boilerplate code in code coverage reports as it cannot be marked as covered and also is for deprecated code. What: * Wrap the boilerplate code (requiring Maintenance.php and then later defining the maintenance script class and running if the maintenance script was called directly) with @codeCoverageIgnore comments. * Some files use a different boilerplate code, however, these should also be marked as ignored for coverage for the same reason that coverage is not properly reported for files. Bug: T371167 Change-Id: I32f5c6362dfb354149a48ce9c28da9a7fc494f7c
* Replace IDatabase::buildLike with expression builderUmherirrender2024-05-181-1/+4
| | | | | Bug: T361023 Change-Id: I0fcb61ff1b15931477bce3c8f850a8ce97405a36
* maintenance: Introduce getReplicaDB() and getPrimaryDB()Amir Sarabadani2024-01-181-1/+1
| | | | | | | | | | | | And start using them instead of wfGetDB(), LB/LBF connection methods or worse, $this->getDB(). $this->getDB() reuses the database object regardless of whether you're calling a replica or primary, leading to returning a replica on a primary and other way around. Bug: T330641 Change-Id: I9e2cf85ca277022284fc26b9f37db57bd12aaa81
* maintenance: Migrate to expression buildersAmir Sarabadani2024-01-171-3/+3
| | | | | | | This was somehow left out Bug: T210206 Change-Id: I70851b5b99fa865dbfd629caf2c1866c85418350
* maintenance: Begin using `Maintenance::getServiceContainer()`Derick Alangi2023-09-041-2/+1
| | | | | | | | | | | Maintenance class provides a method for getting a fresh reference of the MW services container instance. Let's make use of these in maintenance scripts now that we have it. NOTE: There are still some static methods like in refreshLinks.php that makes use of services that we can't use this method for now. Change-Id: Idba744057577896fc97c9ecf4724db27542bf01c
* Introduce FileSelectQueryBuilderAmir Sarabadani2023-08-071-38/+29
| | | | | | | So much can be cleaned up with this Bug: T311866 Change-Id: Ia4d46679c540c731b2ae8da2f8022fd6f5b931a4
* maintenance: Use $this->waitForReplication()Amir Sarabadani2022-10-241-3/+1
| | | | | | | This adds reconfiguring db pools in case a replica gets depooled Bug: T298485 Change-Id: Id052ce8ed45c51e51b071778858d27b48605bf93
* Use short array destructuring instead of list()Tim Starling2022-10-211-1/+1
| | | | | | | | Introduced in PHP 7.1. Because it's shorter and looks nice. I used regex replacement. Change-Id: I0555e199d126cd44501f859cb4589f8bd49694da
* phan: Remove PhanPossiblyUndeclaredVariable suppressionUmherirrender2022-03-301-0/+1
| | | | | | | | | Make phan stricter about conditional variable declaration Remaining false positive issues are suppressed. The suppression and the setting change can only be done together Bug: T259172 Change-Id: I1f200ac37df7448453688bf464a8250c97313e5d
* maintenance: Add support for oldimage table metadata refreshAmir Sarabadani2022-01-031-16/+26
| | | | | | | | This doesn't handle deleted files yet but that can be done in later patches. I have been struggling to find a solution for it. Bug: T298417 Change-Id: I4f33afa92be8b55d6b8f9797cb4d3aa03a37ad79
* media: Drop XML metadata support from DjvuHandlerAmir Sarabadani2021-11-211-1/+3
| | | | | | | | All of production has been migrated to json and this patch also adds an update entry for third parties. Bug: T275268 Change-Id: I916127896bdce95472823ae7be12fc5e6e16691a
* Add --sleep option to refreshImageMetadata.phpAmir Sarabadani2021-06-291-0/+10
| | | | | | | | | For running long-term maintenance scripts, having waitForReplication() is not enough as that method doesn't count for cross DC-replication and replication the cloud making issues if kept running more than a couple of hours. This mitigate the aforementioned problem. Change-Id: Ifa0b31dde4a57a4d3f2b00c29410be568cfa6b91
* Manual and automatic image metadata reserializationTim Starling2021-06-261-11/+36
| | | | | | | | | | | | | | | | | | | | | | | * Add automatic splitting of large metadata on upload or refresh. If the reserializeMetadata option is enabled, metadata stored with PHP serialization will be automatically reserialized to JSON. * Inject configuration variable $wgUpdateCompatibleMetadata via LocalRepo instead of accessing it directly. * In refreshImageMetadata.php and rebuildImages.php, construct a new LocalRepo with config overrides, instead of overwriting config globals. Add a helper to RepoGroup to help with this. * In refreshImageMetadata.php, add new options --convert-to-json and --split which reserialize metadata and optionally split out large items to blob storage. Also, refreshImageMetadata.php was totally broken in the non-force mode since metadata refresh on page view was disabled in b814245d9f3bb. The maintenance script was relying on newFileFromRow() magically upgrading the row, which doesn't happen anymore. So, call maybeUpgradeRow() directly. Bug: T275268 Change-Id: I7bf7d9cef71641e287ca4346b568b381f4ada50e
* Use the unserialized form of image metadata internallyTim Starling2021-06-081-26/+1
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Image metadata is usually a serialized string representing an array. Passing the string around internally and having everything unserialize it is an awkward convention. Also, many image handlers were reading the file twice: once for getMetadata() and again for getImageSize(). Often getMetadata() would actually read the width and height and then throw it away. So, in filerepo: * Add File::getMetadataItem(), which promises to allow partial loading of metadata per my proposal on T275268 in a future commit. * Add File::getMetadataArray(), which returns the unserialized array. Some file handlers were returning non-serializable strings from getMetadata(), so I gave them a legacy array form ['_error' => ...] * Changed MWFileProps to return the array form of metadata. * Deprecate the weird File::getImageSize(). It was apparently not called by anything, but was overridden by UnregisteredLocalFile. * Wrap serialize/unserialize with File::getMetadataForDb() and File::loadMetadataFromDb() in preparation for T275268. In MediaHandler: * Merged MediaHandler::getImageSize() and MediaHandler::getMetadata() into getSizeAndMetadata(). Deprecated the old methods. * Instead of isMetadataValid() we now have isFileMetadataValid(), which only gets a File object, so it can decide what data it needs to load. * Simplified getPageDimensions() by having it return false for non-paged media. It was not called in that case, but was implemented anyway. In specific handlers: * Rename DjVuHandler::getUnserializedMetadata() and extractTreesFromMetadata() for clarity. "Metadata" in these function names meant an XML string. * Updated DjVuImage::getImageSize() to provide image sizes in the new style. * In ExifBitmapHandler, getRotationForExif() now takes just the Orientation tag, rather than a serialized string. Also renamed for clarity. * In GIFMetadataExtractor, return the width, height and bits per channel instead of throwing them away. There was some conflation in decodeBPP() which I picked apart. Refer to GIF89a section 18. * In JpegMetadataExtractor, process the SOF0/SOF2 segment to extract bits per channel, width, height and components (channel count). This is essentially a port of PHP's getimagesize(), so should be bugwards compatible. * In PNGMetadataExtractor, return the width and height, which were previously assigned to unused local variables. I verified the implementation by referring to the specification. * In SvgHandler, retain the version validation from unpackMetadata(), but rename the function since it now takes an array as input. In tests: * In ExifBitmapTest, refactored some tests by using a provider. * In GIFHandlerTest and PNGHandlerTest, I removed the tests in which getMetadata() returns null, since it doesn't make sense when ported to getMetadataArray(). I added tests for empty arrays instead. * In tests, I retained serialization of input data since I figure it's useful to confirm that existing database rows will continue to be read correctly. I removed serialization of expected values, replacing them with plain data. * In tests, I replaced access to private class constants like BROKEN_FILE with string literals, since stability is essential. If the class constant changes, the test should fail. Elsewhere: * In maintenance/refreshImageMetadata.php, I removed the check for shrinking image metadata, since it's not easy to implement and is not future compatible. Image metadata is expected to shrink in future. Bug: T275268 Change-Id: I039785d5b6439d71dcc21dcb972177dba5c3a67d
* Replace uses of DB_MASTER with DB_PRIMARYJames D. Forrester2021-04-291-1/+1
| | | | | | Just an auto-replace from codesniffer for now. Change-Id: I5240dc9ac5929d291b0ef1c743ea2bfd3f428266
* Replace wfWaitForSlaves() with LBFactory::waitForReplication()Reedy2020-05-021-1/+2
| | | | Change-Id: I337147d61e2ec686a8672d0340dff4b6783f78cd
* maintenance: Avoid deprecated usage of RepoGroup::singleton()Derick A2020-02-181-1/+2
| | | | Change-Id: I694cf124a5f90f368309b04f8ae0b843a98816eb
* Set method visibility on maintenance scriptsUmherirrender2019-11-161-2/+2
| | | | Change-Id: I44c82fbe65e1d002803ce065df6563f06dd39cd4
* Set public for override of Maintenance functionsUmherirrender2019-10-091-1/+1
| | | | | | | | | | | | The following function are set to public in the parent class and cannot have another visibility in subclasses Maintenance::__construct Maintenance::execute Maintenance::getDbType Maintenance::validateParamsAndArgs Maintenance::setDB Change-Id: I0cd6514642d479aca20f1221bf673b0713c21631
* Use ::class to resolve class names in maintenance scriptsUmherirrender2018-01-231-1/+1
| | | | | | | This helps to find renamed or misspelled classes earlier. Phan will check the class names Change-Id: I1d4567f47f93eb1436cb98558388e48d35258666
* Maintenance: add fatalError() methodBryan Davis2017-11-211-2/+2
| | | | | | | | Deprecate the second argument to Maintenance::error() in favor of a new Maintenance::fatalError() method. This is intended to make it easier to review flow control in maintenance scripts. Change-Id: I75699008638f7e99b11210c7bb9e2e131fca7c9e
* Finish migration to Maintenance::getBatchSize()Max Semenik2017-11-071-3/+4
| | | | Change-Id: I02d89f71d820e4d00a39e86a30397b614bbdb432
* Replace selectFields() methods with getQueryInfo()Brad Jorsch2017-10-301-3/+6
| | | | | | | | | | | | | | Several classes have a "selectFields()" static method to tell callers which fields to select from the database. With the recent comment table change and the upcoming actor table change, this pattern has become too simplistic as a SELECT will need to join several tables to be able to retrieve all the needed fields. Thus, we deprecate the selectFields() methods in favor of getQueryInfo() methods that return tables and join conditions in addition to the fields. Change-Id: Idcfd15568489d9f03a7ba4460e96610d33bc4089
* Replace more uses of "SELECT *"Brad Jorsch2017-10-131-1/+1
| | | | | | | | | With the introduction of CommentStore, selects from various table require certain joins or column aliases for proper operation. The upcoming actor table change, and the suggested title table change, will add more such requirements. Change-Id: Ic8213bff74b8350b15cd271d0ef252e63e7e79bd
* Improve output of refreshImageMetadata and refreshFileHeadersGilles Dubuc2017-05-161-3/+1
| | | | | Bug: T150741 Change-Id: Ie5f787fd77ecd31b8852d0f66de912baced4ca46
* Use IDatabase type hints in /maintenanceAaron Schulz2017-04-071-2/+5
| | | | | | Relatedly, move lockTables()/unlockTables() to IMaintainableDatabase Change-Id: Ib53e9fa948deb2f9a70f0ce16c002613d0060bf9
* Make refreshImageMetadata not fail completely if it doesn't like a single fileReedy2016-12-091-31/+39
| | | | Change-Id: I1170cf91ba6dea08d35457c9859b8c18482d4aa6
* Replace DatabaseBase => Database in more placesAaron Schulz2016-09-281-2/+2
| | | | Change-Id: If37a7909056bf2c31a8228cbc84f0fbbf5f1c517
* Defer maybeUpgradeRow() post-send since they can trigger on non-POSTAaron Schulz2016-08-111-0/+1
| | | | Change-Id: I791e7133d49ed9cd6dd40bb3fa35ea38d1ceba10
* refreshImageMetadata: Allow filtering by 'img_media_type' tooBartosz Dziewoński2016-03-291-1/+12
| | | | | | | | Unlike 'img_major_mime' and 'img_minor_mime', this shouldn't be "inefficient", since there's an index on it. Bug: T131157 Change-Id: I4985cade41c23ef68f5caf276d4934cf24de2bb6
* Convert all array() syntax to []Kunal Mehta2016-02-171-5/+5
| | | | | | | | | | Per wikitech-l consensus: https://lists.wikimedia.org/pipermail/wikitech-l/2016-February/084821.html Notes: * Disabled CallTimePassByReference due to false positives (T127163) Change-Id: I2c8ce713ce6600a0bb7bf67537c87044c7a45c4b
* Use addDescription() instead of accessing mDescription directlyMax Semenik2016-01-301-1/+1
| | | | Change-Id: I0e2aa83024b8abf5298cfea4b21bf45722ad3103
* Update wfGetDB calls in Maintenance scripts to use getDB()Reedy2016-01-021-1/+1
| | | | Change-Id: I9ad6745d84506b736dae94747256caac89715899
* Unify the spelling of MIME in documentationrillke2014-08-071-1/+1
| | | | | | Writing MIME as written in Wikipedia and some documentation clean up. Change-Id: I9dfc36d2bf55d72d9374c4075bd6d45eef0415a4
* Update formatting in maintenance/ (4/4)Siebrand Mazeland2014-04-231-7/+4
| | | | Change-Id: I6b58d014a4bfd6600e4e6f80188fdcfce18482ca
* Pass phpcs-strict on maintenance/ (2/8)Siebrand Mazeland2014-04-221-7/+38
| | | | Change-Id: I69e2bca3c98fe9d3713c852699f49b7b4c868338
* Fixed some @params documentation (maintenance)umherirrender2014-04-171-3/+3
| | | | | | | | | Swapped some "$var type" to "type $var" or added missing types before the $var. Changed some other types to match the more common spelling. Makes beginning of some text in captial. Also added some missing @param. Change-Id: I727deec35a712de0f0c676cc87dfa661f1ee965b
* Change URLs to mediawiki.org in comments to HTTPSLadsgroup2014-03-201-1/+1
| | | | | | | These are only documentation fixes http://www.mediawiki.org --> https://www.mediawiki.org Change-Id: I62ad42be1a3aac410cc53e98ce79389ceddd8988
* Fixed spacingumherirrender2013-11-211-1/+0
| | | | | | | | - Removed trailing spaces in comments - Removed multiple empty lines - Removed space after object operator Change-Id: I9fd3256ab490c7cd2034de3fd94e6be6e6d6d8f2
* phpcs: More require/include is not a functionTimo Tijhof2013-05-211-1/+1
| | | | | | | | | | | | | Follows-up I1343872de7, Ia533aedf63 and I2df2f80b81. Also updated usage in text in documentation and the installer LocalSettingsGenerator. Most of them were handled by this regex: - find: (require|include|require_once|include_once)\s*\(\s*(.+?)\s*\)\s*;$ - replace: $1 $2; Change-Id: I6b38aad9a5149c9c43ce18bd8edbab14b8ce43fa
* phpcs: Fix WhiteSpace.LanguageConstructSpacing warningsTimo Tijhof2013-05-091-1/+1
| | | | | | | | | | | | | | | | | | | | | | | | Squiz.WhiteSpace.LanguageConstructSpacing: Language constructs must be followed by a single space; expected "require_once expression" but found "require_once(expression)" It is a keyword (e.g. like `new`, `return` and `print`). As such the parentheses don't make sense. Per our code conventions, we use a space after keywords like these. We appeared to have an unwritten exception for `require` that doesn't make sense. About 60% of require/include usage was missing the space and/or had superfluous parentheses. It is as silly as print("foo") or return("foo"), it works because keywords have no significance for whitespace between it and the expression that follows, and since experessions can be wrapped in parentheses for clarity (e.g. when doing string concatenation or mathematical operations) the parenthesis before and after basiclaly just ignored. Change-Id: I2df2f80b8123714bea7e0771bf94b51ad5bb4b87
* Fixed spacing around parenthesis in languages/tests/maintenanceumherirrender2013-04-281-3/+3
| | | | Change-Id: Idd4299d17f1fcf98ab1d635484cb4e880f35ee24
* Fixed some spacing in maintenance folderumherirrender2013-04-181-2/+2
| | | | | | | Added spaces before if, foreach Added some braces for one line statements Change-Id: I9657f72996358f8c1c154cea1ea97970d973723c
* style: fix up commas in function argumentsAntoine Musso2013-02-061-1/+1
| | | | | | | | | | | | Fix up spaces in our function calls, we do not want spaces before a comma and try to avoid multiple commas whenever possible. Errors: * No space found after comma in function call * Space found before comma in function call Change-Id: I51aec02016f742422fa60b92ad35ba3f0ef59ba3
* Use __DIR__ instead of dirname( __FILE__ )jeroendedauw2012-08-271-1/+1
| | | | | | We can now do this since we finally switched to PHP 5.3 for MW 1.20 and get rid of the silly dirname(__FILE__) stuff :) Change-Id: Id9b2c9cd2e678197aa81c78adced5d1d31ff57b1
* Improve documentation of maintenance scripts.Alexandre Emsenhuber2012-08-091-1/+6
| | | | Change-Id: I21b4fb873e88026108754eb7206e62c82648df0e
* Fix doc for maintenance/Antoine Musso2012-02-081-1/+1
| | | | Notes: http://mediawiki.org/wiki/Special:Code/MediaWiki/110941
* Followup r95458Sam Reed2011-09-061-1/+9
| | | | | | | | | | | Add a couple of bits of documentation Removed an unused global No need to 1.18 this Notes: http://mediawiki.org/wiki/Special:Code/MediaWiki/96345
generated by cgit v1.2.3 (git 2.25.1) at 2025-04-21 16:31:48 +0000