aboutsummaryrefslogtreecommitdiffstats
path: root/includes/libs/filebackend
Commit message (Collapse)AuthorAgeFilesLines
* In .htaccess deny files, use "Satisfy All"Tim Starling2025-04-041-1/+2
| | | | | | | | | | | | | | | These .htaccess files are intended to prohibit all web access. But if the user sets "Satisfy Any" on a parent directory, in conjunction with any permissive require directive like "Require all granted", access will be allowed despite "Require all denied" in .htaccess. So, override Satisfy so that the "Require all denied" will reliably take effect. Note that "Satisfy All" is the default. This only affects non-default installations. Change-Id: Ia5862fb69e439b7ea2ed7af011e1ebf8f1b1f6d6
* Merge "libs: Use type declaration on undocumented private functions"jenkins-bot2025-04-041-1/+1
|\
| * libs: Use type declaration on undocumented private functionsUmherirrender2025-03-311-1/+1
| | | | | | | | Change-Id: I9a74c316b87ae35597ce846a830a55542d9aa14c
* | FileBackend: PHP Deprecated: strrpos(): Passing null to parameter #1 ($haystack)Paladox2025-03-311-1/+1
|/ | | | | | | | | ``` PHP Deprecated: strrpos(): Passing null to parameter #1 ($haystack) of type string is deprecated in /srv/mediawiki/1.42/includes/libs/filebackend/FileBackend.php on line 1588 ``` Bug: T384851 Change-Id: I36c401587d94e7c1ab444d355b03ea88da8fb6e8
* Merge "Replace call_user_func_array with dynamic function call"jenkins-bot2025-02-121-1/+1
|\
| * Replace call_user_func_array with dynamic function callUmherirrender2025-02-031-1/+1
| | | | | | | | | | | | | | | | | | Use modern php syntax to call a callable. Reduce the stack trace to improve performance and better IDE and static analyzer Also replace func_get_args with variables Change-Id: Ie193c4e996ab136e08f1e16c988448d054520aa7
* | Merge "libs: Replace call_user_func with dynamic function call"jenkins-bot2025-02-082-3/+3
|\ \
| * | libs: Replace call_user_func with dynamic function callUmherirrender2025-01-312-3/+3
| |/ | | | | | | | | | | | | | | Use modern php syntax to call a callable. Reduce the stack trace to improve performance and better IDE and static analyzer support Change-Id: I8e563140ac3465ce3b3f8ec5e01bfd493c7269fe
* | Merge "filebackend: Renamespace TempFSFileFactory to match rest of library"jenkins-bot2025-02-083-4/+5
|\ \
| * | filebackend: Renamespace TempFSFileFactory to match rest of libraryGergő Tisza2025-01-213-4/+5
| |/ | | | | | | | | Bug: T382910 Change-Id: I717e0b45b840ddedb67f402e7b62b46a9b628d85
* / Use $string === '' instead of strlen( $string ) === 0Fomafix2025-01-302-4/+4
|/ | | | | | | | | Add $string === false or $string === null where $string can have other types than a string. Also document null as possible return value in FileRepo. Change-Id: Iaa29ba01c3fd6bea506debdc6f929edfe881c808
* Remove 2-line PHPDocs that just repeat the types from the codethiemowmde2025-01-172-4/+0
| | | | | | | | | | | | | | | | Same as Ia294bf4 did for 1-line comments. This patch removes slightly more complex 2-line PHPDoc comments that don't add any new information to the code, but literally repeat what the code already says. They say "don't document the code, code the documentation", and we are doing this more and more. We just tend to forget to remove the obsolete comments. Note I'm also removing a line of text in a few cases when it's very short and literally says the same as the method name. Again, such comments add zero new information. Change-Id: I01535404bab458c6c47e48e5456403b7a64198ed
* filebackend: Remove dependencies on MediaWiki codedaniel2025-01-083-8/+23
| | | | | | | | | | | | Why: - code under include/libs must not depend on MediaWiki code What: - replaced DeferredUpdates with a callback - replaced MWTimestamp with ConvertibleTimestamp Bug: T382910 Change-Id: I85bc06364af65bdd5a4e16474ad8a6f644d4f940
* filebackend: Improve debug messages by passing native context or CSVTimo Tijhof2025-01-074-20/+17
| | | | | | | | | | | | | | | | | | By passing arrays as native PSR3 context, the values end up as part of the structured debug logs in Logstash where they can be shown on-screen, selected, and copied into bug reports as their original values instead of with JSON-escaping for various characters. This is not a big deal today since FormatJson defaults to JSON_UNESCAPED_SLASHES (which is important because most of these values contain file paths that definitely include slashes). But, this will change as part of T382910 by removing use FormatJson, and inlining these gets tedious so instead pass it as native data where the data is non-trivial, and for others embed the paths within the string unchanged comma-separated. Bug: T382910 Change-Id: I79fba128a8129b9ac4da17b513b5da0b03d90bf9
* Replace isset() with null checksUmherirrender2024-12-191-2/+2
| | | | | | | | | isset() should only be used to suppress errors, not for null check. When the property is always defined, there is no need to use isset. Found by a new phan plugin (2efea9f989) https://www.mediawiki.org/wiki/Manual:Coding_conventions/PHP#isset Change-Id: Ib84b7d71e8308a36409f30ecfd16e9de149e97b3
* Add type hints to detect bool/null message paramsBartosz Dziewoński2024-11-141-6/+11
| | | | | | | | | | | | Add more precise type hints to every method I could find that takes message params as arguments, so that Phan will warn if bool or null is passed, which is deprecated (T378876). Errors found in this repository thanks to these checks are fixed in I286a4a51e879bdf61f65c87dc078621c51045bee. Bug: T378876 Change-Id: I3e0df790ff9db2fa630f82408a7254a359fe61ca
* FileRepo: Add support for the new Shellbox large file featureTim Starling2024-10-295-47/+150
| | | | | | | | | | | | | | | | | | | * Add FileBackend::addShellboxInputFile(), allowing Shellbox to read directly from a FileBackend. Add generic, FS and Swift implementations. * Add FileRepo wrapper, which takes a FileRepo virtual URL. * Add File::addToShellboxCommand(), which allows a File to be used as input in a Shellbox command. * Add configuration. * Extend FileBackend::getFileHttpUrl(), adding method and ipRange parameters. Unindent existing code. I was going to add support for PUT requests, but I reverted it due to the impossibility of supporting FileBackendMultiWrite. I left the method parameter in getFileHttpUrl(). Bug: T292322 Change-Id: If9487a0c9586065bf044b69ac04cc7a06b6e8856
* Use namespaced classesUmherirrender2024-10-211-1/+1
| | | | | | | Changes to the use statements done automatically via script Addition of missing use statement done manually Change-Id: I73fb416573f5af600e529d224b5beb5d2e3d27d3
* Use explicit nullable type on parameter argumentsUmherirrender2024-10-164-5/+5
| | | | | | | | | | | Implicitly marking parameter $... as nullable is deprecated in php8.4, the explicit nullable type must be used instead Created with autofix from Ide15839e98a6229c22584d1c1c88c690982e1d7a Break one long line in SpecialPage.php Bug: T376276 Change-Id: I807257b2ba1ab2744ab74d9572c9c3d3ac2a968e
* Add namespace to remaining parts of Wikimedia\FileBackendJames D. Forrester2024-10-0333-13/+253
| | | | | Bug: T353458 Change-Id: I49c843c9d8f6459c0fbf774afeea7a82fa564b59
* Add namespace to remaining parts of Wikimedia\ObjectCacheJames D. Forrester2024-09-272-0/+2
| | | | | Bug: T353458 Change-Id: I3b736346550953e3b2977c14dc3eb10edc07cf97
* Remove unchecked exception annotationsAdam Wight2024-09-173-4/+0
| | | | | | | | | | | | | | | Callers should not catch an unchecked exception, so it doesn't belong in a function signature. Unchecked exceptions indicate a coding error, which by definition the code will not be able to handle correctly. If any of these exceptions were supposed to be in response to an edge case, user input, or initial conditions, then they should be changed to a runtime error. If the exception class cannot be changed, then the annotation should include a comment explaining its purpose and prognosis. Bug: T240672 Change-Id: I2e640b9737cb68090a8e1cb70067d1b74037d647
* libs: Add missing documentation to class propertiesUmherirrender2024-09-131-1/+2
| | | | | | | | | | Add doc-typehints to class properties found by the PropertyDocumentation sniff to improve the documentation. Once the sniff is enabled it avoids that new code is missing type declarations. This is focused on documentation and does not change code. Change-Id: I46f46f1855ca32c89a276b06f4e2051ff541886e
* StreamFile: Support streaming webp from thumb_handler.phpFunc2024-09-031-1/+3
| | | | | Bug: T366422 Change-Id: Ib3ba7dccc10f8308fc706500c74a2beca357efdf
* Add namespace to the root classes of ObjectCacheEbrahim Byagowi2024-07-102-0/+4
| | | | | | | | | And deprecated aliases for the the no namespaced classes. ReplicatedBagOStuff that already is deprecated isn't moved. Bug: T353458 Change-Id: Ie01962517e5b53e59b9721e9996d4f1ea95abb51
* Add namespace and deprecation alias to MultiHttpClientEbrahim Byagowi2024-05-201-0/+1
| | | | | | | | | This patch introduces a namespace declaration for the Wikimedia\Http to MultiHttpClient and establishes a class alias marked as deprecated since version 1.43. Bug: T353458 Change-Id: I4ebc1a3b496de08b2b430301da376578d13fcfe6
* Add namespace and deprecation alias to FileBackendEbrahim Byagowi2024-05-1911-6/+32
| | | | | | | | | This patch introduces a namespace declaration for the Wikimedia\FileBackend to FileBackend and establishes a class alias marked as deprecated since version 1.43. Bug: T353458 Change-Id: Id897687b1d679fd7d179e3a32e617aae10ebff33
* Add namespace and deprecation alias to FormatJsonEbrahim Byagowi2024-05-164-0/+5
| | | | | | | | | This patch introduces a namespace declaration for the MediaWiki\Json to FormatJson and establishes a class alias marked as deprecated since version 1.43. Bug: T353458 Change-Id: I5e1311e4eb7a878a7db319b725ae262f40671c32
* Merge "Make HTTPFileStreamer testable"jenkins-bot2024-04-274-21/+64
|\
| * Make HTTPFileStreamer testabledaniel2024-04-264-21/+64
| | | | | | | | | | | | | | This also improves ThumbnailEntryPointTest by allowing it to assert which headers got sent. Change-Id: I33911775bce1b3cc7a53a03c2be50d53a28fabd1
* | Merge "Allow to set .rlistings on a container"jenkins-bot2024-04-261-0/+9
|\ \
| * | Allow to set .rlistings on a containerPaladox2024-04-171-0/+9
| |/ | | | | | | | | | | Needed for if you use large objects and you access the url anonymously. Change-Id: I9aa384f525b0747793bba0c0691cc36b6369ed9b
* / Fix some line indentUmherirrender2024-04-201-1/+1
|/ | | | Change-Id: I8f82724197d20f9289d80e138d80310f1eab29f2
* filebackend: simplify "headless" parameter and STREAM_HEADLESS flagAaron Schulz2024-04-043-5/+7
| | | | | | | | | | | | In FileBackend::streamFile(), do not send "headers" in "headless" mode. In HTTPFileStreamer::stream(), do not treat $headers as an exception. Just use $headerFunc like with all the other headers. Nothing relies on the old behavior, and the only things using "headless" are integration tests anyway. The same holds true for STREAM_HEADLESS. Change-Id: I75faf9195a713d6a8a8f1273cb9e747980f4d57f
* Update Apache config syntax in .htaccess filesKunal Mehta2024-03-231-1/+1
| | | | | | | | | | "Deny from all" is deprecated; the replacement syntax has been available since Apache 2.4 (originally released in 2012). See <https://httpd.apache.org/docs/2.4/howto/access.html>. Bug: T360850 Change-Id: I825053ccefe34f6ca4e04af5ad2601f79e4d51a7
* filebackend: Retry Swift requests with new auth token on 401Tim Starling2024-03-152-174/+261
| | | | | | | | | | | | | | | | | | | | | Every Wednesday, we get hundreds of user-visible storage failures, because there is only one global storage token and it has a lifetime of one week. The code assumes that requesting a token regularly will avoid expiry, but that is not the case. So, when a request fails with a 401, refresh the auth token, then retry the request. Factor out HTTP request handling to allow this. Swift gives us the remaining lifetime of the token in seconds, so use that as the server and process cache expiry time. There will be a stampede when it expires, but my laptop can do 700 req/s single- threaded, so it should be OK. Add tests to cover the cases previously handled by checking the return value of getAuthentication(). Bug: T358830 Change-Id: If9fe4dde4127592adae1b81e8eb925c4f59443d9
* filebackend: Fix MemoryFileBackend move resultTim Starling2024-03-132-15/+33
| | | | | | | | | | | | | | | | MemoryFileBackend, a class which is only used for testing, does not correctly respect the ignoreMissingSource option to its move operation. The copy succeeds due to the forwarded ignoreMissingSource option, so it proceeds with the source deletion, which fails. This was implemented in the base class method FileBackendStore ::doMoveInternal, but every other subclass was overriding it. So, make it abstract, by analogy with doCopyInternal and doDeleteInternal. Add an implementation specific to MemoryFileBackend which handles ignoreMissingSource correctly. Change-Id: Ib5733046d741b962c65db42b5364705b073c4b06
* Merge "filebackend: Fix Swift integration test failures"jenkins-bot2024-03-121-1/+1
|\
| * filebackend: Fix Swift integration test failuresTim Starling2024-03-121-1/+1
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | It's possible to run FileBackendIntegrationTest against an actual Swift server, but there were a few failures when I tried it: * In SwiftFileBackend::getDirListPageInternal(), fix a PHP warning due to strpos(null). * In setUp(), fix exception "Backend domain ID not provided" due to missing domainId parameter. * In doTestGetLocalCopyAndReference404(), getLocalCopy() on a sharded backend for a file that cannot exist, due to having no shard, returns null not false, which I think is fine. Use a path that can exist so that the assertion passes. * In doTestGetFileList(), there is an assertion to confirm that getFileList() on a non-existent container will return null, but in Swift this is not known until iteration begins, so it returns an iterator. Fix the assertion. Change-Id: If3fc47eb285b51366e7cfd8b6cf6e64bd0f8cd41
* | filebackend: Clean up batch handling in doQuickOperationsInternal()Tim Starling2024-03-081-15/+10
|/ | | | | | | | | | | Remove $fileOpHandles since it was effectively no longer used. Rename $curFileOpHandles to $batch and note its type. Remove unnecessary comments and improve the comment about T230245 by explaining what was going on with that bug. Followup to I67f9f25b3bc68a389c49c39b7b6def2343bda34e. Change-Id: I809afc7ce4f2f32018c975ec0073cd8a4af21763
* Respect $maxConcurrency when queuing async FileOpsPlatonides2024-03-071-1/+3
| | | | | | | | | Too many queued FileOps may end up causing 'Too many open files' failures, and mysterious error messages claiming that existing files were not found. Bug: T230245 Change-Id: I67f9f25b3bc68a389c49c39b7b6def2343bda34e
* Merge "filebackend: improve documentation of FileOp fields"jenkins-bot2024-03-044-10/+12
|\
| * filebackend: improve documentation of FileOp fieldsAaron Schulz2024-02-214-10/+12
| | | | | | | | | | | | Add comments to the STATE_* constants and rename cancelled -> noOp. Change-Id: I17fd8bd296e8845fcddd4e9676d100fbb82b5e6d
* | filebackend: add FileStatePredicates helper class for file operationsAaron Schulz2024-02-219-162/+298
|/ | | | | | | | | | | | | | | | | This replaces the old pass-by-reference $predicates array and makes the intent clearer. It also supports lazy-loading of predicated properties, which useful for avoiding expensive computations like SHA-1 hashes when they end up not being needed. Pass FileOp::doPrecheck() a mutable "batch level" FileStatePredicates instance along with an immutable "operation level" FileStatePredicates instance. The later can be used in callbacks that need to recall the predicated file states just before the start of the operation. Make FileOp::precheckDestExistence() support lazy size/sha1 closures. Bug: T348294 Change-Id: I709c1920d2bee4b76f69e38c1643a44a73d63265
* build: Upgrade mediawiki/mediawiki-phan-config from 0.13.0 to 0.14.0 manuallyJames D. Forrester2024-02-101-1/+0
| | | | | | | | | | | * Switch out raw Exceptions, mostly for InvalidArgumentExceptions. * Fake exceptions triggered to give Monolog a backtrace are for some reason "traditionally" RuntimeExceptions, instead, so we continue to use that pattern in remaining locations. * Just entirely give up on PostgresResultWrapper's resource vs. object mess. * Drop now-unneeded false positive hits. Change-Id: Id183ab60994cd9c6dc80401d4ce4de0ddf2b3da0
* Merge "Use thousands separators in selected integer literals"jenkins-bot2023-12-121-1/+1
|\
| * Use thousands separators in selected integer literalsTim Starling2023-12-121-1/+1
| | | | | | | | | | | | | | | | | | | | For readability. Allowed since PHP 7.4. I searched for integer literals of 6 or more digits, and also changed some nearby smaller numbers for consistency. Bug: T353205 Change-Id: I8518e04889ba8fd52e0f9476a74f8e3e1454b678
* | SwiftFileBackend: Fix "PHP Deprecated: strlen(): Passing null to parameter ↵Paladox2023-12-111-2/+2
|/ | | | | | #1 ($string) of type string is deprecated" Change-Id: Ie4023ba59653c749f717356a2b6d9d52537d8e35
* Namespace remaining files under includes/deferredJames D. Forrester2023-11-221-0/+1
| | | | | Bug: T166010 Change-Id: Ibd40734b96fd2900e3ce12239d09becfb4150059
* filebackend: Allow uploading files up to 32 GB with FSFileBackendBrian Wolff2023-10-252-2/+7
| | | | | | | | | | | | | | | | | | | | | | | | The SwiftBackend theoretical max file size was increased to 5 GB Sysadmins of MediaWiki are in a better position to judge what is acceptable performance-wise for uploaded files than we are. The answer presumably depends significantly on the hardware mediawiki is running under. This increases the max limit to 32 GB (previously 4GB). With files this big, operations such as copying and calculating SHA1 do become expensive. However everything still works, albeit with some delay (I tested on a laptop with HDD, I imagine things are significantly faster with an NVMe drive). This changes the overall FileBackendStore limit which affects all subclasses that haven't overriden the limit, including those from extensions. Swift was set to 5GB as that backend has to change the way it uses the Swift API for objects larger than 5GB. This should not affect ordinary users as $wgMaxUploadSize is by default only 100 MB in MediaWiki (4GB in Wikimedia). Bug: T191805 Change-Id: Id091d51b620edbeea2d9df8c75422807bfc7aea0
generated by cgit v1.2.3 (git 2.25.1) at 2025-04-24 19:40:29 +0000