aboutsummaryrefslogtreecommitdiffstats
path: root/includes/filerepo
Commit message (Collapse)AuthorAgeFilesLines
...
* In FileBackend:Aaron Schulz2012-01-255-8/+16
| | | | | | | | | | | | * Use 'b' param in some fopen() calls as needed for Windows and newline handling. * Removed some useless padding code in FileBackend::getContainerShard(). Initialized $m to make IDE happy. * Updated some code comments. In SwiftFileBackend: * Manually set the ETag when using php-cloudfiles for creating files to avoid https://github.com/rackspace/php-cloudfiles/issues/59. * Manually set the content type based on how StreamFile::getType(). This makes it safe to read files directly out of the proxy to end-users. The streamFile() backend functions already uses a similar content-type check. Notes: http://mediawiki.org/wiki/Special:Code/MediaWiki/109980
* r109904: worked around PHP < 5.3 suckage to keep this compatibleAaron Schulz2012-01-241-17/+27
| | | | Notes: http://mediawiki.org/wiki/Special:Code/MediaWiki/109938
* Made FileOp classes enforce required params. Also reverts r109823.Aaron Schulz2012-01-241-29/+29
| | | | Notes: http://mediawiki.org/wiki/Special:Code/MediaWiki/109904
* * Moved FSFile classes from file/ to backend/Aaron Schulz2012-01-242-0/+0
| | | | | | | * Improved $this->lockCount check in LockServerDaemon Notes: http://mediawiki.org/wiki/Special:Code/MediaWiki/109901
* reverts r109814 r109815Antoine Musso2012-01-232-12/+7
| | | | | | | | | | | | * Ternary operator avoided a call to wfWikiID() * We really want to explicitly choose the manager use LockManagerGroup->get( 'nullLockManager' ) * defaultConfig no more needed if it handles just one parameter that we want to set explicitly Notes: http://mediawiki.org/wiki/Special:Code/MediaWiki/109855
* FileRepo: check constructor parametersAntoine Musso2012-01-231-1/+10
| | | | | | | | | | | new FileRepo() requires an array of parameters having at least the 'name' and 'backend' key setup. TODO: 'backend' keyword should probably default to FileBackend. Notes: http://mediawiki.org/wiki/Special:Code/MediaWiki/109824
* FileOp: allow creation of an empty fileAntoine Musso2012-01-231-2/+2
| | | | | | | | | | When creating a file but ommitting the 'content' key, it would throw a notice. Using getParams let us forget about the 'content' key and hence create files with no content. Notes: http://mediawiki.org/wiki/Special:Code/MediaWiki/109823
* cleanly handle wikiId on FileBackendBase constructionAntoine Musso2012-01-231-4/+2
| | | | | | | | Use the default array added by r109814 Notes: http://mediawiki.org/wiki/Special:Code/MediaWiki/109815
* get a null lock manager by defaultAntoine Musso2012-01-232-3/+10
| | | | Notes: http://mediawiki.org/wiki/Special:Code/MediaWiki/109814
* * Added some wfProfileIn() calls to file backend code.Aaron Schulz2012-01-232-31/+71
| | | | | | | | | * Made FileBackend::getFileProps() final. * Added exception needed in SwiftFileBackend::getConnection(). * Various FileBackendTests fixes and additions. Optimized it a bit by keeping the backend instance in memory. Notes: http://mediawiki.org/wiki/Special:Code/MediaWiki/109809
* Simplified last_modified timestamp conversion in SwiftFileBackend to just ↵Aaron Schulz2012-01-221-11/+6
| | | | | | | use wfTimestamp() Notes: http://mediawiki.org/wiki/Special:Code/MediaWiki/109719
* * Another fix for SwiftFileBackend file listingsAaron Schulz2012-01-221-2/+2
| | | | | | | * Fixed bogus array key reference in SwiftFileBackend constructor Notes: http://mediawiki.org/wiki/Special:Code/MediaWiki/109706
* * Fixed bugs in SwiftFileBackend file listing codeAaron Schulz2012-01-223-4/+16
| | | | | | | | * Clear container cache in SwiftFileBackend when clearCache() is called * Updated a comment and fixed a comment typo Notes: http://mediawiki.org/wiki/Special:Code/MediaWiki/109703
* * r109659: actually return the exact type we say we doAaron Schulz2012-01-212-7/+10
| | | | | | | * Made use of FileRepo::cleanupBatch() in ForeignAPIFile::purgeThumbnails() Notes: http://mediawiki.org/wiki/Special:Code/MediaWiki/109664
* * Added FileBackend::parentStoragePath() convenience function for getting ↵Aaron Schulz2012-01-201-0/+14
| | | | | | | | | | parent directories. * In StoreBatchTest: sed proper clean() function to remove temp dirs (follows up r109641). Also removed some commented out code. * Fixed temp dir leakage in FileBackendTest. Notes: http://mediawiki.org/wiki/Special:Code/MediaWiki/109659
* Make sure FSFileBackend clears the php stat cache in clearCache(). Otherwise ↵Aaron Schulz2012-01-202-1/+19
| | | | | | | it will just clear the FileBackend cache and refetch the same cached data from PHP's stat cache. Notes: http://mediawiki.org/wiki/Special:Code/MediaWiki/109652
* * Follow-up r109009: Check that paths are usable in FileOp::doPrecheck(). ↵Aaron Schulz2012-01-195-47/+118
| | | | | | | | | | | | Also lock parent directories to avoid prepare()/clean() race conditions for FS backends. * Fixed bogus $params var in logException() call in SwiftFileBackend. * Added 'latest' param to FileBackendMultliWrite::consistencyCheck(). * Dummy-proof FileBackend::getFileStat() w.r.t the 'latest' param and removed related FileOp::allowStaleReads() comment. * Tweaked backend-fail-batchsize message from r109469. Notes: http://mediawiki.org/wiki/Special:Code/MediaWiki/109583
* Renamed 'overwriteDest' FileBackend operation parameter to just 'overwrite', ↵Aaron Schulz2012-01-196-27/+27
| | | | | | | which is shorter and more consistent with 'overwriteSame' Notes: http://mediawiki.org/wiki/Special:Code/MediaWiki/109473
* In FileBackend/FileOp:Aaron Schulz2012-01-192-4/+87
| | | | | | | | | * Added a sane default max file size to FileBackend. Operation batches need to check this before trying anything. * Temporarily adjust the PHP execution time limit in attemptBatch() to reduce the chance of dying in the middle of it. Also added a maximum batch size limit. * Added some code comments. Notes: http://mediawiki.org/wiki/Special:Code/MediaWiki/109469
* In SwiftFileBackend:Aaron Schulz2012-01-181-129/+182
| | | | | | | | | | | | | | | | * r108944: doCleanInternal() should delete empty containers only if the container dir was given. * Made doSecureInternal() set container permissions. Also renamed swiftProxyUser -> swiftAnonUser. * Made doGetFileStat() respect the 'latest' parameter (using r109235). * Fixed connTTL default and renamed it to authTTL. Also added explicit close() call to getConnection(). * Reduced RTTs in doPrepareInternal() by checking getContainer() first (which is process cached). * Killed an RTT in doStoreInternal(), doCreateInternal(), and doCopyInternal() by using create_object(). Also cleaned up logic with regards to the destination CF_Object object fields getting preloaded before write(). * Cleanups to getLocalCopy(); only create the tmp file if get_object() succeeds to short-circuits things. * Made getContainer() limit the container cache size for sanity. * Simplified doDeleteInternal() code a bit. * Renamed $destRel => $dstRel for consistency. Notes: http://mediawiki.org/wiki/Special:Code/MediaWiki/109428
* * Fixed type check in StreamFile::prepareForStream() for 404sAaron Schulz2012-01-172-11/+12
| | | | | | | * Code comment tweaks and fixes Notes: http://mediawiki.org/wiki/Special:Code/MediaWiki/109106
* Follow-up r108185: removed now unused ALLOW_STALE paramAaron Schulz2012-01-161-5/+0
| | | | Notes: http://mediawiki.org/wiki/Special:Code/MediaWiki/109071
* In FileOp/FileBackend:Aaron Schulz2012-01-152-412/+125
| | | | | | | | | | | | * Added SHA-1 to FileOp::attemptBatch $predicates (since concatenate was removed from FileOp). This lets us get rid of temp file backups, as the remaining failure case is just the backend medium going down (the ops would break in that case anyway). Doing so reduces RTTs and backup file I/O overhead. This also simplifies expiry object support by avoiding having to stash the expiry values of temp backup objects somewhere. * Improved precheck() and attempt() status logic in FileOp::attemptBatch() a bit. Use separate $subStatus var to check if each op is OK. * A few minor code cleanups and comment tweaks. * Fixed MoveFileOp bug found in unit tests when the source/dest are the same and an overwriteDest/overwriteSame param is given (the file would just get deleted). Improved CopyFileOp in this case too. Other: * Added more unit tests. Notes: http://mediawiki.org/wiki/Special:Code/MediaWiki/109009
* * Killed an RTT in SwiftFileBackend::doStreamFile(). We already check ↵Aaron Schulz2012-01-151-10/+7
| | | | | | | | | existence in FileBackend::streamFile() * A few minor code cleanups and comment tweaks. Notes: http://mediawiki.org/wiki/Special:Code/MediaWiki/109007
* r108944: fixed bogus field...php-cloudfiles documentation misspelled this, ↵Aaron Schulz2012-01-141-1/+1
| | | | | | | it's 'object_count' not 'count' Notes: http://mediawiki.org/wiki/Special:Code/MediaWiki/108945
* In SwiftFileBackend:Aaron Schulz2012-01-141-95/+127
| | | | | | | | | | * Added getContainer(), createContainer(), and deleteContainer() helper functions, which cache container objects to avoid RTTs * Made doCleanInternal() delete empty containers * Bumped authentication TTL to 120 seconds * Some comment tweaks Notes: http://mediawiki.org/wiki/Special:Code/MediaWiki/108944
* * Fixed bogus var in LSLockManager.Aaron Schulz2012-01-143-2/+6
| | | | | | | | * Made DBLockManager constructor a bit more convenient to use with just 'localDBMaster' in the db buckets. * Documentation tweaks. Notes: http://mediawiki.org/wiki/Special:Code/MediaWiki/108891
* * Renamed ContainerShardListIterator -> FileBackendShardListIteratorAaron Schulz2012-01-143-13/+9
| | | | | | | | | * Renamed FSFileIterator -> FSFileBackendFileList * Renamed SwiftFileIterator -> SwiftFileBackendFileList and added it to autoloader * A few minor doc fixes Notes: http://mediawiki.org/wiki/Special:Code/MediaWiki/108889
* Fixed annoying bug in FileRepo::deleteBatch() when a file is deleted that ↵Aaron Schulz2012-01-141-12/+8
| | | | | | | has multiple identical versions. The operation would fail due to the file already existing. Notes: http://mediawiki.org/wiki/Special:Code/MediaWiki/108886
* In FileBackend:Aaron Schulz2012-01-1310-28/+18
| | | | | | | | | | | | | * Made secure() call doPrepare() for caller dummy proofing (especially those that don't check the status). In FSFileBackend: * Removed redundant wfMkdirParents() calls in FSFileBackend, we already have prepare() for this purpose. This also keeps it's behavior more consistent with the other backends. * Made use of 'backend-fail-store' message in doStoreInternal(). Other: * Updated unit tests and renamed $src => $source in some functions for consistency. * Added some documentation comments and @since tags. Notes: http://mediawiki.org/wiki/Special:Code/MediaWiki/108878
* * Distinguish "does not exist" from failure in FSFileBackend::doGetFileStat().Aaron Schulz2012-01-132-5/+34
| | | | | | | | * Added clearstatcache() to doConcatenate() to make it safe for callers to stat the file. * A few minor comment tweaks. Notes: http://mediawiki.org/wiki/Special:Code/MediaWiki/108791
* A couple more for r108785Sam Reed2012-01-131-1/+1
| | | | | | | More doucmentations Notes: http://mediawiki.org/wiki/Special:Code/MediaWiki/108786
* * Collapse multiple directory separators in ↵Aaron Schulz2012-01-123-18/+45
| | | | | | | | | | | FileBackend::normalizeStoragePath(). Also renamed the function to normalizeContainerPath() to be less confusing. * Made SwiftFileBackend respect the 'latest' parameter of various functions via "X-Newest: true" header. Added to TODO in the doGetFileStat() function, as cloudfiles needs upstream changes for that. * Fixed SwiftFileIterator to return paths relative to the given directory. * Clean up for FSFileBackend trailing slash handling. Notes: http://mediawiki.org/wiki/Special:Code/MediaWiki/108766
* * Added backend name to 'backend-fail-internal' messageAaron Schulz2012-01-121-14/+13
| | | | | | | * Removed TODO comment in SwiftFileBackend Notes: http://mediawiki.org/wiki/Special:Code/MediaWiki/108750
* * Merged (added) SwiftFileBackend class from branch.Aaron Schulz2012-01-121-0/+697
| | | | | | | * Added i18n messages used by the new class. Notes: http://mediawiki.org/wiki/Special:Code/MediaWiki/108749
* Reverted r108743 per CR comment. This should at least be discussed first.Aaron Schulz2012-01-129-2/+28
| | | | Notes: http://mediawiki.org/wiki/Special:Code/MediaWiki/108745
* * Fixed a bit of bogus code left in r108740Aaron Schulz2012-01-121-3/+4
| | | | | | | * Added some class comments Notes: http://mediawiki.org/wiki/Special:Code/MediaWiki/108744
* Kill "* @return void"Sam Reed2012-01-129-28/+2
| | | | Notes: http://mediawiki.org/wiki/Special:Code/MediaWiki/108743
* * Added 'basePath' config param to FSFileBackend and tweaked it to behave ↵Aaron Schulz2012-01-121-35/+92
| | | | | | | | | more similarly to the backends with real containers (e.g. everything else). Resolution to absolute paths is now deferred after resolveStoragePath(), using resolveToFSPath(). * Fixed whitespace in FSFileIterator. Notes: http://mediawiki.org/wiki/Special:Code/MediaWiki/108740
* r108353: Distinguish null/false in FileBackend::fileExists(). This is ↵Aaron Schulz2012-01-101-5/+10
| | | | | | | intended for things that might really care. Notes: http://mediawiki.org/wiki/Special:Code/MediaWiki/108488
* Removed debug code from r108472Aaron Schulz2012-01-091-1/+0
| | | | Notes: http://mediawiki.org/wiki/Special:Code/MediaWiki/108473
* Fix r108185: file transform store op needs 'overwriteDest'Aaron Schulz2012-01-091-1/+2
| | | | Notes: http://mediawiki.org/wiki/Special:Code/MediaWiki/108472
* * Fixed FileOp::attemptBatch() bug that had status->ok set wrong if 'force' ↵Aaron Schulz2012-01-092-5/+14
| | | | | | | | | | option was used. * Added missing prepare() call to File::maybeDoTransform(). * Improved documentation. Notes: http://mediawiki.org/wiki/Special:Code/MediaWiki/108471
* filerepo: always use realpath()Antoine Musso2012-01-091-1/+2
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | The suffix to strip was determined using realpath() but applied on the given path. This could lead to errors when the path is made of a symbolic link or use consecutives slashes. Example on Mac OS: $ php -a php > print realpath( '/private///tmp' ); /private/tmp php > print realpath( '///tmp///////' ); /private/tmp php > $ Fix FileBackendTest::testGetFileList test which failed with: 1) FileBackendTest::testGetFileList Correct file listing (FSFileBackend). Failed asserting that two arrays are equal. --- Expected +++ Actual @@ @@ Array ( - 0 => 'subdir1/test1.txt' - 1 => 'subdir1/test2.txt' - 2 => 'subdir2/subdir/sub/120-px-file.txt' - 3 => 'subdir2/subdir/sub/test0.txt' - 4 => 'subdir2/subdir/test1.txt' - 5 => 'subdir2/subdir/test2.txt' - 6 => 'subdir2/subdir/test3.txt' - 7 => 'subdir2/subdir/test4.txt' - 8 => 'subdir2/subdir/test5.txt' - 9 => 'subdir2/test3.txt' - 10 => 'subdir2/test4.txt' - 11 => 'test1.txt' - 12 => 'test2.txt' - 13 => 'test3.txt' + 0 => '/subdir/sub/120-px-file.txt' + 1 => '/subdir/sub/test0.txt' + 2 => '/subdir/test1.txt' + 3 => '/subdir/test2.txt' + 4 => '/subdir/test3.txt' + 5 => '/subdir/test4.txt' + 6 => '/subdir/test5.txt' + 7 => '/test1.txt' + 8 => '/test2.txt' + 9 => '/test3.txt' + 10 => '/test4.txt' + 11 => 'xt' + 12 => 'xt' + 13 => 'xt' ) Notes: http://mediawiki.org/wiki/Special:Code/MediaWiki/108408
* * r107986: Added readOnly checks to prepare(), secure(), and clean() in ↵Aaron Schulz2012-01-093-46/+78
| | | | | | | | | FileBackendBase. * Added some prepare()/clean() tests. Notes: http://mediawiki.org/wiki/Special:Code/MediaWiki/108377
* * Fixed 'success' value of doOperations() Status to match documentation.Aaron Schulz2012-01-082-26/+83
| | | | | | | | * Made 'success', 'successCount', and 'failCount' fields reflect the overall operation in FileBackendMultiWrite::doOperationsInternal(). This makes it match up with single-write backends. * Made FileBackend::clearCache() part of the public API. Notes: http://mediawiki.org/wiki/Special:Code/MediaWiki/108369
* * Follow-up r107170: Moved FileBackend::concatenate() outside of ↵Aaron Schulz2012-01-084-104/+54
| | | | | | | | | | doOperations() as it's own separate operation. It does not mutate storage files like the others and having it in doOperations() broke FileBackendMultiWrite. This change also makes overriding doOperationsInternal() easier (suching as using a custom batch operation storage API). * Added sanity check to FileBackendMultiWrite constructor. * Moved FileBackend::create() function up a bit. Notes: http://mediawiki.org/wiki/Special:Code/MediaWiki/108355
* In FileBackend:Aaron Schulz2012-01-086-181/+250
| | | | | | | | | | | | | | | | | | * Added getFileSize()/getFileStat() functions. Refactored some functions to use the stat function for better reuse and caching/consistency. * Refactored streamFile() to allow for subclasses to avoid local file copying with less duplication. Also make last-modified check actually work since we always get the timestamp of the original file. * Renamed 'ignoreErrors' parameter to 'force'. In FileBackendMultiWrite: * Simplified how read ops are done (use 'master' backend for consistency). * Added consistency check to doOperationsInternal() to check if the files are synced. * Various fixes after testing. In StreamFile: * Split out prepareForStream() function from stream() in StreamFile for code reuse. In FileBackendTest: * Properly cover FileBackendMultiWrite in tests. * Various test improvements. Notes: http://mediawiki.org/wiki/Special:Code/MediaWiki/108353
* * Fixed bogus dollar signs left in $tmpGlobals array keys in r108300.Aaron Schulz2012-01-074-1/+17
| | | | | | | * Various documentation cleanups. Notes: http://mediawiki.org/wiki/Special:Code/MediaWiki/108304
* Partially reverted bits from r108300. Not allowing underscores is too ↵Aaron Schulz2012-01-071-5/+19
| | | | | | | restrictive and breaks on default DB name value suggested by installer. Azure can use resolveContainerName() to enforce its extra restrictions. A new resolveWikiId() function was added to allow for it to mangle the wiki ID by translating underscores. Also restored previous size limit in isValidContainerName() since it makes sense to either go all the way lowest-common-denominator or not here (in this case not). Preserved "first char is a alphanumeric" check since it's pretty weird to have anything else. Notes: http://mediawiki.org/wiki/Special:Code/MediaWiki/108303
generated by cgit v1.2.3 (git 2.25.1) at 2025-04-23 01:10:26 +0000