Add `phpunit:prepare-parallel:extensions` command

In T361190 and Quibble 1.9.0, we introduced parallel execution of
PHPUnit tests to speed up the CI jobs. The existing implementation
is purely Python/Quibble, and cannot directly be used by developers
locally. With this patch, we re-implement the test splitting logic
already implemented in CI as a composer task so that the parallel
tests can be run locally.

There are a couple of different approaches to running PHPUnit tests
in parallel. The different approaches have been discussed at length
in T50217. Ideally, we would just install the `paratest` extension
and use that to parallelise the execution. Unfortunately we have
complex test suites (specifically Parser tests and the Scribunto
test suite) that dynamically create tests as they run, which makes
it hard for `paratest` to work out which tests will run.

To overcome this limitation, we use the `phpunit --list-tests`
function to create a list of test classes that would be included in
the execution of the test suite, then scan the filesystem for
classes named in the `tests-list.xml` output. The classes we find
are then collected into smaller groups (`split_group_X`) which we
can run in parallel in separate processes.

We split into 7-8 groups here, as that experimentally leads to an
even spread of the tests and consumes 100% of all cores on a 4-core
processor.

Because `ParserIntegrationTest.php` is a single test class that
generates thousands of integration tests, we put that in its own
bucket rather than allocating it round-robin to one of the split
buckets. This again helps to keep the buckets roughly the same size.

The current implementation only supports splitting the `extensions`
test suite. We need to do some more development and testing to
support splitting other suites.

The new composer command `phpunit:prepare-parallel:extensions` will
generate a `phpunit.xml` file with the same contents as
`phpunit.xml.dist`, but with the split-group suites added. The
result of running all of the split groups should be the same as the
result of running the whole test suite.

Bug: T365976
Change-Id: I2d841ab236c5367961603bb526319053551bec2e

19 files changed, 870 insertions, 0 deletions

diff --git a/autoload.php b/autoload.php
index 7abec9ec6ecd..7de115bc78e2 100644
--- a/autoload.php
+++ b/autoload.php
@@ -972,6 +972,17 @@ $wgAutoloadLocalClasses = [
 	'MediaWiki\\Composer\\ComposerPhpunitXmlCoverageEdit' => __DIR__ . '/includes/composer/ComposerPhpunitXmlCoverageEdit.php',
 	'MediaWiki\\Composer\\ComposerVendorHtaccessCreator' => __DIR__ . '/includes/composer/ComposerVendorHtaccessCreator.php',
 	'MediaWiki\\Composer\\LockFileChecker' => __DIR__ . '/includes/composer/LockFileChecker.php',
+	'MediaWiki\\Composer\\PhpUnitSplitter\\MissingNamespaceMatchForTestException' => __DIR__ . '/includes/composer/PhpUnitSplitter/MissingNamespaceMatchForTestException.php',
+	'MediaWiki\\Composer\\PhpUnitSplitter\\PhpUnitTestFileScanner' => __DIR__ . '/includes/composer/PhpUnitSplitter/PhpUnitTestFileScanner.php',
+	'MediaWiki\\Composer\\PhpUnitSplitter\\PhpUnitTestFileScannerFilter' => __DIR__ . '/includes/composer/PhpUnitSplitter/PhpUnitTestFileScannerFilter.php',
+	'MediaWiki\\Composer\\PhpUnitSplitter\\PhpUnitTestListProcessor' => __DIR__ . '/includes/composer/PhpUnitSplitter/PhpUnitTestListProcessor.php',
+	'MediaWiki\\Composer\\PhpUnitSplitter\\PhpUnitXml' => __DIR__ . '/includes/composer/PhpUnitSplitter/PhpUnitXml.php',
+	'MediaWiki\\Composer\\PhpUnitSplitter\\PhpUnitXmlManager' => __DIR__ . '/includes/composer/PhpUnitSplitter/PhpUnitXmlManager.php',
+	'MediaWiki\\Composer\\PhpUnitSplitter\\SuiteGenerationException' => __DIR__ . '/includes/composer/PhpUnitSplitter/SuiteGenerationException.php',
+	'MediaWiki\\Composer\\PhpUnitSplitter\\TestDescriptor' => __DIR__ . '/includes/composer/PhpUnitSplitter/TestDescriptor.php',
+	'MediaWiki\\Composer\\PhpUnitSplitter\\TestListMissingException' => __DIR__ . '/includes/composer/PhpUnitSplitter/TestListMissingException.php',
+	'MediaWiki\\Composer\\PhpUnitSplitter\\TestSuiteBuilder' => __DIR__ . '/includes/composer/PhpUnitSplitter/TestSuiteBuilder.php',
+	'MediaWiki\\Composer\\PhpUnitSplitter\\UnlocatedTestException' => __DIR__ . '/includes/composer/PhpUnitSplitter/UnlocatedTestException.php',
 	'MediaWiki\\Composer\\VersionChecker' => __DIR__ . '/includes/composer/VersionChecker.php',
 	'MediaWiki\\Config\\Config' => __DIR__ . '/includes/config/Config.php',
 	'MediaWiki\\Config\\ConfigException' => __DIR__ . '/includes/config/ConfigException.php',
diff --git a/composer.json b/composer.json
index 1ec1a6a3f8a6..d106a5d975aa 100644
--- a/composer.json
+++ b/composer.json
@@ -179,6 +179,12 @@
 		"phpunit:coverage": "@phpunit --testsuite=core:unit --exclude-group Dump,Broken",
 		"phpunit:coverage-edit": "MediaWiki\\Composer\\ComposerPhpunitXmlCoverageEdit::onEvent",
 		"phpunit:entrypoint": "@phpunit",
+		"phpunit:prepare-parallel:extensions": [
+			"MediaWiki\\Composer\\PhpUnitSplitter\\PhpUnitXmlManager::listTestsNotice",
+			"@phpunit --list-tests-xml=tests-list.xml --testsuite=extensions",
+			"MediaWiki\\Composer\\PhpUnitSplitter\\PhpUnitXmlManager::splitTestsList"
+		],
+		"phpunit:prepare-parallel:split-file": "MediaWiki\\Composer\\PhpUnitSplitter\\PhpUnitXmlManager::splitTestsCustom",
 		"maintenance": "@php maintenance/run.php"
 	},
 	"config": {
diff --git a/includes/composer/PhpUnitSplitter/MissingNamespaceMatchForTestException.php b/includes/composer/PhpUnitSplitter/MissingNamespaceMatchForTestException.php
new file mode 100644
index 000000000000..71292d18a2e4
--- /dev/null
+++ b/includes/composer/PhpUnitSplitter/MissingNamespaceMatchForTestException.php
@@ -0,0 +1,19 @@
+<?php
+
+declare( strict_types = 1 );
+
+namespace MediaWiki\Composer\PhpUnitSplitter;
+
+use Exception;
+
+/**
+ * @license GPL-2.0-or-later
+ */
+class MissingNamespaceMatchForTestException extends Exception {
+	public function __construct( TestDescriptor $testDescriptor ) {
+		parent::__construct(
+			"Could not match " . $testDescriptor->getFullClassname() . " to a namespace in a php test file"
+		);
+	}
+
+}
diff --git a/includes/composer/PhpUnitSplitter/PhpUnitTestFileScanner.php b/includes/composer/PhpUnitSplitter/PhpUnitTestFileScanner.php
new file mode 100644
index 000000000000..7e1dc9f20038
--- /dev/null
+++ b/includes/composer/PhpUnitSplitter/PhpUnitTestFileScanner.php
@@ -0,0 +1,46 @@
+<?php
+
+declare( strict_types = 1 );
+
+namespace MediaWiki\Composer\PhpUnitSplitter;
+
+use RecursiveDirectoryIterator;
+use RecursiveIteratorIterator;
+
+/**
+ * @license GPL-2.0-or-later
+ */
+class PhpUnitTestFileScanner {
+
+	private string $rootDir;
+
+	public function __construct( string $rootDir ) {
+		$this->rootDir = $rootDir;
+	}
+
+	/**
+	 * @return array Returns an list of `.php` files found on the filesystem
+	 * inside `$rootDir`. The array maps file basenames (i.e.
+	 * `MyClassTest.php`) to lists of paths where that basename
+	 * is found (e.g. `[ 'tests/phpunit/MyClassTest.php',
+	 * 'extensions/MyExtension/tests/phpunit/MyClassTest.php' ]`)
+	 */
+	public function scanForFiles(): array {
+		$phpFiles = [];
+		$iterator = new RecursiveIteratorIterator(
+			new PhpUnitTestFileScannerFilter(
+				new RecursiveDirectoryIterator( $this->rootDir )
+			)
+		);
+		foreach ( $iterator as $file ) {
+			if ( $file->isFile() && $file->getExtension() === 'php' ) {
+				$filename = $file->getFilename();
+				if ( !array_key_exists( $filename, $phpFiles ) ) {
+					$phpFiles[$filename] = [];
+				}
+				$phpFiles[$filename][] = $file->getPathname();
+			}
+		}
+		return $phpFiles;
+	}
+}
diff --git a/includes/composer/PhpUnitSplitter/PhpUnitTestFileScannerFilter.php b/includes/composer/PhpUnitSplitter/PhpUnitTestFileScannerFilter.php
new file mode 100644
index 000000000000..3411e63e2098
--- /dev/null
+++ b/includes/composer/PhpUnitSplitter/PhpUnitTestFileScannerFilter.php
@@ -0,0 +1,50 @@
+<?php
+
+declare( strict_types = 1 );
+
+namespace MediaWiki\Composer\PhpUnitSplitter;
+
+use RecursiveFilterIterator;
+
+/**
+ * @license GPL-2.0-or-later
+ */
+class PhpUnitTestFileScannerFilter extends RecursiveFilterIterator {
+
+	/**
+	 * @var string[] list of folders and files to skip. We want to avoid
+	 *               loading PHP files from the vendor folder since that's
+	 *               not our code. ParserIntegrationTest is a complex suite
+	 *               that we can't handle in the usual way, so we will add
+	 *               that to a suite on its own, manually, as required.
+	 *               Likewise `LuaSandbox\\SandboxTest` from the Scribunto
+	 *               extension is a dynamic suite that generates classes
+	 *               during `--list-tests-xml` which we can't run on their
+	 *               own. We skip it in the scan and add it manually back
+	 *               later.
+	 *               `LuaEngineTestSkip` is an empty test case
+	 *               generated by Scribunto's dynamic test suite when a
+	 *               particular Lua engine isn't available. Since this is
+	 *               just an empty test suite with a skipped test, we can
+	 *               filter this from the list of test classes.
+	 * @see T345481
+	 */
+	private const IGNORE = [
+		"vendor",
+		"ParserIntegrationTest.php",
+		"SandboxTest.php",
+		"LuaEngineTestSkip.php"
+	];
+
+	public function accept(): bool {
+		$filename = $this->current()->getFilename();
+		if ( $filename[0] === '.' ) {
+			return false;
+		}
+		if ( in_array( $filename, self::IGNORE ) ) {
+			return false;
+		}
+		return true;
+	}
+
+}
diff --git a/includes/composer/PhpUnitSplitter/PhpUnitTestListProcessor.php b/includes/composer/PhpUnitSplitter/PhpUnitTestListProcessor.php
new file mode 100644
index 000000000000..f4a54457416d
--- /dev/null
+++ b/includes/composer/PhpUnitSplitter/PhpUnitTestListProcessor.php
@@ -0,0 +1,40 @@
+<?php
+
+declare( strict_types = 1 );
+
+namespace MediaWiki\Composer\PhpUnitSplitter;
+
+use SimpleXMLElement;
+
+/**
+ * @license GPL-2.0-or-later
+ */
+class PhpUnitTestListProcessor {
+
+	private SimpleXMLElement $xml;
+
+	public function __construct( string $testListFile ) {
+		$this->xml = new SimpleXMLElement( file_get_contents( $testListFile ) );
+	}
+
+	/**
+	 * @return TestDescriptor[] A list of TestDescriptor objects representing the
+	 *                          test classes found in the `--list-tests` XML
+	 *                          output
+	 */
+	public function getTestClasses(): array {
+		if ( !property_exists( $this->xml, "testCaseClass" ) ) {
+			return [];
+		}
+		return array_map(
+			fn ( $element ) => self::extractNamespace( (string)$element->attributes()["name"] ),
+			iterator_to_array( $this->xml->testCaseClass, false )
+		);
+	}
+
+	private static function extractNamespace( string $qualifiedClassName ): TestDescriptor {
+		$parts = explode( '\\', $qualifiedClassName );
+		$className = array_pop( $parts );
+		return new TestDescriptor( $className, $parts );
+	}
+}
diff --git a/includes/composer/PhpUnitSplitter/PhpUnitXml.php b/includes/composer/PhpUnitSplitter/PhpUnitXml.php
new file mode 100644
index 000000000000..a1ee6e439814
--- /dev/null
+++ b/includes/composer/PhpUnitSplitter/PhpUnitXml.php
@@ -0,0 +1,92 @@
+<?php
+
+declare( strict_types = 1 );
+
+namespace MediaWiki\Composer\PhpUnitSplitter;
+
+use DOMDocument;
+use SimpleXMLElement;
+
+/**
+ * @license GPL-2.0-or-later
+ */
+class PhpUnitXml {
+
+	public const PHP_UNIT_XML_FILE = "phpunit.xml";
+
+	private SimpleXMLElement $xml;
+
+	public function __construct( string $phpUnitXmlFile ) {
+		$this->xml = new SimpleXMLElement( file_get_contents( $phpUnitXmlFile ) );
+	}
+
+	public function containsSplitGroups(): bool {
+		if ( !property_exists( $this->xml, "testsuites" ) ||
+			!property_exists( $this->xml->testsuites, "testsuite" ) ) {
+			return false;
+		}
+		foreach ( $this->xml->testsuites->testsuite as $child ) {
+			if ( isset( $child->attributes()["name"] ) &&
+				strpos( (string)$child->attributes()["name"], "split_group_" ) === 0 ) {
+				return true;
+			}
+		}
+		return false;
+	}
+
+	public function addSplitGroups( array $splitGroups ) {
+		$groups = count( $splitGroups );
+		for ( $i = 0; $i < $groups; $i++ ) {
+			$suite = $this->xml->testsuites->addChild( "testsuite" );
+			$suite->addAttribute( "name", "split_group_" . $i );
+			$group = $splitGroups[$i];
+			if ( !empty( $group["list"] ) ) {
+				foreach ( $group["list"] as $file ) {
+					$suite->addChild( "file", $file );
+				}
+			}
+		}
+	}
+
+	/**
+	 * @throws SuiteGenerationException
+	 */
+	private function getSplitGroupSuite( int $groupId ): SimpleXMLElement {
+		foreach ( $this->xml->testsuites->testsuite as $child ) {
+			if ( isset( $child->attributes()["name"] ) &&
+				(string)$child->attributes()["name"] === "split_group_" . $groupId ) {
+				return $child;
+			}
+		}
+		throw new SuiteGenerationException( $groupId );
+	}
+
+	/**
+	 * There are some tests suites / classes where the test listing does not work because test
+	 * cases are generated dynamically. For this special cases, we need to add the classes
+	 * manually back into the suites list to ensure that they get included in a test run.
+	 * @see T345481
+	 * @see T358394
+	 * @throws SuiteGenerationException
+	 */
+	public function addSpecialCaseTests( int $groupCount ) {
+		$suite = $this->xml->testsuites->addChild( "testsuite" );
+		$suite->addAttribute( "name", "split_group_" . ( $groupCount - 1 ) );
+		$suite->addChild( "file", "tests/phpunit/suites/ExtensionsParserTestSuite.php" );
+
+		$sandboxTest = "extensions/Scribunto/tests/phpunit/Engines/LuaSandbox/SandboxTest.php";
+		if ( file_exists( $sandboxTest ) ) {
+			$suite = $this->getSplitGroupSuite( 0 );
+			$suite->addChild( "file", $sandboxTest );
+		}
+	}
+
+	public function saveToDisk( string $targetXml ) {
+		$dom = new DOMDocument( '1.0' );
+		$dom->preserveWhiteSpace = false;
+		$dom->formatOutput = true;
+		$dom->loadXML( $this->xml->asXML() );
+		file_put_contents( $targetXml, $dom->saveXML() );
+	}
+
+}
diff --git a/includes/composer/PhpUnitSplitter/PhpUnitXmlManager.php b/includes/composer/PhpUnitSplitter/PhpUnitXmlManager.php
new file mode 100644
index 000000000000..68a89a4058bc
--- /dev/null
+++ b/includes/composer/PhpUnitSplitter/PhpUnitXmlManager.php
@@ -0,0 +1,176 @@
+<?php
+
+declare( strict_types = 1 );
+
+namespace MediaWiki\Composer\PhpUnitSplitter;
+
+/**
+ * @license GPL-2.0-or-later
+ */
+class PhpUnitXmlManager {
+
+	private string $rootDir;
+
+	/**
+	 * The `SkippedTestCase` is generated dynamically by PHPUnit for tests
+	 * that are marked as skipped. We don't need to find a matching filesystem
+	 * file for these.
+	 *
+	 * The `ParserIntegrationTest` is a special case - it's a single test class
+	 * that generates very many tests. To balance out the test suites, we exclude
+	 * the class from the scan, and add it back in PhpUnitXml::addSpecialCaseTests
+	 */
+	private const EXPECTED_MISSING_CLASSES = [
+		"PHPUnit\\Framework\\SkippedTestCase",
+		"MediaWiki\\Extension\\Scribunto\\Tests\\Engines\\LuaCommon\\LuaEngineTestSkip",
+		"\\ParserIntegrationTest",
+	];
+
+	public function __construct( string $rootDir ) {
+		$this->rootDir = $rootDir;
+	}
+
+	private function getPhpUnitXmlTarget(): string {
+		return $this->rootDir . DIRECTORY_SEPARATOR . PhpUnitXml::PHP_UNIT_XML_FILE;
+	}
+
+	private function getPhpUnitXmlDist(): string {
+		return $this->rootDir . DIRECTORY_SEPARATOR . "phpunit.xml.dist";
+	}
+
+	private function getTestsList(): string {
+		return $this->rootDir . DIRECTORY_SEPARATOR . "tests-list.xml";
+	}
+
+	public function isPhpUnitXmlPrepared(): bool {
+		if ( !file_exists( $this->getPhpUnitXmlTarget() ) ) {
+			return false;
+		}
+		$unitFile = $this->loadPhpUnitXml( $this->getPhpUnitXmlTarget() );
+		return $unitFile->containsSplitGroups();
+	}
+
+	private function loadPhpUnitXmlDist(): PhpUnitXml {
+		return $this->loadPhpUnitXml( $this->getPhpUnitXmlDist() );
+	}
+
+	private function loadPhpUnitXml( string $targetFile ): PhpUnitXml {
+		return new PhpUnitXml( $targetFile );
+	}
+
+	private function loadTestClasses(): array {
+		if ( !file_exists( $this->getTestsList() ) ) {
+			throw new TestListMissingException( $this->getTestsList() );
+		}
+		return ( new PhpUnitTestListProcessor( $this->getTestsList() ) )->getTestClasses();
+	}
+
+	private function scanForTestFiles(): array {
+		return ( new PhpUnitTestFileScanner( $this->rootDir ) )->scanForFiles();
+	}
+
+	private static function extractNamespaceFromFile( $filename ): array {
+		$contents = file_get_contents( $filename );
+		$matches = [];
+		if ( preg_match( '/^namespace\s+([^\s;]+)/m', $contents, $matches ) ) {
+			return explode( '\\', $matches[1] );
+		}
+		return [];
+	}
+
+	/**
+	 * @param TestDescriptor $testDescriptor
+	 * @param array $phpFiles
+	 * @return ?string
+	 * @throws MissingNamespaceMatchForTestException
+	 * @throws UnlocatedTestException
+	 */
+	private function resolveFileForTest( TestDescriptor $testDescriptor, array $phpFiles ): ?string {
+		$filename = $testDescriptor->getClassName() . ".php";
+		if ( !array_key_exists( $filename, $phpFiles ) ) {
+			if ( !in_array( $testDescriptor->getFullClassname(), self::EXPECTED_MISSING_CLASSES ) ) {
+				throw new UnlocatedTestException( $testDescriptor );
+			} else {
+				return null;
+			}
+		}
+		if ( count( $phpFiles[$filename] ) === 1 ) {
+			return $phpFiles[$filename][0];
+		}
+		$possibleNamespaces = [];
+		foreach ( $phpFiles[$filename] as $file ) {
+			$namespace = self::extractNamespaceFromFile( $file );
+			if ( $namespace === $testDescriptor->getNamespace() ) {
+				return $file;
+			}
+			$possibleNamespaces[] = $namespace;
+		}
+		throw new MissingNamespaceMatchForTestException( $testDescriptor );
+	}
+
+	private function buildSuites( array $testClasses, int $groups ): array {
+		return ( new TestSuiteBuilder() )->buildSuites( $testClasses, $groups );
+	}
+
+	/**
+	 * @return void
+	 * @throws MissingNamespaceMatchForTestException
+	 * @throws TestListMissingException
+	 * @throws UnlocatedTestException
+	 * @throws SuiteGenerationException
+	 */
+	public function createPhpUnitXml( int $groups ) {
+		$unitFile = $this->loadPhpUnitXmlDist();
+		$testFiles = $this->scanForTestFiles();
+		$testClasses = $this->loadTestClasses();
+		$seenFiles = [];
+		foreach ( $testClasses as $testDescriptor ) {
+			$file = $this->resolveFileForTest( $testDescriptor, $testFiles );
+			if ( is_string( $file ) && !array_key_exists( $file, $seenFiles ) ) {
+				$testDescriptor->setFilename( $file );
+				$seenFiles[$file] = 1;
+			}
+		}
+		$suites = $this->buildSuites( $testClasses, $groups - 1 );
+		$unitFile->addSplitGroups( $suites );
+		$unitFile->addSpecialCaseTests( $groups );
+		$unitFile->saveToDisk( $this->getPhpUnitXmlTarget() );
+	}
+
+	public static function listTestsNotice() {
+		print( PHP_EOL );
+		print( 'Running `phpunit --list-tests-xml` to get a list of expected tests ... ' . PHP_EOL );
+		print( PHP_EOL );
+	}
+
+	/**
+	 * @throws TestListMissingException
+	 * @throws UnlocatedTestException
+	 * @throws MissingNamespaceMatchForTestException
+	 * @throws SuiteGenerationException
+	 */
+	public static function splitTestsList() {
+		/**
+		 * We split into 8 groups here, because experimentally that generates 100% CPU load
+		 * on developer machines and results in groups that are similar in size to the
+		 * Parser tests (which we have to run in a group on their own - see T345481)
+		 */
+		( new PhpUnitXmlManager( getcwd() ) )->createPhpUnitXml( 8 );
+		print( PHP_EOL . 'Created modified `phpunit.xml` with test suite groups' . PHP_EOL );
+	}
+
+	/**
+	 * @throws TestListMissingException
+	 * @throws UnlocatedTestException
+	 * @throws MissingNamespaceMatchForTestException
+	 * @throws SuiteGenerationException
+	 */
+	public static function splitTestsCustom() {
+		if ( $_SERVER["argc"] < 3 ) {
+			print( 'Specify a filename to split' . PHP_EOL );
+			exit( 1 );
+		}
+		$filename = $_SERVER["argv"][2];
+		self::splitTestsList( $filename );
+	}
+}
diff --git a/includes/composer/PhpUnitSplitter/SuiteGenerationException.php b/includes/composer/PhpUnitSplitter/SuiteGenerationException.php
new file mode 100644
index 000000000000..344c39f6c6eb
--- /dev/null
+++ b/includes/composer/PhpUnitSplitter/SuiteGenerationException.php
@@ -0,0 +1,16 @@
+<?php
+
+declare( strict_types = 1 );
+
+namespace MediaWiki\Composer\PhpUnitSplitter;
+
+/**
+ * @license GPL-2.0-or-later
+ */
+class SuiteGenerationException extends \Exception {
+
+	public function __construct( int $groupId ) {
+		parent::__construct( "Unable to find suite split_group_" . $groupId );
+	}
+
+}
diff --git a/includes/composer/PhpUnitSplitter/TestDescriptor.php b/includes/composer/PhpUnitSplitter/TestDescriptor.php
new file mode 100644
index 000000000000..78bde3f910fa
--- /dev/null
+++ b/includes/composer/PhpUnitSplitter/TestDescriptor.php
@@ -0,0 +1,46 @@
+<?php
+
+declare( strict_types = 1 );
+
+namespace MediaWiki\Composer\PhpUnitSplitter;
+
+/**
+ * @license GPL-2.0-or-later
+ */
+class TestDescriptor {
+
+	private string $className;
+	private array $namespace;
+	private ?string $filename;
+
+	public function __construct( string $className, array $namespace, ?string $filename = null ) {
+		$this->className = $className;
+		$this->namespace = $namespace;
+		$this->filename = $filename;
+	}
+
+	public function getNamespace(): array {
+		return $this->namespace;
+	}
+
+	public function getClassName(): string {
+		return $this->className;
+	}
+
+	public function setFilename( string $filename ): void {
+		$this->filename = $filename;
+	}
+
+	public function getFilename(): ?string {
+		return $this->filename;
+	}
+
+	public function getFullClassname(): string {
+		return implode( '\\', $this->namespace ) . '\\' . $this->className;
+	}
+
+	public function getDuration() {
+		return 0;
+	}
+
+}
diff --git a/includes/composer/PhpUnitSplitter/TestListMissingException.php b/includes/composer/PhpUnitSplitter/TestListMissingException.php
new file mode 100644
index 000000000000..5986383b7325
--- /dev/null
+++ b/includes/composer/PhpUnitSplitter/TestListMissingException.php
@@ -0,0 +1,16 @@
+<?php
+
+declare( strict_types = 1 );
+
+namespace MediaWiki\Composer\PhpUnitSplitter;
+
+/**
+ * @license GPL-2.0-or-later
+ */
+class TestListMissingException extends \Exception {
+
+	public function __construct( string $testListFile ) {
+		parent::__construct( "Could not find test list at " . $testListFile );
+	}
+
+}
diff --git a/includes/composer/PhpUnitSplitter/TestSuiteBuilder.php b/includes/composer/PhpUnitSplitter/TestSuiteBuilder.php
new file mode 100644
index 000000000000..95f183317efd
--- /dev/null
+++ b/includes/composer/PhpUnitSplitter/TestSuiteBuilder.php
@@ -0,0 +1,61 @@
+<?php
+
+declare( strict_types = 1 );
+
+namespace MediaWiki\Composer\PhpUnitSplitter;
+
+/**
+ * @license GPL-2.0-or-later
+ */
+class TestSuiteBuilder {
+
+	private static function sortByTimeDescending( TestDescriptor $a, TestDescriptor $b ): int {
+		if ( $a->getDuration() === $b->getDuration() ) {
+			return 0;
+		}
+		return ( $a->getDuration() > $b->getDuration() ? -1 : 1 );
+	}
+
+	private static function smallestGroup( array $suites ): int {
+		$min = 10000;
+		$minIndex = 0;
+		$groups = count( $suites );
+		for ( $i = 0; $i < $groups; $i++ ) {
+			if ( $suites[$i]["time"] < $min ) {
+				$min = $suites[$i]["time"];
+				$minIndex = $i;
+			}
+		}
+		return $minIndex;
+	}
+
+	public function buildSuites( array $testDescriptors, int $groups ): array {
+		$suites = array_fill( 0, $groups, [ "list" => [], "time" => 0 ] );
+		$roundRobin = 0;
+		usort( $testDescriptors, [ self::class, "sortByTimeDescending" ] );
+		foreach ( $testDescriptors as $testDescriptor ) {
+			if ( !$testDescriptor->getFilename() ) {
+				// We didn't resolve a matching file for this test, so we skip it
+				// from the suite here. This only happens for "known" missing test
+				// classes (see PhpUnitXmlManager::EXPECTED_MISSING_CLASSES) - in
+				// all other cases a missing test file will throw an exception during
+				// suite building.
+				continue;
+			}
+			if ( $testDescriptor->getDuration() === 0 ) {
+				// If no explicit timing information is available for a test, we just
+				// drop it round-robin into the next bucket.
+				$nextSuite = $roundRobin;
+				$roundRobin = ( $roundRobin + 1 ) % $groups;
+			} else {
+				// If we have information about the test duration, we try and balance
+				// out the tests suites by having an even amount of time spent on
+				// each suite.
+				$nextSuite = self::smallestGroup( $suites );
+			}
+			$suites[$nextSuite]["list"][] = $testDescriptor->getFilename();
+			$suites[$nextSuite]["time"] += $testDescriptor->getDuration();
+		}
+		return $suites;
+	}
+}
diff --git a/includes/composer/PhpUnitSplitter/UnlocatedTestException.php b/includes/composer/PhpUnitSplitter/UnlocatedTestException.php
new file mode 100644
index 000000000000..ab33f597f15b
--- /dev/null
+++ b/includes/composer/PhpUnitSplitter/UnlocatedTestException.php
@@ -0,0 +1,16 @@
+<?php
+
+declare( strict_types = 1 );
+
+namespace MediaWiki\Composer\PhpUnitSplitter;
+
+/**
+ * @license GPL-2.0-or-later
+ */
+class UnlocatedTestException extends \Exception {
+
+	public function __construct( TestDescriptor $testDescriptor ) {
+		parent::__construct( "Could not find file for class " . $testDescriptor->getFullClassname() );
+	}
+
+}
diff --git a/tests/phpunit/unit/includes/composer/PhpUnitSplitter/PhpUnitTestFileScannerTest.php b/tests/phpunit/unit/includes/composer/PhpUnitSplitter/PhpUnitTestFileScannerTest.php
new file mode 100644
index 000000000000..b694f0a0b4d5
--- /dev/null
+++ b/tests/phpunit/unit/includes/composer/PhpUnitSplitter/PhpUnitTestFileScannerTest.php
@@ -0,0 +1,26 @@
+<?php
+
+declare( strict_types = 1 );
+
+namespace MediaWiki\Tests\Unit\composer\PhpUnitSplitter;
+
+use MediaWiki\Composer\PhpUnitSplitter\PhpUnitTestFileScanner;
+use PHPUnit\Framework\TestCase;
+
+/**
+ * @license GPL-2.0-or-later
+ * @covers \MediaWiki\Composer\PhpUnitSplitter\PhpUnitTestFileScanner
+ */
+class PhpUnitTestFileScannerTest extends TestCase {
+
+	public function testScanForTestFiles() {
+		$scanner = new PhpUnitTestFileScanner( __DIR__ );
+		$files = $scanner->scanForFiles();
+		$expected = [];
+		foreach ( glob( __DIR__ . DIRECTORY_SEPARATOR . "*Test.php" ) as $testFile ) {
+			$expected[ basename( $testFile ) ] = [ $testFile ];
+		}
+		$this->assertEquals( $expected, $files, "Expected PhpUnitSplitter test files to be found" );
+	}
+
+}
diff --git a/tests/phpunit/unit/includes/composer/PhpUnitSplitter/PhpUnitTestListProcessorTest.php b/tests/phpunit/unit/includes/composer/PhpUnitSplitter/PhpUnitTestListProcessorTest.php
new file mode 100644
index 000000000000..04526c1b7e8a
--- /dev/null
+++ b/tests/phpunit/unit/includes/composer/PhpUnitSplitter/PhpUnitTestListProcessorTest.php
@@ -0,0 +1,29 @@
+<?php
+
+declare( strict_types = 1 );
+
+namespace MediaWiki\Tests\Unit\composer\PhpUnitSplitter;
+
+use MediaWiki\Composer\PhpUnitSplitter\PhpUnitTestListProcessor;
+use PHPUnit\Framework\TestCase;
+
+/**
+ * @license GPL-2.0-or-later
+ * @covers \MediaWiki\Composer\PhpUnitSplitter\PhpUnitTestListProcessor
+ */
+class PhpUnitTestListProcessorTest extends TestCase {
+
+	private const FIXTURE_FILE = __DIR__ . "/fixtures/tests-list.xml";
+
+	public function testGetTestClasses() {
+		$testList = new PhpUnitTestListProcessor( self::FIXTURE_FILE );
+		$this->assertCount(
+			5, $testList->getTestClasses(), "Expected classes to be loaded"
+		);
+		$test5 = $testList->getTestClasses()[4];
+		$this->assertEquals(
+			[ [ 'MediaWiki', 'Tests', 'Unit', 'composer', 'PhpUnitSplitter' ], 'TestSuiteBuilderTest' ],
+			[ $test5->getNamespace(), $test5->getClassName() ]
+		);
+	}
+}
diff --git a/tests/phpunit/unit/includes/composer/PhpUnitSplitter/PhpUnitXmlManagerTest.php b/tests/phpunit/unit/includes/composer/PhpUnitSplitter/PhpUnitXmlManagerTest.php
new file mode 100644
index 000000000000..af4c0d1a2d9d
--- /dev/null
+++ b/tests/phpunit/unit/includes/composer/PhpUnitSplitter/PhpUnitXmlManagerTest.php
@@ -0,0 +1,96 @@
+<?php
+
+declare( strict_types = 1 );
+
+namespace MediaWiki\Tests\Unit\composer\PhpUnitSplitter;
+
+use MediaWiki\Composer\PhpUnitSplitter\PhpUnitXmlManager;
+use MediaWiki\Composer\PhpUnitSplitter\TestListMissingException;
+use PHPUnit\Framework\TestCase;
+
+/**
+ * @license GPL-2.0-or-later
+ * @covers \MediaWiki\Composer\PhpUnitSplitter\PhpUnitXmlManager
+ */
+class PhpUnitXmlManagerTest extends TestCase {
+
+	private string $testDir;
+	private PhpUnitXmlManager $manager;
+
+	public function setUp(): void {
+		parent::setUp();
+		$this->testDir = implode( DIRECTORY_SEPARATOR, [ sys_get_temp_dir(), uniqid( 'PhpUnitTest' ) ] );
+		mkdir( $this->testDir );
+		$this->manager = new PhpUnitXmlManager( $this->testDir );
+		$this->setupTestFolder();
+	}
+
+	private static function getSourcePhpUnitDistXml(): string {
+		return __DIR__ . DIRECTORY_SEPARATOR . implode(
+		DIRECTORY_SEPARATOR, [ '..', '..', '..', '..', '..', '..', 'phpunit.xml.dist' ]
+		);
+	}
+
+	private function setupTestFolder() {
+		copy(
+			self::getSourcePhpUnitDistXml(),
+			$this->testDir . DIRECTORY_SEPARATOR . 'phpunit.xml.dist'
+		);
+		mkdir( $this->testDir . DIRECTORY_SEPARATOR . "tests" );
+		foreach ( glob( __DIR__ . DIRECTORY_SEPARATOR . "*Test.php" ) as $file ) {
+			copy(
+				$file,
+				implode( DIRECTORY_SEPARATOR, [ $this->testDir, "tests", basename( $file ) ] )
+			);
+		}
+	}
+
+	private function tearDownTestFolder() {
+		foreach ( [ 'phpunit.xml', 'phpunit.xml.dist', 'tests-list.xml' ] as $file ) {
+			$path = $this->testDir . DIRECTORY_SEPARATOR . $file;
+			if ( file_exists( $path ) ) {
+				unlink( $path );
+			}
+		}
+		$testsFolder = $this->testDir . DIRECTORY_SEPARATOR . "tests";
+		foreach ( glob( $testsFolder . DIRECTORY_SEPARATOR . "*.php" ) as $file ) {
+			unlink( $file );
+		}
+		rmdir( $testsFolder );
+		rmdir( $this->testDir );
+	}
+
+	private function copyTestListIntoPlace() {
+		copy(
+			__DIR__ . DIRECTORY_SEPARATOR . 'fixtures' . DIRECTORY_SEPARATOR . 'tests-list.xml',
+			$this->testDir . DIRECTORY_SEPARATOR . 'tests-list.xml'
+		);
+	}
+
+	public function tearDown(): void {
+		parent::tearDown();
+		$this->tearDownTestFolder();
+	}
+
+	public function testIsPrepared() {
+		$this->copyTestListIntoPlace();
+		$this->assertFalse( $this->manager->isPhpUnitXmlPrepared(), "Expected no PHPUnit Xml to be present" );
+		$this->manager->createPhpUnitXml( 4 );
+		$this->assertTrue( $this->manager->isPhpUnitXmlPrepared(), "Expected PHPUnit Xml to have been prepared" );
+	}
+
+	public function testFailsIfNoListIsPresent() {
+		$this->assertFalse( $this->manager->isPhpUnitXmlPrepared(), "Expected no PHPUnit Xml to be present" );
+		$this->expectException( TestListMissingException::class );
+		$this->manager->createPhpUnitXml( 4 );
+	}
+
+	public function testPhpUnitXmlDistNotPrepared() {
+		$this->assertFalse( $this->manager->isPhpUnitXmlPrepared(), "Expected no PHPUnit Xml to be present" );
+		copy( self::getSourcePhpUnitDistXml(), implode( DIRECTORY_SEPARATOR, [ $this->testDir, "phpunit.xml" ] ) );
+		$this->copyTestListIntoPlace();
+		$this->manager->createPhpUnitXml( 4 );
+		copy( self::getSourcePhpUnitDistXml(), implode( DIRECTORY_SEPARATOR, [ $this->testDir, "phpunit.xml" ] ) );
+		$this->assertFalse( $this->manager->isPhpUnitXmlPrepared(), "Expected phpunit.dist.xml to be treated as unprepared" );
+	}
+}
diff --git a/tests/phpunit/unit/includes/composer/PhpUnitSplitter/PhpUnitXmlTest.php b/tests/phpunit/unit/includes/composer/PhpUnitSplitter/PhpUnitXmlTest.php
new file mode 100644
index 000000000000..6b30596e90bf
--- /dev/null
+++ b/tests/phpunit/unit/includes/composer/PhpUnitSplitter/PhpUnitXmlTest.php
@@ -0,0 +1,52 @@
+<?php
+
+declare( strict_types = 1 );
+
+namespace MediaWiki\Tests\Unit\composer\PhpUnitSplitter;
+
+use MediaWiki\Composer\PhpUnitSplitter\PhpUnitXml;
+use PHPUnit\Framework\TestCase;
+
+/**
+ * @license GPL-2.0-or-later
+ * @covers \MediaWiki\Composer\PhpUnitSplitter\PhpUnitXml
+ */
+class PhpUnitXmlTest extends TestCase {
+
+	private const BASIC_XML = '<?xml version="1.0" encoding="UTF-8"?>
+<phpunit bootstrap="tests/phpunit/bootstrap.php">
+<testsuites>
+	<testsuite name="core:unit">
+		<directory>tests/phpunit/unit</directory>
+	</testsuite>
+</testsuites>
+</phpunit>';
+
+	public function createFixtureFile( string $data ): string {
+		$filename = tempnam( sys_get_temp_dir(), "phpunit-test" );
+		file_put_contents( $filename, $data );
+		return $filename;
+	}
+
+	public function testFixtureContainsNoSplitGroups() {
+		$phpUnitXmlFile = $this->createFixtureFile( self::BASIC_XML );
+		$phpUnitXml = new PhpUnitXml( $phpUnitXmlFile );
+		$this->assertFalse( $phpUnitXml->containsSplitGroups(), "No split groups expected in fixture" );
+		unlink( $phpUnitXmlFile );
+	}
+
+	public function testAddSplitGroups() {
+		$phpUnitXmlFile = $this->createFixtureFile( self::BASIC_XML );
+		$phpUnitXml = new PhpUnitXml( $phpUnitXmlFile );
+		$phpUnitXml->addSplitGroups( [
+			[ "file1.php", "file2.php" ],
+			[ "file3.php", "file4.php" ],
+			[ "file7.php", "file6.php" ],
+			[ "file9.php", "file8.php" ],
+			[ "file11.php", "file10.php" ],
+			[ "file13.php", "file12.php" ],
+		] );
+		$this->assertTrue( $phpUnitXml->containsSplitGroups(), "Expected groups to be added" );
+		unlink( $phpUnitXmlFile );
+	}
+}
diff --git a/tests/phpunit/unit/includes/composer/PhpUnitSplitter/TestSuiteBuilderTest.php b/tests/phpunit/unit/includes/composer/PhpUnitSplitter/TestSuiteBuilderTest.php
new file mode 100644
index 000000000000..ce8a99504a6e
--- /dev/null
+++ b/tests/phpunit/unit/includes/composer/PhpUnitSplitter/TestSuiteBuilderTest.php
@@ -0,0 +1,50 @@
+<?php
+
+declare( strict_types = 1 );
+
+namespace MediaWiki\Tests\Unit\composer\PhpUnitSplitter;
+
+use MediaWiki\Composer\PhpUnitSplitter\TestDescriptor;
+use MediaWiki\Composer\PhpUnitSplitter\TestSuiteBuilder;
+use PHPUnit\Framework\TestCase;
+
+/**
+ * @license GPL-2.0-or-later
+ * @covers \MediaWiki\Composer\PhpUnitSplitter\TestSuiteBuilder
+ */
+class TestSuiteBuilderTest extends TestCase {
+
+	public function testBuildSuites() {
+		$testList = [
+			new TestDescriptor( "ATest", [ "MediaWiki" ], "MediaWiki/ATest.php" ),
+			new TestDescriptor( "BTest", [ "MediaWiki" ], "MediaWiki/BTest.php" ),
+			new TestDescriptor( "CTest", [ "MediaWiki" ], "MediaWiki/CTest.php" ),
+			new TestDescriptor( "DTest", [ "MediaWiki" ], "MediaWiki/DTest.php" ),
+			new TestDescriptor( "ETest", [ "MediaWiki" ], "MediaWiki/ETest.php" ),
+		];
+		$suites = ( new TestSuiteBuilder() )->buildSuites( $testList, 3 );
+		$expected = [
+			[
+				"list" => [
+					"MediaWiki/ATest.php",
+					"MediaWiki/DTest.php",
+				],
+				"time" => 0
+			],
+			[
+				"list" => [
+					"MediaWiki/BTest.php",
+					"MediaWiki/ETest.php",
+				],
+				"time" => 0
+			],
+			[
+				"list" => [
+					"MediaWiki/CTest.php",
+				],
+				"time" => 0
+			]
+		];
+		$this->assertEquals( $expected, $suites, "Expected suites to be built correctly" );
+	}
+}
diff --git a/tests/phpunit/unit/includes/composer/PhpUnitSplitter/fixtures/tests-list.xml b/tests/phpunit/unit/includes/composer/PhpUnitSplitter/fixtures/tests-list.xml
new file mode 100644
index 000000000000..942fefe72e48
--- /dev/null
+++ b/tests/phpunit/unit/includes/composer/PhpUnitSplitter/fixtures/tests-list.xml
@@ -0,0 +1,22 @@
+<?xml version="1.0" encoding="UTF-8"?>
+<tests>
+	<testCaseClass name="MediaWiki\Tests\Unit\composer\PhpUnitSplitter\PhpUnitTestFileScannerTest">
+		<testCaseMethod name="testSchemaChangesPassValidation" groups="default" dataSet="&quot;patch-change_tag-rename-indexes.json&quot;"/>
+	</testCaseClass>
+	<testCaseClass name="MediaWiki\Tests\Unit\composer\PhpUnitSplitter\PhpUnitTestListProcessorTest">
+		<testCaseMethod name="testPrefixes" groups="API,Database"/>
+		<testCaseMethod name="testValidCovers" groups="API,Database"/>
+	</testCaseClass>
+	<testCaseClass name="MediaWiki\Tests\Unit\composer\PhpUnitSplitter\PhpUnitXmlManagerTest">
+		<testCaseMethod name="testConfigSchemaIsLoadable" groups="default"/>
+		<testCaseMethod name="testConfigSchemaDefaultsValidate" groups="default"/>
+		<testCaseMethod name="testCurrentSettingsValidate" groups="default"/>
+		<testCaseMethod name="testCurrentSettingsNotDeprecated" groups="default"/>
+	</testCaseClass>
+	<testCaseClass name="MediaWiki\Tests\Unit\composer\PhpUnitSplitter\PhpUnitXmlTest">
+		<testCaseMethod name="testOnBeforePageDisplay" groups="Marius Hoch &lt; hoo@online.de &gt;,WikimediaBadges,__phpunit_covers_wikimediabadges\beforepagedisplayhookhandler"/>
+	</testCaseClass>
+	<testCaseClass name="MediaWiki\Tests\Unit\composer\PhpUnitSplitter\TestSuiteBuilderTest">
+		<testCaseMethod name="testOnBeforePageDisplay" groups="Marius Hoch &lt; hoo@online.de &gt;,WikimediaBadges,__phpunit_covers_wikimediabadges\beforepagedisplayhookhandler"/>
+	</testCaseClass>
+</tests>