aboutsummaryrefslogtreecommitdiffstats
path: root/python/wpt
Commit message (Collapse)AuthorAgeFilesLines
* Remove legacy layout (layout 2013) (#35943)Oriol Brufau2025-03-133-34/+4
| | | | | | We were already not compiling it and not running tests on it by default. So it's simpler to just completely remove it. Signed-off-by: Oriol Brufau <obrufau@igalia.com>
* change terminal wrapper library from blessing to blessed to support running ↵TIN TUN AUNG2025-02-061-2/+2
| | | | | mach test-wpt on windows. (#35327) Signed-off-by: rayguo17 <rayguo17@gmail.com>
* Correctly report number of expected test results from ./mach test-wpt (#35068)Simon Wülker2025-01-191-1/+4
| | | | | | The previous code was simply reporting the number of categories, instead of the sum of the number of expected tests in each category. Signed-off-by: Simon Wülker <simon.wuelker@arcor.de>
* Fix check for unused subdir in wpt import script (#35036)Samson2025-01-171-3/+1
| | | | | | | Before we appended all subdirs of unused_dir to unused_dirs, which caused errors on removing because root dir was already removed. Fix #35026, tested locally. Signed-off-by: Samson <16504129+sagudev@users.noreply.github.com>
* Enable wdspec and crashtest tests in servodriver. (#34955)Josh Matthews2025-01-121-1/+1
| | | Signed-off-by: Josh Matthews <josh@joshmatthews.net>
* Iterate PR commits in reverse (oldest->newest) when preparing WPT export. ↵Josh Matthews2024-12-111-1/+3
| | | | | (#34567) Signed-off-by: Josh Matthews <josh@joshmatthews.net>
* mach: fix logic to override paths for legacy layout (#34467)Mukilan Thiyagarajan2024-12-041-9/+9
| | | | | | | | | | | | | | | | | | | | | | | | | | | | wptrunner internally derives the path to the MANIFEST.json file from the `metadata_path` passed via `test_paths`. The current logic overrides only the `metadata_path`, leaving the manifest_path pointing at the wrong (non-legacy) layout's MANIFEST.json. In #34436 we observed that the recent WPT imports create a transient `.cache` diretory that wptrunner logic uses for optimization. This has not been observed until #34436 because a [recent bump in MANIFEST.json's schema version][1] triggered the creation of the cache. Because of the above issue with incorrect path and the fact that we *first* trigger `mach update-wpt` for legacy layout during WPT import, the MANIFEST.json of non-legacy layout gets incorrectly migrated during the invocation of `update-wpt` for legacy layout but the cache is still created under legacy-layout's path as it is not based on `manifest_path`. The subsequent invocation of `mach update-wpt` for non-legacy finds the MANIFEST.json already migrated so the `.cache` directory is not constucted. This change simply replaces the whole object using the wptrunner's `TestRoot` class constructor so that all derived paths are calculated correctly. We also add the `.cache` folders to gitignore as it seems like they are expected to be created during such version migrations. [1]: https://github.com/web-platform-tests/wpt/pull/49406 Signed-off-by: Mukilan Thiyagarajan <mukilan@igalia.com>
* CI: use new intermittent-tracker deployment (#34068)Delan Azabani2024-11-041-4/+4
| | | | | | | | | | | | | | | | | * CI: use new intermittent-tracker deployment Signed-off-by: Delan Azabani <dazabani@igalia.com> * Plumb new secrets into linux-wpt.yml environment Signed-off-by: Delan Azabani <dazabani@igalia.com> * Print effective dashboard URL in WPT runner Signed-off-by: Delan Azabani <dazabani@igalia.com> --------- Signed-off-by: Delan Azabani <dazabani@igalia.com>
* ci: Fix the WPT export job after the repository change (#33228)Martin Robinson2024-08-293-6/+7
| | | | | | | | The GitHub search API is a bit sensitive. There isn't a great way to search for the repository organization and the branch name when looking for open PRs. Instead use the bot username as the author name, which should likely have been having before. This fixes the WPT export job. Signed-off-by: Martin Robinson <mrobinson@igalia.com>
* ci: Use Servo's `wpt` repository for WPT export (#33224)Martin Robinson2024-08-282-31/+31
| | | | | | | | | | | | This switches the WPT export script to use Servo's fork the `wpt` upstream repository. The reasoning is that Servo developers's can fix issues with branches there. This isn't possible because no one has permissions to the servo-wpt-sync fork of the repository. I have given `servo-wpt-sync` write persmissions to `servo/wpt`. This also updates the tests a bit to reflect the new default repository. Signed-off-by: Martin Robinson <mrobinson@igalia.com>
* Add fallback value for data['message'] (#33185)Samson2024-08-251-1/+1
| | | Signed-off-by: Samson <16504129+sagudev@users.noreply.github.com>
* Print reason when tests are skipped. (#33077)Josh Matthews2024-08-231-11/+20
| | | | | | | | | | | | | | * Print reason when tests are skipped. Signed-off-by: Josh Matthews <josh@joshmatthews.net> * Print reason tests are skipped in Servo's WPT formatter Signed-off-by: Martin Robinson <mrobinson@igalia.com> --------- Signed-off-by: Josh Matthews <josh@joshmatthews.net> Signed-off-by: Martin Robinson <mrobinson@igalia.com>
* ci: Use a servo.org email address for the servo-wpt-sync GitHub bot (#32515)Martin Robinson2024-06-182-2/+2
| | | | | This is the new address of the WPT sync bot. Signed-off-by: Martin Robinson <mrobinson@igalia.com>
* wpt: Limit the console output sent to the intermittent tracker (#31895)Martin Robinson2024-03-271-1/+5
| | | | | | | This is a speculative fix for #31845. Instead of sending all of the output to the dashboard, send just the first 1024 characters. This value can be adjusted in the future if it is too large or too small. Fixes #31845.
* tidy: A few small improvements and fixes (#30941)Martin Robinson2024-01-021-2/+3
| | | | | | | | | | 1. Make the tidy output easier to follow 2. Integrate the WPT manifest cleanliness step into tidy itself and don't run it if nothing has changed in the WPT directory. 3. Fix an issue where Python test requirements were not installed, which could cause issues with some modules not being found. Fixes #30002.
* Improve formatting of Python files (#30919)Taym Haddadi2023-12-241-1/+1
| | | Signed-off-by: Bentaimia Haddadi <haddadi.taym@gmail.com>
* Fix the WPT exporter (#30870)Delan Azabani2023-12-154-11/+17
| | | | | | | | | * Fix the WPT exporter * apply fixes by @mrobinson * fix mach test-scripts on NixOS * rename main_branch_name to default_branch
* Rename the `master` branch to `main`Martin Robinson2023-12-0410-39/+39
|
* Sync WPT with upstream (24-10-2023) (#30607)Servo WPT Sync2023-10-253-8/+8
| | | | | | | | | | | | | | | | | | | | | | | | | | * wpt sync: adapt code for new TestRoot class Fixes #30558 Signed-off-by: Mukilan Thiyagarajan <mukilan@igalia.com> * temp change to allow wpt sync build to pass Signed-off-by: Mukilan Thiyagarajan <mukilan@igalia.com> * Update web-platform-tests to revision b'0d5028f200c8651b17bb224657d6e1065adcff37' * Revert "temp change to allow wpt sync build to pass" This reverts commit bad72c7f87c0df16085763ee15ebe63a53462ab3. * revert metadata update for intermittent timeouts Signed-off-by: Mukilan Thiyagarajan <mukilan@igalia.com> --------- Signed-off-by: Mukilan Thiyagarajan <mukilan@igalia.com> Co-authored-by: Mukilan Thiyagarajan <mukilan@igalia.com>
* Fix filtered test counts in WPT summaries (#30591)Delan Azabani2023-10-201-1/+1
|
* Fix WPT import by overriding product name to servo (#30457)Mukilan Thiyagarajan2023-10-042-1/+2
| | | | | | | | | Upstream wpt script has bug where 'product' defaults to 'firefox' and this causes import of uninstalled python modules specific to firefox runners. Fixes #30452 Signed-off-by: Mukilan Thiyagarajan <mukilan@igalia.com>
* Let WPT commands take the --legacy-layout argument (#30239)Martin Robinson2023-08-303-23/+27
| | | This also prints a better message when starting tests.
* Fix `mach test-wpt` to make crash tests work (#29832)Mukilan Thiyagarajan2023-08-182-5/+3
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | * Fix `mach test-wpt` to make crash tests work There are two issues related to crash tests: 1. test-wpt is unable to find existing crash tests even when called with --test-types=crashtests. The fix here is to add crashtests to the default test suite types to python/wpt/run.py 2. When running in headless mode, crashes in style threads don't cause servo to crash because the logic in constellation.rs currently calls handle_panic only when the top-level browsing context id is some value. Since style pool threads are shared, they always generate Panic messages with None as top-level browsing context id. Co-authored-by: Martin Robinson <mrobinson@igalia.com> Signed-off-by: Mukilan Thiyagarajan <mukilan@igalia.com> * Send bactrace to stderr and capture it in test runner Servo's panic hook writes backtraces to stdout. This patch changes it so they are written to stderr. The crash test executor for servo in WPT grouping formatter was also not capturing the output correctly for crashtests as the log events were being aggregated based on thread name which doesn't seem to match correctly in case of crashtests. This patch also fixes the log grouping logic to be based on test name. Co-authored-by: Martin Robinson <mrobinson@igalia.com> Signed-off-by: Mukilan Thiyagarajan <mukilan@igalia.com> * crashtests: update expectations for layout 2020 Co-authored-by: Martin Robinson <mrobinson@igalia.com> Signed-off-by: Mukilan Thiyagarajan <mukilan@igalia.com> * crashtests: update expectations for layout 2013 Co-authored-by: Martin Robinson <mrobinson@igalia.com> Signed-off-by: Mukilan Thiyagarajan <mukilan@igalia.com> * remove outdated & intemittent test expectations Signed-off-by: Mukilan Thiyagarajan <mukilan@igalia.com> --------- Signed-off-by: Mukilan Thiyagarajan <mukilan@igalia.com> Co-authored-by: Martin Robinson <mrobinson@igalia.com>
* Run same tests for layout-2020 and layout-2013 (#30092)Oriol Brufau2023-08-151-2/+2
| | | | | | | | | | Most tests were only being run for layout-2013, not for layout-2020. This wasn't great since layout-2020 is now the default. So this patch unifies the lists of included tests for both layouts. For layout-2013 this implies adding css/css-content/, css/css-logical/ and css/css-masking/clip/. For layout-2020 this implies adding several additional css tests, and also tests like dom/, js/, html/, etc.
* Make the `--release`/`--dev` more consistent and less surprising (#30091)Martin Robinson2023-08-142-26/+2
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | There were some issues with the way that the `--release` and `--dev` arguments were handled in mach commands. - Not all commands accepted them in the same way. For instance `./mach test-wpt` didn't really accept them at all. - If you did not pass either of them, mach would try to guess which build you meant. This guess was often quite surprising as it wasn't printed and it depended on the state of the your target directory, which is difficult to remember. - The `dev` profile is colloquially called a "debug" profile and some commands accepted `-d` or `--debug...` like arguments, but `--debug` with `./mach run` meant run in a debugger. It was easy to mix this up. This change: - Centralizes where build type argument processing happens. Now it the same shared decorator in CommandBase. - Uses a `BuildType` enum instead of passing around two different booleans. This reduces the error checking for situations where both are true. - Be much less clever about guessing what build to use. Now if you don't specify a build type, `--dev` is chosen. I think this behavior matches cargo. - Makes it so that `./mach test-wpt` accepts the exact same arguments and has the same behavior as other commands. In addition, the suite correct for `test-wpt` is removed. There are only two suites now and it's quite unlikely that people will confuse WPT tests for rust unit tests.
* Switch default layout option to `--layout-2020` in some mach commands (#30048)Yutaro Ohno2023-08-023-16/+15
| | | | | | | | | | Currently, `./mach test-wpt` family of commands (`test-wpt`, `test-wpt-android`, and `test-wpt-failure`) and `./mach update-wpt` default to using the legacy-layout option `--layout-2013` unless `--layout-2020` is specified. Given that we are now using layout-2020 by default, this change updates these commands to default to using the `--layout-2020` option instead of `--layout-2013`.
* Fix WPT sync and simplify the update scripts (#29999)Martin Robinson2023-07-173-269/+113
| | | | | | | | | Much of the code used to import WPT tests from upstream has been moved to the WPT repository itself, so this can be reused. In addition, simplify the workflows by merging the entire process into mach and also directly into the GitHub workflow. This should fix WPT imports after combining compilation of layout and legacy layout. Signed-off-by: Martin Robinson <mrobinson@igalia.com>
* Enable the GitHub merge queue (#29989)Martin Robinson2023-07-131-3/+11
| | | Signed-off-by: Martin Robinson <mrobinson@igalia.com>
* Make the choice of layout runtime settingMartin Robinson2023-07-061-1/+5
| | | | Co-authored-by: Samson <16504129+sagudev@users.noreply.github.com>
* Auto merge of #29923 - mrobinson:test-scripts, r=jdmbors-servo2023-06-305-17/+23
|\ | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Combine all script tests into `test-scripts` Remove: - tidy self test support from `./mach test` - `./mach test-idl` Adds a `./mach test-scripts` command that is responsible for running all Python script tests. Run this during the CI to catch regressions in changes to scripts. The WebIDL tests are still *very* slow and there are from Gecko, so only run them when "-a" is passed meaning all tests. <!-- Please describe your changes on the following line: --> --- <!-- Thank you for contributing to Servo! Please replace each `[ ]` by `[X]` when the step is complete, and replace `___` with appropriate data: --> - [x] `./mach build -d` does not report any errors - [x] `./mach test-tidy` does not report any errors - [x] There are tests for these changes <!-- Also, please make sure that "Allow edits from maintainers" checkbox is checked, so that we can help you if you get stuck somewhere along the way.--> <!-- Pull requests that do not address these steps are welcome, but they will require additional verification as part of the review process. -->
| * Combine all script tests into `test-scripts`Martin Robinson2023-06-295-17/+23
| | | | | | | | | | | | | | | | | | | | | | Remove: - tidy self test support from `./mach test` - `./mach test-idl` Adds a `./mach test-scripts` command that is responsible for running all Python script tests. Run this during the CI to catch regressions in changes to scripts. The WebIDL tests are still *very* slow and there are from Gecko, so only run them when "-a" is passed meaning all tests.
* | Update scripts to reflect new WPT directory nameMartin Robinson2023-06-2613-19/+19
|/
* Rename metadata directoriesMartin Robinson2023-06-223-14/+15
| | | | | | | | This renames: - `tests/wpt/metadata` → `tests/wpt/meta-legacy-layout` - `tests/wpt/metadata-layout-2020` → `tests/wpt/meta` - `tests/wpt/mozilla/meta` → `tests/wpt/mozilla/meta-legacy-layout` - `tests/wpt/mozilla/meta-layout-2020` → `tests/wpt/mozilla/meta`
* Set `id_hash` as default wpt chunkersagudev2023-06-091-0/+3
|
* Make WPT results output more usefulMartin Robinson2023-04-261-1/+2
| | | | | | | | Before when a subtest failed, the text of the failed assertion was not printed. This changes makes sure that it is printed in both the console and the aggregated test output. Also fix a couple typing errors.
* Allow wpt commands to accept --with-layout-2013sagudev2023-04-252-0/+4
|
* Fixed bug where test-wpt was unable to run on windows due to servowpt.py not ↵switchpiggy2023-04-231-3/+4
| | | | correctly setting default binary path
* Reorganize Servo's WPT Python scriptsMartin Robinson2023-04-2039-0/+6261
This change moves all of Servo's WPT Python support scripts into one directory as they were previously scattered throughout the directory structure. This should allow more code reuse and make it easier to understand how everything fits together. The changes: - `tests/wpt/update` → `python/wpt/importer` - `etc/ci/upstream-wpt-changes/wptupstreamer` → `python/wpt/exporter` - `etc/ci/upstream-wpt-changes/test.py` → `python/wpt/test.py` - `etc/ci/upstream-wpt-changes/tests` → `python/wpt/tests` - `tests/wpt/servowpt.py` → - `python/wpt/update.py` - `python/wpt/run.py` - `tests/wpt/manifestupdate.py` → `python/wpt/manifestupdate.py` This change also removes - The ability to run the `update-wpt` and `test-wpt` commands without using `mach`. These didn't work very well, because it was difficult to get all of the wptrunner and mach dependencies installed outside of the Python virtualenv. It's simpler if they are always run through `mach`. - The old WPT change upstreaming script that was no longer used.
generated by cgit v1.2.3 (git 2.25.1) at 2025-04-27 11:08:19 +0000