aboutsummaryrefslogtreecommitdiffstats
path: root/python/servo
Commit message (Collapse)AuthorAgeFilesLines
...
| * | Reorganize Servo's WPT Python scriptsMartin Robinson2023-04-201-57/+17
| |/ | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | This change moves all of Servo's WPT Python support scripts into one directory as they were previously scattered throughout the directory structure. This should allow more code reuse and make it easier to understand how everything fits together. The changes: - `tests/wpt/update` → `python/wpt/importer` - `etc/ci/upstream-wpt-changes/wptupstreamer` → `python/wpt/exporter` - `etc/ci/upstream-wpt-changes/test.py` → `python/wpt/test.py` - `etc/ci/upstream-wpt-changes/tests` → `python/wpt/tests` - `tests/wpt/servowpt.py` → - `python/wpt/update.py` - `python/wpt/run.py` - `tests/wpt/manifestupdate.py` → `python/wpt/manifestupdate.py` This change also removes - The ability to run the `update-wpt` and `test-wpt` commands without using `mach`. These didn't work very well, because it was difficult to get all of the wptrunner and mach dependencies installed outside of the Python virtualenv. It's simpler if they are always run through `mach`. - The old WPT change upstreaming script that was no longer used.
* / Build on Mac with the dummy media stackMartin Robinson2023-04-231-2/+6
|/ | | | | | This is a temporary fix for broken GStreamer homebrew packages. Fixes 29653.
* Make errors during build status notification warningsMartin Robinson2023-04-171-2/+5
| | | | | | | Instead of raising an exception, simply print a warning on stderr when notification fails. Fixes #29635.
* Auto merge of #29629 - servo:fix-nightly-job-upload-error, r=mukilanbors-servo2023-04-131-5/+9
|\ | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Fix nightly upload to GH release logic. The boto3 S3 client automatically [closes the given fileobj](https://github.com/boto/s3transfer/blob/develop/s3transfer/upload.py#L106) after the transfer is complete. This prevents us from reusing the package_hash_fileobj between s3 and github upload methods. This is causing the [upload to github to fail](https://github.com/servo/servo/actions/runs/4685791796/jobs/8303739124) with: ``` ValueError: I/O operation on closed file. File "/home/runner/work/servo/servo/python/servo/package_commands.py", line 792, in upload_nightly upload_to_github_release(platform, package, package_hash_fileobj) File "/home/runner/work/servo/servo/python/servo/package_commands.py", line 635, in upload_to_github_release package_hash_fileobj.getbuffer().nbytes, ``` This PR changes fixes the issue by creating fresh instances of io.BytesIO within the two upload_to_* methods. I've triggered a [manual nightly job](https://github.com/servo/servo/actions/runs/4686470246) based on this branch. This PR can be kept open until the build completes, in case other issues are surfaced. --- <!-- Thank you for contributing to Servo! Please replace each `[ ]` by `[X]` when the step is complete, and replace `___` with appropriate data: --> - [x] `./mach build -d` does not report any errors - [x] `./mach test-tidy` does not report any errors - [ ] These changes fix #___ (GitHub issue number if applicable) <!-- Either: --> - [ ] There are tests for these changes OR - [x] These changes do not require tests because they fix an error in nightly CI Job
| * Fix nightly upload to GH release logic.Mukilan Thiyagarajan2023-04-131-5/+9
| | | | | | | | | | | | | | | | | | | | | | | | | | The boto3 S3 client automatically closes the given fileobj after the transfer is complete. This prevents us from reusing the package_hash_fileobj between s3 and github upload methods. This PR changes fixes the issue by creating fresh instances of io.BytesIO within the upload_to_* methods. Signed-off-by: Mukilan Thiyagarajan <me@mukilan.in>
* | Auto merge of #29610 - mrobinson:notify, r=mukilanbors-servo2023-04-133-161/+53
|\ \ | |/ |/| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Use notify-py to send notifications - Use notify-py to send notifications, but use a custom notifier on Linux since transient (ie non-sticky) notifications are not supported. - Add an icon to the notification. - Don't send a notification after doing `./mach check` because that can trigger notifications when `rust-analyzer` is working. --- <!-- Thank you for contributing to Servo! Please replace each `[ ]` by `[X]` when the step is complete, and replace `___` with appropriate data: --> - [x] `./mach build -d` does not report any errors - [x] `./mach test-tidy` does not report any errors - [x] These changes do not require tests because they just change build notifications. <!-- Also, please make sure that "Allow edits from maintainers" checkbox is checked, so that we can help you if you get stuck somewhere along the way.--> <!-- Pull requests that do not address these steps are welcome, but they will require additional verification as part of the review process. -->
| * Use notify-py to send notificationsMartin Robinson2023-04-133-161/+53
| | | | | | | | | | | | | | | | - Use notify-py to send notifications, but use a custom notifier on Linux since transient (ie non-sticky) notifications are not supported. - Add an icon to the notification. - Don't send a notification after doing `./mach check` because that can trigger notifications when `rust-analyzer` is working.
* | Upload nightly builds to Github ReleasesMukilan Thiyagarajan2023-04-121-14/+40
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | This change extends the `mach upload-nightly` command to publish the nightly builds for all platforms as GH Release assets. The GH releases are made on a separate repository so that we can persist older nightly builds without having to accumulate git tags for them. Some design tradeoffs in this approach are: 1. To allow the 'latest' link from servo.org to remain stable, the release assets are named 'servo-latest.{ext}' instead of containing the release tag/date. 2. The release is created as draft and published atomically when all platforms have been built successfully. This allows us to link to the 'latest' alias from servo.org while gauranteeing that it contains builds for all platforms. The other option here would be to have code in servo.org UI that uses GH API to find the most recent release with a successful build for a given platform. 3. The tags in the nightly repo are all based on the same commit that has no relation to servo code base. Signed-off-by: Mukilan Thiyagarajan <me@mukilan.in>
* | Remove more Taskcluster and Treeherder integrationMartin Robinson2023-04-101-33/+5
| | | | | | | | | | Servo no longer uses Taskcluster and Treeherder, so this change removes script references to those services and support files.
* | Replace usage of six.moves.urllib with urllibMartin Robinson2023-04-107-26/+27
|/ | | | | | Also organize some of the imports. Now that Servo only uses Python 3, this module is unnecessary. This is part of the gradual migration to using only Python 3.
* Have all mach command accept "--with-layout-2020" and "--layout-2020"Martin Robinson2023-04-072-4/+4
| | | | | | Some command accepted one and some the other. This changes makes them all accept both so it's no longer necessary to remember which uses which.
* Auto merge of #29431 - sagudev:mtry, r=jdmbors-servo2023-03-211-0/+38
|\ | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Add try command to mach & try build partitioned Adds `./mach try` command that enables anybody to easily test their changes without opening PR and requesting try from bors-servo, by force pushing HEAD to appropriate branch. Command accepts branches names to select only partial runs of CI (same like bors try command). So if you only want to test mac build (that would be `@bors-servo try=mac`) you run `./mach try mac`. If no job is specified, try branch is used. As partitioned CI jobs were not working after migration to GitHub Actions I remade them by using if guards. Also WPT jobs were failing due to empty `INTERMITTENT_TRACKER_DASHBOARD_SECRET` on my fork, so I added additional check to prevent failed run. And that concludes my work on #29379 🎉 --- <!-- Thank you for contributing to Servo! Please replace each `[ ]` by `[X]` when the step is complete, and replace `___` with appropriate data: --> - [x] `./mach build -d` does not report any errors - [x] `./mach test-tidy` does not report any errors - [x] These changes fix #29379 <!-- Either: --> - [ ] There are tests for these changes OR - [x] These changes do not require tests because it's CI <!-- Also, please make sure that "Allow edits from maintainers" checkbox is checked, so that we can help you if you get stuck somewhere along the way.--> <!-- Pull requests that do not address these steps are welcome, but they will require additional verification as part of the review process. -->
| * mach try & partitioned try jobssagudev2023-03-211-0/+38
| |
* | Package and upload linux w/ layout-2020 engine.Mukilan Thiyagarajan2023-03-101-0/+3
| | | | | | | | | | | | | | | | | | | | | | | | | | The layout-2020 variant will be consumed by the internal WPT trend [dashboard][1] only and won't be made available on servo.org. This change will be removed once the layout engine we will use going forward is finalized. [1]: https://servo.github.io/internal-wpt-dashboard/ Signed-off-by: Mukilan Thiyagarajan <me@mukilan.in>
* | Improve winit/mach segfault outputDelan Azabani2023-02-281-1/+4
|/
* Add cargo-fix and cargo-clippy to machsagudev2023-02-181-0/+48
|
* Warning instead of exception on unsupported Ubuntusagudev2023-02-171-1/+1
|
* Conditionally use certain GStreamer pluginsMartin Robinson2023-02-161-97/+112
| | | | | | | | | | | In old versions of GStreamer gstvideoconvertscale was split into two plugins, gstvideoscale and gstvideoconvert. This change makes the use of these three plugins conditional on their existence and has Windows use the two plugins from the older version of GStreamer. In addition, it starts to clean up the gstreamer.py file which is responsible for managing GStreamer libraries and plugins. This is the first change in a series of changes that seek to clean up the way these libraries are managed during the build.
* Auto merge of #29346 - mrobinson:aggregate-unexpected-logs, r=delanbors-servo2023-02-141-0/+3
|\ | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Aggregate unexpected results into logs This makes it easier to run `update-wpt` based on results from the bots by writing a new version of the raw log that filters out all tests with expected results. Then aggregate this output into the filtered results archive. A future version of this could aggregate all unexpected results that were not filtered as intermittents. <!-- Please describe your changes on the following line: --> --- <!-- Thank you for contributing to Servo! Please replace each `[ ]` by `[X]` when the step is complete, and replace `___` with appropriate data: --> - [x] `./mach build -d` does not report any errors - [x] `./mach test-tidy` does not report any errors - [x] These changes do not require tests because they are infrastructure changes. <!-- Also, please make sure that "Allow edits from maintainers" checkbox is checked, so that we can help you if you get stuck somewhere along the way.--> <!-- Pull requests that do not address these steps are welcome, but they will require additional verification as part of the review process. -->
| * Aggregate unexpected results into logsMartin Robinson2023-02-141-0/+3
| | | | | | | | | | | | This makes it easier to run `update-wpt` based on results from the bots. A future version of this could aggregate all unexpected results that were not filtered as intermittents.
* | Update gst plugin for videoconvert & videoscale elementsMukilan Thiyagarajan2023-02-111-2/+1
|/
* Auto merge of #29294 - servo:integrate-filter-intermittents, r=delanbors-servo2023-01-271-114/+7
|\ | | | | | | | | | | | | | | | | | | | | | | | | | | | | Integrate filter-intermittents into test-wpt This change integrates the `filter-intermittents` command into `test-wpt`. This is in preparation for future work on tracking intermittent failures. This change also: - Removes `ServoJsonFormatter` and replaces it with a generic WPT log handler which tracks unexpected results. - The intermittent filter is now controlled via environment variables and the GitHub version requires a token instead of credentials. - Output is saved to a single file and is always text. --- <!-- Thank you for contributing to Servo! Please replace each `[ ]` by `[X]` when the step is complete, and replace `___` with appropriate data: --> - [x] `./mach build -d` does not report any errors - [x] `./mach test-tidy` does not report any errors - [x] These changes do not require tests because they are changes to build infrastructure.
| * Integrate filter-intermittents into test-wptMartin Robinson2023-01-241-114/+7
| | | | | | | | | | | | | | | | | | | | | | This change integrates the filter-intermittents command into test-wpt. This is in preparation for future work on tracking intermittent failures. This change also: - Removes the SrvoJson logger and replaces it with a generic WPT log handler which tracks unexpected results. - The intermittent filter is now controlled via environment variables and the GitHub version requires a token instead of credentials. - Output is saved to a single file and is always text.
* | Invalidate latest nightly files in CloudFront.Mukilan Thiyagarajan2023-01-261-1/+24
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | After a new nightly build is uploaded to S3, bust the cache in CloudFront Edge servers with the CreateInvalidation AWS API. For each platform we use the `/nightly/<platform>/servo-latest.<ext>*` pattern to invalidate both package and sha256 files. As part of this change, a new policy has been attached to the "download.servo.org-uploads-from-travis" IAM user to allow the "cloudfront:CreateInvalidation" action. Since CloudFront [invalidates every version][1] of the cached file, regardless of the headers used for that version, this change should invalidate the different caches for 'Accept-Encoding' header. [1]: https://docs.aws.amazon.com/AmazonCloudFront/latest/DeveloperGuide/Invalidation.html#invalidation-specifying-objects Closes #29034 Signed-off-by: Mukilan Thiyagarajan <mukilanthiagarajan@gmail.com>
* | Update clangfmt version.Josh Matthews2023-01-241-1/+1
| |
* | Update LLVM toolchain version for WindowsMukilan Thiyagarajan2023-01-241-1/+1
|/ | | | | | | | | | | | | | | | | | | | | | Windows MSCV nightly builds have been failing since Jan 18. The failure is similar to the one @delan identified where the CI build started to fail after the version of LLVM included in Github Actions runner images was bumped from 14 to 15. As discussed on Zulip, we already have support in ./mach bootstrap to download the LLVM binaries from our s3 bucket. Following the documented [process], new LLVM 15.0.5 binaries have been uploaded to S3. This PR updates the bootstrap script to use this new LLVM version. [process]: https://github.com/servo/servo/wiki/Upgrading-the-windows-LLVM-binaries Signed-off-by: Mukilan Thiyagarajan <mukilanthiagarajan@gmail.com>
* Remove unused salt bootstrapping process.Josh Matthews2023-01-192-145/+0
|
* Auto merge of #29238 - servo:ci-fail-install-deps, r=jdmbors-servo2023-01-171-4/+6
|\ | | | | | | | | | | | | | | | | | | | | | | | | | | Fail faster on CI when Linux bootstrap fails Raise an exception when dependencies fail to install. Also split the run phase of the Linux bootstrap so that either of these failing commands will cause the job to fail. <!-- Please describe your changes on the following line: --> --- <!-- Thank you for contributing to Servo! Please replace each `[ ]` by `[X]` when the step is complete, and replace `___` with appropriate data: --> - [x] `./mach build -d` does not report any errors - [x] `./mach test-tidy` does not report any errors - [x] These changes do not require tests because they just change minor script issues.
| * Fail faster on CI when Linux bootstrap failsMartin Robinson2023-01-131-4/+6
| | | | | | | | | | | | Raise an exception when dependencies fail to install. Also split the run phase of the Linux bootstrap so that either of these failing commands will cause the job to fail.
* | Simplify Python code for running WPT testsMartin Robinson2023-01-131-45/+7
|/ | | | | | | | | | | Combine `run.py` and `update.py` into `servowpt.py` in order to allow them to share code. Import them directly into the mach script to avoid having to call `compile` and `exec` on the code. This makes it clearer how they are executed. In addition, move all of the setup into `setupwpt.py` to avoid differences between tests executed via mach and not. Finally, be more ambitious when detecting the build to use. If none was specified, try to use the one that exists between "release" and "debug."
* mach filter-intermittents: add progress and --json output modeDelan Azabani2023-01-101-3/+17
|
* Arch Linux patch for mach tools.michaelgrigoryan252022-12-151-0/+2
|
* Removed a typoMichael G2022-11-281-1/+1
|
* Added required dependencies for FedoraMichael G2022-11-281-1/+2
|
* Keep WPT import path around while running lints.Josh Matthews2022-11-091-1/+1
|
* Update bootstrap.pysagu2022-08-271-1/+1
| | | support Ubuntu 22.04
* Remove unnecessary always succeed flagcybai2022-08-161-8/+2
|
* Run WPT sync nightly on GitHub Actioncybai2022-08-161-2/+14
|
* Auto merge of #28909 - teymour-aldridge:minor-updates, r=jdmbors-servo2022-08-061-0/+1
|\ | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Minor updates --- <!-- Thank you for contributing to Servo! Please replace each `[ ]` by `[X]` when the step is complete, and replace `___` with appropriate data: --> - [x] `./mach build -d` does not report any errors - [x] `./mach test-tidy` does not report any errors - [ ] These changes fix #28905 (GitHub issue number if applicable) <!-- Either: --> - [ ] There are tests for these changes OR - [x] These changes do not require tests because they are very small changes <!-- Also, please make sure that "Allow edits from maintainers" checkbox is checked, so that we can help you if you get stuck somewhere along the way.--> <!-- Pull requests that do not address these steps are welcome, but they will require additional verification as part of the review process. -->
| * Fix ./mach bootstrap on fedora.Teymour Aldridge2022-08-031-0/+1
| |
* | Update clangfmt version.Josh Matthews2022-08-061-1/+1
| |
* | Upgade to LLVM 14 on Windows.Josh Matthews2022-08-051-1/+1
|/
* Update bootstrap to support Pop!_OS 22.04Thomas Hansen2022-04-291-2/+8
| | | linux mint kde neon too
* Unbreak lldb invocations.Josh Matthews2022-03-081-2/+1
|
* Update rustc to 3/6 nightly.Josh Matthews2022-03-061-1/+1
|
* Allow bootstrap on Pop!_OS 21.10Wu Yu Wei2021-12-301-1/+3
|
* update Ubuntu version check to > 21.10Gio de Guzman2021-12-152-2/+2
| | | | Update clang-format version check to use find() instead of startswith() to deal with distro-specific discrepancy
* chore(mach): update the required clang-format version to 13yvt2021-10-161-1/+1
|
* chore(mach): update the required clang-format version to 12yvt2021-10-041-1/+1
|
* Auto merge of #28598 - witte:simpleservo-macos, r=jdmbors-servo2021-09-141-1/+6
|\ | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Fix simpleservo binary check on macos <!-- Please describe your changes on the following line: --> When compiling libsimpleservo on MacOS 11.5.2 I would get an error at the end of the process saying ``` Error running mach: ['build', '-d', '--libsimpleservo'] The error occurred in code that was called by the mach command. This is either a bug in the called code itself or in the way that mach is calling it. You can invoke |./mach busted| to check if this issue is already on file. If it isn't, please use |./mach busted file| to report it. If |./mach busted| is misbehaving, you can also inspect the dependencies of bug 1543241. If filing a bug, please include the full output of mach, including this error message. The details of the failure are as follows: servo.command_base.BuildNotFound: No Servo binary found. Perhaps you forgot to run `./mach build`? File "/Users/yuriwitte/_HtmlTests/servo/python/servo/build_commands.py", line 734, in build self.get_binary_path(release, dev, target=target, simpleservo=libsimpleservo) File "/Users/yuriwitte/_HtmlTests/servo/python/servo/command_base.py", line 394, in get_binary_path raise BuildNotFound('No Servo binary found.' ``` The binary is there, it's just that `get_binary_path` on `command_base.py` checks for a libsimpleservo **.so** instead of a **.dylib**. This pr fixes this check. --- <!-- Thank you for contributing to Servo! Please replace each `[ ]` by `[X]` when the step is complete, and replace `___` with appropriate data: --> - [X] `./mach build -d` does not report any errors - [X] `./mach test-tidy` does not report any errors - [ ] These changes fix: didn't find a specific issue for this, but these two are also related to file extensions on MacOS: https://github.com/servo/servo/issues/27654 https://github.com/servo/servo/issues/27318 <!-- Either: --> - [ ] There are tests for these changes OR - [X] These changes do not require tests because: they _are_ the tests I guess? <!-- Also, please make sure that "Allow edits from maintainers" checkbox is checked, so that we can help you if you get stuck somewhere along the way.--> <!-- Pull requests that do not address these steps are welcome, but they will require additional verification as part of the review process. -->
generated by cgit v1.2.3 (git 2.25.1) at 2025-05-07 19:26:40 +0000