ci: Run browser integration tests also on Firefox and Webkit #4232
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
We configure a job matrix to run browser tests in parallel, even though Karma could connect to all browsers at once. The reason for this is that, if run time is not a concern (less than 2min at the moment), running each browser as a separate job seemed more robust and easier to spot browser-specific problems when a test fails. We get all browser binaries from Playwright, so we don't need to rely on versions installed on the host system. (And this is also the easiest way to get a headless Webkit browser on Ubuntu) We include the Playwright browsers in cache to avoid having to download them again over and over. We install Webkit system dependencies on every run of the Webkit tests because it is inconvenient to cache and restore system dependencies. Installing then is safer, and perhaps even faster, than dealing with the cache.
rhcarvalho
requested review from
a team,
AbhiPrasad and
iker-barriocanal
and removed request for
a team
size-limit report
|
rhcarvalho
commented
Dec 3, 2021
| needs: job_build | ||
| runs-on: ubuntu-latest | ||
| timeout-minutes: 15 | ||
| timeout-minutes: 10 |
There was a problem hiding this comment.
Reducing this from 15 to 10 (another value we use in jobs in this workflow) to emphasize these tests should run relatively fast. The 10 min timeout is for each browser/job.
rhcarvalho
commented
Dec 3, 2021
| @@ -10,6 +10,7 @@ env: | |||
| CACHED_DEPENDENCY_PATHS: | | |||
| ${{ github.workspace }}/node_modules | |||
| ${{ github.workspace }}/packages/**/node_modules | |||
| ~/.cache/ms-playwright/ | |||
There was a problem hiding this comment.
An alternative to this was to
Set
PLAYWRIGHT_BROWSERS_PATH=0as the environment variable before runningnpm install. This will download the browser binaries in thenode_modulesdirectory and cache them with the package code.
rhcarvalho
deleted the
rhcarvalho/browser-integration-tests-firefox-webkit
branch
onurtemizkan
pushed a commit
that referenced
this pull request
Dec 19, 2021
We configure a job matrix to run browser tests in parallel, even though Karma could connect to all browsers at once. The reason for this is that, if run time is not a concern (less than 2min at the moment), running each browser as a separate job seemed more robust and easier to spot browser-specific problems when a test fails. We get all browser binaries from Playwright, so we don't need to rely on versions installed on the host system. (And this is also the easiest way to get a headless Webkit browser on Ubuntu) We include the Playwright browsers in cache to avoid having to download them again over and over. We install Webkit system dependencies on every run of the Webkit tests because it is inconvenient to cache and restore system dependencies. Installing then is safer, and perhaps even faster, than dealing with the cache.
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
This is a follow up to #4226, expanding the target browsers from Chrome to Chrome + Firefox + Webkit.
We configure a job matrix to run browser tests in parallel, even though Karma could connect to all browsers at once. The reason for this is that, if run time is not a concern (less than 2min at the moment), running each browser as a separate job seemed more robust and easier to spot browser-specific problems when a test fails.
We get all browser binaries from Playwright, so we don't need to rely on versions installed on the host system. (And this is also the easiest way to get a headless Webkit browser on Ubuntu)
We include the Playwright browsers in cache to avoid having to download them again over and over.
We install Webkit system dependencies on every run of the Webkit tests because it is inconvenient to cache and restore system dependencies. Installing then is safer, and perhaps even faster, than dealing with the cache.