Skip to content

reenable more of the CI matrix #499

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 3 commits into from
Mar 12, 2021
Merged

Conversation

Conduitry
Copy link
Member

Now that this is free, we can afford it. Rather than simply uncommenting the old lines, I've also enabled tests on Node 15. When 16 is released and 15 goes EOL, we can update this again.

@Conduitry
Copy link
Member Author

I don't know why the heck Node 12 on Ubuntu is failing. That's one of the ones that was already running.

@GrygrFlzr
Copy link
Member

Possibly #404 at play. Let me force re-run all jobs.

@Conduitry
Copy link
Member Author

That's possible. I had tried running them twice already, and Node 12 Ubuntu failed both times, although I didn't look carefully to see whether it was the same failure message both times.

@GrygrFlzr
Copy link
Member

GrygrFlzr commented Mar 12, 2021

Okay, it seems to be at least consistently erroring on:

dev:basics
2:54:59 AM [vite] Error when evaluating SSR module /home/runner/work/kit/kit/packages/kit/test/apps/basics/src/routes/errors/module-scope-server.svelte:
Error: Crashing now
    at /home/runner/work/kit/kit/packages/kit/test/apps/basics/src/routes/errors/module-scope-server.svelte:5:12
    at instantiateModule (/home/runner/work/kit/kit/node_modules/.pnpm/[email protected]/node_modules/vite/dist/node/chunks/dep-e0f09032.js:68041:166)
2:54:59 AM [vite] Error when evaluating SSR module /home/runner/work/kit/kit/packages/kit/test/apps/basics/src/routes/errors/module-scope-server.svelte:
Error: Crashing now
    at /home/runner/work/kit/kit/packages/kit/test/apps/basics/src/routes/errors/module-scope-server.svelte:5:12
    at instantiateModule (/home/runner/work/kit/kit/node_modules/.pnpm/[email protected]/node_modules/vite/dist/node/chunks/dep-e0f09032.js:68041:166) (x2)

I can also replicate that specific error locally, so I think there's an actual issue involved that was previously skipped in CI somehow.

@GrygrFlzr
Copy link
Member

GrygrFlzr commented Mar 12, 2021

I debugged it a bit and it seems that the error page in that test case is correctly rendering, but vite is also re-throwing the SSR error and triggering uvu. Not sure why this wasn't an issue before, though.

@GrygrFlzr
Copy link
Member

Nevermind - while the vite SSR error does appear, it doesn't actually fail the uvu test - the set of failing tests keep changing between runs, usually something related to clicking a link, regexp routes, or redirect loops.

I think it's just a case of more test runs = more chances of failure due to the tests themselves being flaky and timing-sensitive.

@GrygrFlzr
Copy link
Member

Okay, I merged #501 into master which should have solved most of the test stability issues.

@Conduitry go ahead and pull master to re-test the full matrix.

@Conduitry
Copy link
Member Author

Success! Just the Windows ones failing now. Thanks @GrygrFlzr !

@GrygrFlzr
Copy link
Member

Huh, the failing Windows tests cancelled the still-running macOS test for Node 15. Should macOS be shifted earlier in the OS matrix so that it has a chance to complete the run before the Windows CIs fail?

@Conduitry
Copy link
Member Author

That's probably a good idea. Updated.

@benmccann benmccann merged commit 55eb2d7 into sveltejs:master Mar 12, 2021
@Conduitry Conduitry deleted the expand-ci-matrix branch March 12, 2021 16:52
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants