test: spike Vitest browser mode as WTR replacement by iOvergaard · Pull Request #1299 · umbraco/Umbraco.UI

iOvergaard · 2026-02-26T10:58:34Z

Summary

Spike for AB#65042 — evaluate Vitest as a replacement for Web Test Runner.

Full migration of all 80 component test files from @open-wc/testing + WTR to vitest-browser-lit + Vitest 4 browser mode.

All 809 tests pass (3 skipped), across 80 test files
Removes all @open-wc/testing and WTR dependencies
CI runs all 3 browsers (Chromium + Firefox + WebKit), local dev runs Chromium only — same as WTR setup

Performance (local, Chromium only, same machine)

Runner	Run 1	Run 2	Run 3	Tests
WTR	18s	21s	21.5s	807 passed
Vitest 4	8.5s	14.6s	9.7–10.8s	809 passed

Vitest is roughly ~2x faster on warm runs. The variance comes from Vite's dependency pre-bundling cache — cold starts are slower, cached runs are fast. Actual test execution is consistently ~5s in Vitest.

Key findings

	WTR	Vitest
Runner	`@web/test-runner` + Mocha + Chai	`vitest` + `@vitest/browser`
Rendering	`@open-wc/testing` `fixture()`	`vitest-browser-lit` `render()`
Assertions	Chai + plugins	Vitest (Jest-compatible)
a11y	`@open-wc/testing` `.accessible()`	`axe-core` direct + custom matcher
Mouse interaction	`@web/test-runner-commands` `sendMouse`	`@vitest/browser/context` `userEvent.click()`
Shadow DOM assertions	`@open-wc/testing` `.shadowDom.toBe()`	No equivalent — test behavior instead
Browsers (local)	Playwright (Chromium)	Playwright (Chromium)
Browsers (CI)	Playwright (Chromium + Firefox + WebKit)	Playwright (Chromium + Firefox + WebKit)

Migration patterns

render(html...).container.querySelector('tag')! replaces fixture(html...)
await element.updateComplete replaces elementUpdated(element)
vitest-axe does not work in browser mode — use axe-core directly with a custom matcher
Mocha done() callbacks → async/await with Promises
Chai .to.equal() → .toBe(), .to.have.property() → .toHaveProperty(), etc.
UUITestMouse now uses userEvent.click() from @vitest/browser/context
No shadow DOM string comparison available — assert on behavior (text content, attributes) instead
Empty describe() blocks error in Vitest ("No test found in suite")

Manual fixes needed beyond script

boolean-input — dynamic tag names require lit/static-html.js + manual fixture
file-dropzone — done() callback pattern → Promise-based
form-validation-message — nested describe inside it (Vitest forbids this)
avatar — shadowDom.toBe() → behavioral assertions
breadcrumb-item — fixture() calls missed by script
pagination — buttons re-render on click, need .querySelector('.active') + double updateComplete
Chai property chains (.to.false, .to.undefined) — silently pass but don't assert

Test plan

npx vitest run — 809 tests pass, 3 skipped, 80 files
CI passes (SonarCloud clean)
CI passes with all 3 browsers

🤖 Generated with Claude Code

github-actions · 2026-02-26T11:00:10Z

Azure Static Web Apps: Your stage site is ready! Visit it here: https://delightful-beach-055ecb503-1299.westeurope.azurestaticapps.net

github-actions · 2026-02-26T11:19:48Z

Azure Static Web Apps: Your stage site is ready! Visit it here: https://delightful-beach-055ecb503-1299.westeurope.azurestaticapps.net

github-actions · 2026-02-26T11:22:28Z

Azure Static Web Apps: Your stage site is ready! Visit it here: https://delightful-beach-055ecb503-1299.westeurope.azurestaticapps.net

github-actions · 2026-02-26T11:24:10Z

Azure Static Web Apps: Your stage site is ready! Visit it here: https://delightful-beach-055ecb503-1299.westeurope.azurestaticapps.net

github-actions · 2026-02-26T11:38:43Z

Azure Static Web Apps: Your stage site is ready! Visit it here: https://delightful-beach-055ecb503-1299.westeurope.azurestaticapps.net

github-actions · 2026-02-26T14:50:28Z

Azure Static Web Apps: Your stage site is ready! Visit it here: https://delightful-beach-055ecb503-1299.westeurope.azurestaticapps.net

github-actions · 2026-02-26T15:22:20Z

Azure Static Web Apps: Your stage site is ready! Visit it here: https://delightful-beach-055ecb503-1299.westeurope.azurestaticapps.net

github-actions · 2026-02-26T15:24:20Z

Azure Static Web Apps: Your stage site is ready! Visit it here: https://delightful-beach-055ecb503-1299.westeurope.azurestaticapps.net

github-actions · 2026-02-26T15:26:55Z

Azure Static Web Apps: Your stage site is ready! Visit it here: https://delightful-beach-055ecb503-1299.westeurope.azurestaticapps.net

github-actions · 2026-02-26T15:30:58Z

Azure Static Web Apps: Your stage site is ready! Visit it here: https://delightful-beach-055ecb503-1299.westeurope.azurestaticapps.net

github-actions · 2026-02-26T15:34:52Z

Azure Static Web Apps: Your stage site is ready! Visit it here: https://delightful-beach-055ecb503-1299.westeurope.azurestaticapps.net

github-actions · 2026-02-26T15:36:07Z

Azure Static Web Apps: Your stage site is ready! Visit it here: https://delightful-beach-055ecb503-1299.westeurope.azurestaticapps.net

Replace Web Test Runner with Vitest 4 browser mode + vitest-browser-lit for the button and input component test suites (81 tests). - Add Vitest config with Playwright chromium browser provider - Convert button.test.ts and input.test.ts to vitest-browser-lit - Add axe-core custom matcher (vitest-axe incompatible with browser mode) - Add src/internal/test/a11y.ts helper for axe runs - Remove WTR config and all @open-wc/testing dependencies - Update CI workflow for Vitest - Exclude vitest.setup.ts and vite.config.ts from build tsconfig - Restrict test include to converted files only (spike scope) Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

Full migration of test suite from @open-wc/testing + WTR to vitest-browser-lit + Vitest 4. All 809 tests pass (3 skipped), total runtime ~8.5s (down from ~45s with WTR). Key changes: - fixture() -> render().container.querySelector() - Chai assertions -> Vitest expect() - @open-wc/testing imports -> vitest-browser-lit - UUITestMouse uses userEvent.click() from @vitest/browser/context - done() callbacks converted to async/await with Promises - Conversion script at scripts/convert-tests.mjs for reference Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

The convert-tests.mjs script was used during the WTR → Vitest migration and is no longer needed. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

Matches the previous WTR behavior: Chromium only locally, all 3 browsers on CI via process.env.CI check. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

Firefox returns subpixel scrollTop/scrollLeft values (e.g. 42.316 instead of 42). Use Math.round() for cross-browser compatibility. Also fix Assertion type augmentation generic for toHaveNoViolations. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

…ontext Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

github-actions · 2026-02-26T16:06:04Z

Azure Static Web Apps: Your stage site is ready! Visit it here: https://delightful-beach-055ecb503-1299.westeurope.azurestaticapps.net

Copilot

Pull request overview

This PR migrates the test infrastructure from Web Test Runner (WTR) with @open-wc/testing to Vitest 4 browser mode. The migration converts all 80 test files (809 tests) to use vitest-browser-lit for rendering and Vitest's Jest-compatible assertions. Performance is improved by approximately 2x (warm runs: ~10s vs ~20s). The migration includes a custom axe-core accessibility matcher, replaces the oneEvent helper, and updates CI to run tests across Chromium, Firefox, and WebKit browsers.

Changes:

Replaces WTR + Mocha + Chai testing stack with Vitest 4 browser mode
Migrates all test files to use render() from vitest-browser-lit and Vitest assertions
Implements custom toHaveNoViolations() matcher for axe-core accessibility testing

Reviewed changes

Copilot reviewed 90 out of 92 changed files in this pull request and generated no comments.

Show a summary per file

File	Description
web-test-runner.config.js	Removed WTR configuration file
vitest.setup.ts	New custom matcher for accessibility testing
vite.config.ts	Added Vitest configuration with browser mode
tsconfig.json	Updated types to include Vitest globals and browser
package.json	Updated dependencies and test scripts
.github/workflows/tests.yml	Updated CI to install Playwright browsers
src/internal/test/index.ts	Simplified mouse helper using userEvent
src/internal/test/a11y.ts	New axe-core wrapper for accessibility testing
src/internal/version.test.ts	Sample migration showing before/beforeAll changes
src/components//*.test.ts	Migrated all 80 component test files

Removes 32 duplicate oneEvent() definitions from individual test files and exports it from src/internal/test/index.ts instead. Also narrows lint-staged tsc-files glob to *.element.ts to avoid type-checking test files in pre-commit (they lack the vitest setup context that provides custom matchers). Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

github-actions · 2026-02-27T07:56:45Z

Azure Static Web Apps: Your stage site is ready! Visit it here: https://delightful-beach-055ecb503-1299.westeurope.azurestaticapps.net

github-actions · 2026-02-27T10:02:34Z

Azure Static Web Apps: Your stage site is ready! Visit it here: https://delightful-beach-055ecb503-1299.westeurope.azurestaticapps.net

github-actions · 2026-02-27T10:10:04Z

Azure Static Web Apps: Your stage site is ready! Visit it here: https://delightful-beach-055ecb503-1299.westeurope.azurestaticapps.net

sonarqubecloud · 2026-02-27T12:07:27Z

Quality Gate passed

Issues
0 New issues
0 Accepted issues

Measures
0 Security Hotspots
0.0% Coverage on New Code
0.0% Duplication on New Code

See analysis details on SonarQube Cloud

github-actions · 2026-02-27T12:08:20Z

Azure Static Web Apps: Your stage site is ready! Visit it here: https://delightful-beach-055ecb503-1299.westeurope.azurestaticapps.net

iOvergaard force-pushed the v2/spike/65042-vitest branch from ace783a to dcd57f7 Compare February 26, 2026 11:22

iOvergaard force-pushed the v2/spike/65042-vitest branch from dcd57f7 to e93a2cb Compare February 26, 2026 11:37

iOvergaard force-pushed the v2/spike/65042-vitest branch from f4f335e to 9377f8b Compare February 26, 2026 15:20

iOvergaard and others added 9 commits February 26, 2026 17:03

chore: remove one-time test conversion script

59251ed

The convert-tests.mjs script was used during the WTR → Vitest migration and is no longer needed. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

chore: revert unrelated USAGE-WITHOUT-BUNDLER.md change

5f63ea8

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

test: run all 3 browsers (Chromium, Firefox, WebKit) on CI

14508f5

Matches the previous WTR behavior: Chromium only locally, all 3 browsers on CI via process.env.CI check. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

fix(test): use vitest/browser instead of deprecated @vitest/browser/c…

31705dc

…ontext Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

chore(test): suppress Lit dev mode stderr noise in test output

6d84502

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

test: convert version.test.ts to Vitest and silence UUI version warnings

1e3ea22

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

iOvergaard force-pushed the v2/spike/65042-vitest branch from 19b274e to 1e3ea22 Compare February 26, 2026 16:04

iOvergaard marked this pull request as ready for review February 27, 2026 07:48

Copilot AI review requested due to automatic review settings February 27, 2026 07:48

iOvergaard added the internal Updates to dependencies, rewrite of files, cleanup, chores label Feb 27, 2026

Copilot started reviewing on behalf of iOvergaard February 27, 2026 07:48 View session

iOvergaard added the dependencies Pull requests that update a dependency file label Feb 27, 2026

Copilot AI reviewed Feb 27, 2026

View reviewed changes

iOvergaard and others added 2 commits February 27, 2026 08:54

chore: remove failure screenshots and add __screenshots__ to gitignore

ef9379e

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

Merge remote-tracking branch 'origin/main' into v2/spike/65042-vitest

0c2940f

iOvergaard added 2 commits February 27, 2026 11:08

Merge remote-tracking branch 'origin/main' into v2/spike/65042-vitest

5500609

updates lockfile

313f040

madsrasmussen approved these changes Feb 27, 2026

View reviewed changes

Merge branch 'main' into v2/spike/65042-vitest

3fe65da

iOvergaard enabled auto-merge (squash) February 27, 2026 12:07

iOvergaard merged commit a0d683a into main Feb 27, 2026
10 of 11 checks passed

iOvergaard deleted the v2/spike/65042-vitest branch February 27, 2026 12:09

Conversation

iOvergaard commented Feb 26, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Summary

Performance (local, Chromium only, same machine)

Key findings

Migration patterns

Manual fixes needed beyond script

Test plan

Uh oh!

github-actions Bot commented Feb 26, 2026

Uh oh!

github-actions Bot commented Feb 26, 2026

Uh oh!

github-actions Bot commented Feb 26, 2026

Uh oh!

github-actions Bot commented Feb 26, 2026

Uh oh!

github-actions Bot commented Feb 26, 2026

Uh oh!

github-actions Bot commented Feb 26, 2026

Uh oh!

github-actions Bot commented Feb 26, 2026

Uh oh!

github-actions Bot commented Feb 26, 2026

Uh oh!

github-actions Bot commented Feb 26, 2026

Uh oh!

github-actions Bot commented Feb 26, 2026

Uh oh!

github-actions Bot commented Feb 26, 2026

Uh oh!

github-actions Bot commented Feb 26, 2026

Uh oh!

github-actions Bot commented Feb 26, 2026

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Reviewed changes

Uh oh!

github-actions Bot commented Feb 27, 2026

Uh oh!

github-actions Bot commented Feb 27, 2026

Uh oh!

github-actions Bot commented Feb 27, 2026

Uh oh!

sonarqubecloud Bot commented Feb 27, 2026

Quality Gate passed

Uh oh!

github-actions Bot commented Feb 27, 2026

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

iOvergaard commented Feb 26, 2026 •

edited

Loading