Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
41 commits
Select commit Hold shift + click to select a range
198e2d2
[test] Add demo-based axe-core accessibility tests for mui-material
siriwatknp Apr 21, 2026
d87e773
[test] Move enrollment to config.ts with status field; drop docs widget
siriwatknp Apr 21, 2026
449aaeb
[test] Split a11y results into one JSON per component
siriwatknp Apr 21, 2026
5de4203
[test] Enable every testable component with VRT auto-discovery
siriwatknp Apr 21, 2026
cd91b9c
[test] Rename skipRules to skipAssertions; drop Box from roster
siriwatknp Apr 21, 2026
b2fb66f
[test] Drop layout/behavior components from the roster
siriwatknp Apr 21, 2026
f9b4201
[test] Drop status field; pending entries become TODO comments
siriwatknp Apr 21, 2026
238de9f
[test] Fold axe into VRT screenshot loop
siriwatknp Apr 21, 2026
8c19506
fix ci
siriwatknp Apr 21, 2026
33ea314
[test] Rename config.ts to a11yConfig.ts
siriwatknp Apr 21, 2026
7d70287
[test] Move a11y infra to test/regressions/a11y/
siriwatknp Apr 21, 2026
6751af1
[test] Drop docs:a11y wrapper; fold prettier into test:regressions
siriwatknp Apr 21, 2026
ccc7df4
fix ci
siriwatknp Apr 21, 2026
f337bc7
[test] Separate screenshot vs a11y exclusions via demoMeta
siriwatknp Apr 24, 2026
909aea1
[test] Move a11y results to docs/data/material/a11y
siriwatknp Apr 24, 2026
e7c965a
[test] Regenerate a11y results for expanded demo coverage
siriwatknp Apr 24, 2026
b189f54
[test] Demo showing a11y results consumption
siriwatknp Apr 24, 2026
bc398a2
[test] Refactor demoMeta to rule arrays; per-demo a11y output
siriwatknp Apr 27, 2026
5361e9a
[test] Regenerate a11y results as per-demo files
siriwatknp Apr 27, 2026
d68a6c1
[test] Scope A11Y_RULES to buttons only
siriwatknp Apr 27, 2026
34db217
[test] Trim a11y results to buttons-only scope
siriwatknp Apr 27, 2026
2620d1e
[docs] Document filtered a11y test runs in AGENTS.md
siriwatknp Apr 27, 2026
884d665
[test] Rename ROUTE_RE to ROUTE_REGEX
siriwatknp Apr 27, 2026
cb5f024
Merge remote-tracking branch 'upstream/master' into docs/a11y-demo-st…
siriwatknp Apr 27, 2026
d41ef72
[test] Co-locate a11y output at components/{slug}/{slug}.a11y.json
siriwatknp Apr 27, 2026
de231e9
[test] Point test:regressions prettier at *.a11y.json
siriwatknp Apr 27, 2026
8f61765
Merge branch 'master' into test/a11y-demos
siriwatknp Apr 27, 2026
9492162
fix ci
siriwatknp Apr 27, 2026
d7e0b5e
[test] Drop field-merge in demoMeta; overrides restate every field
siriwatknp Apr 27, 2026
fcea236
Merge branch 'master' of github.com:mui/material-ui into test/a11y-demos
siriwatknp Apr 27, 2026
f3feac8
pnpm dedupe
siriwatknp Apr 27, 2026
0ea22f4
revert lock file
siriwatknp Apr 27, 2026
05c6b9d
update lock file
siriwatknp Apr 27, 2026
f470514
Merge branch 'master' of github.com:mui/material-ui into test/a11y-demos
siriwatknp Apr 30, 2026
15af48c
install
siriwatknp Apr 30, 2026
fb166a0
apply suggestion
siriwatknp May 4, 2026
6aa2965
Merge branch 'master' of github.com:mui/material-ui into test/a11y-demos
siriwatknp May 4, 2026
8af9234
Merge remote-tracking branch 'upstream/master' into test/a11y-demos
siriwatknp May 4, 2026
f0a992c
fix ci
siriwatknp May 4, 2026
64f462e
Merge branch 'master' of github.com:mui/material-ui into test/a11y-demos
siriwatknp May 5, 2026
fb71594
[test] Wire waitForSelector; prune stale a11y JSON
siriwatknp May 5, 2026
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
3 changes: 3 additions & 0 deletions .circleci/config.yml
Original file line number Diff line number Diff line change
Expand Up @@ -267,6 +267,9 @@ jobs:
- run:
name: Run visual regression tests
command: xvfb-run pnpm test:regressions
- run:
name: A11y results committed?
command: git add -A && git diff --exit-code --staged
- run:
name: Build packages for fixtures
command: pnpm release:build
Expand Down
40 changes: 40 additions & 0 deletions AGENTS.md
Original file line number Diff line number Diff line change
Expand Up @@ -161,6 +161,46 @@ describe('Button', () => {
});
```

### Accessibility Testing

axe-core runs inside the visual-regression Playwright loop (`test/regressions/index.test.js`) — no separate browser session. Screenshots and a11y are independent: a demo can opt out of one and still run the other.

Key files:

- `test/regressions/demoMeta.ts` — `SCREENSHOT_RULES` and `A11Y_RULES` arrays, matched last-wins (no inheritance: overrides restate every field) against `docs/data/material/components/{slug}/{Demo}` (minimatch globs).
- `test/regressions/a11y/axe.ts` — asserts `color-contrast` and `link-in-text-block` unless listed in `skipAssertions`.
- `test/regressions/a11y/a11yReporter.ts` — writes one file per slug at `docs/data/material/components/{slug}/{slug}.a11y.json`, an object keyed by demo name.

Enroll a component (slug-wide, or narrow with brace-glob):

```ts
// test/regressions/demoMeta.ts
{ test: 'docs/data/material/components/alert/*', enabled: true, skipAssertions: ['color-contrast'] },
{ test: 'docs/data/material/components/buttons/{BasicButtons,ColorButtons}', enabled: true },
```

Override a specific demo: append a per-demo rule _after_ the slug-wide rule (last-match-wins; the override must restate every field it wants):

```ts
{ test: 'docs/data/material/components/popover/AnchorPlayground', enabled: false }, // Redux isolation
```

Run `pnpm test:regressions` to refresh the `*.a11y.json` files. CI fails if any are stale.

For local iteration, scope the run with vitest's `-t` test-name filter (matched against the `it()` strings, which contain the route). Non-matching tests are skipped — their bodies don't execute, so the browser never navigates to those routes.

```bash
# in one terminal
pnpm test:regressions:server

# in another — note no `--`, pnpm forwards args directly
pnpm test:regressions:run -t '/docs-components-buttons/' # one slug
pnpm test:regressions:run -t '/docs-components-buttons/BasicButtons$' # one demo
pnpm test:regressions:run -t '/docs-components-(buttons|chips)/' # multiple slugs
```

Filtered runs only refresh the matched slugs' `*.a11y.json`. Run the unfiltered `pnpm test:regressions` before pushing.

### Imports

Use one-level deep imports to avoid bundling entire packages:
Expand Down
56 changes: 56 additions & 0 deletions docs/data/material/components/buttons/buttons.a11y.json
Original file line number Diff line number Diff line change
@@ -0,0 +1,56 @@
{
"BasicButtons": {
"passedRules": [
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I would recommend prototyping the docs before generating for all other components. That way we can benchmark this data format first.

Copy link
Copy Markdown
Member Author

@siriwatknp siriwatknp Apr 30, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The structure is brought from #47895 to be able to show like this https://deploy-preview-47895--material-ui.netlify.app/material-ui/react-card/#accessibility-compliance

image

The thought behind:

  • each component/demo could have different scenario tests, so must show what rules are tested
  • the rule name is used to link to the source for the details of the rule to https://dequeuniversity.com/rules/axe/*
  • the passed/failed are for degbuggin/plan to fix them. we can decide if we want to display them or not.

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Will there be an overview? As in, as a customer I'd like to know:

  1. If MUI is committed to a certain standard? Can I Hold MUI to that standard?
  2. Which trade-offs are there to be made?

I can't be required to open each and every demo and aggregate myself to answer that question, right?

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Absolutely, there should be an overview but we are not in the stage to do it (at least in this PR).

The main issue (the obvious one) is the color contrast which the team agree on having the test setup first, see how many affected, update the colors and iterate until the tests passed.

At the end, when we are confident about the compliance. We will have that overview page.

This PR is like a first step toward the goal. Let me know if you think any task should be slot in this PR apart from the setup.

"aria-allowed-attr",
"aria-conditional-attr",
"aria-prohibited-attr",
"aria-valid-attr",
"aria-valid-attr-value",
"button-name",
"color-contrast",
"nested-interactive",
"target-size"
],
"failedRules": [],
"testedRules": {
"wcag2a": [
"aria-allowed-attr",
"aria-conditional-attr",
"aria-prohibited-attr",
"aria-valid-attr-value",
"aria-valid-attr",
"button-name",
"nested-interactive"
],
"wcag2aa": ["color-contrast"],
"wcag22aa": ["target-size"]
}
},
"ColorButtons": {
"passedRules": [
"aria-allowed-attr",
"aria-conditional-attr",
"aria-prohibited-attr",
"aria-valid-attr",
"aria-valid-attr-value",
"button-name",
"color-contrast",
"nested-interactive",
"target-size"
],
"failedRules": [],
"testedRules": {
"wcag2a": [
"aria-allowed-attr",
"aria-conditional-attr",
"aria-prohibited-attr",
"aria-valid-attr-value",
"aria-valid-attr",
"button-name",
"nested-interactive"
],
"wcag2aa": ["color-contrast"],
"wcag22aa": ["target-size"]
}
}
}
4 changes: 3 additions & 1 deletion package.json
Original file line number Diff line number Diff line change
Expand Up @@ -57,7 +57,7 @@
"test:e2e:dev": "pnpm -F ./test/e2e dev",
"test:e2e-website": "playwright test test/e2e-website --config test/e2e-website/playwright.config.ts",
"test:e2e-website:dev": "cross-env PLAYWRIGHT_TEST_BASE_URL=http://localhost:3000 playwright test test/e2e-website --config test/e2e-website/playwright.config.ts",
"test:regressions": "cross-env NODE_ENV=production pnpm test:regressions:build && concurrently --success first --kill-others \"pnpm test:regressions:run\" \"pnpm test:regressions:server\"",
"test:regressions": "cross-env NODE_ENV=production pnpm test:regressions:build && concurrently --success first --kill-others \"pnpm test:regressions:run\" \"pnpm test:regressions:server\" && prettier --write \"docs/data/material/components/**/*.a11y.json\"",
"test:regressions:build": "vite build test/regressions",
"test:regressions:dev": "vite test/regressions --port 5001",
"test:regressions:run": "vitest run -r ./test/regressions/",
Expand Down Expand Up @@ -112,6 +112,7 @@
"@vitejs/plugin-react": "^5.1.1",
"@vitest/browser-playwright": "^4.0.13",
"@vitest/coverage-v8": "^4.0.13",
"axe-core": "4.11.1",
"babel-plugin-istanbul": "7.0.1",
"babel-plugin-module-resolver": "5.0.3",
"chalk": "5.6.2",
Expand All @@ -128,6 +129,7 @@
"jsdom": "26.1.0",
"lerna": "9.0.7",
"markdownlint-cli2": "0.22.1",
"minimatch": "10.2.4",
"nx": "20.8.4",
"pkg-pr-new": "0.0.66",
"playwright": "1.59.1",
Expand Down
40 changes: 27 additions & 13 deletions pnpm-lock.yaml

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

Loading
Loading