Skip to content

docs: ADR-001 Playwright stack decision — Path A (#463) — v1.3.76#689

Merged
Pratiyush merged 1 commit into
masterfrom
docs/463-playwright-decision
Apr 27, 2026
Merged

docs: ADR-001 Playwright stack decision — Path A (#463) — v1.3.76#689
Pratiyush merged 1 commit into
masterfrom
docs/463-playwright-decision

Conversation

@Pratiyush

Copy link
Copy Markdown
Owner

Summary

Architecture Decision Record for the Playwright Test Agents epic (#462). Records the Path-A choice: keep the existing Python tests/e2e/ suite as the gating contract, add @playwright/test Test Agents alongside under tests/agents/ once #464 unblocks. Pure docs PR — no code changes.

Closes #463 (parent: #462).

What changed

  • New ADR at docs/maintainers/ADR-001-playwright-stack.md (150 lines)
  • CHANGELOG entry [1.3.76] under Added
  • Version bump 1.3.75 → 1.3.76

What's new

Surface Change
ADR file new — captures Path A vs B vs C trade-offs and the chosen layout
docs/maintainers/ gains its first ADR; future stack/architecture decisions follow this template
Constraint surfaced #464 needs Node-install OK in sandboxed development; #465 + #466 partially deliverable without the bootstrap; #467 is hard-gated on #464

Decision summary

Path A — TS agents alongside Python pytest-playwright (chosen).

The existing 60+ scenario Python E2E suite is the contract today. Path A keeps that contract gating CI and adds the Test Agents trio (Planner / Generator / Healer) as a parallel surface. After one full epic cycle (after #467 ships) we re-evaluate whether full migration to Path B is worth the porting effort.

Path B (migrate to TS entirely) was rejected for now because:

  • 60+ pytest-bdd scenarios → @playwright/test translation is significant work
  • Visual regression uses Pillow image-diff which would need re-implementation
  • During the porting window the Python suite still gates every PR — there's no "done migrating" until Python is removed, and removing it before TS is at parity creates a coverage hole

Path C (skip agents entirely) was rejected because the auto-generated Healer / drift-detection workflow has a clear value prop for the 9 UI bugs filed this session (#452#460).

How to test it

ls docs/maintainers/ADR-001-playwright-stack.md   # exists, 150 lines
python3 -m pytest tests/test_readme_badges.py -q  # version badge still matches

Pre-merge checklist

  • One intent — single ADR adoption, no code changes
  • All CI checks green — local checks pass; CI to confirm
  • Linked issueCloses #463 in body + ADR file
  • Conventional-commit titledocs: ...
  • Tests added or updated — N/A; doc-only PR. Existing badge + version test still passes.
  • CHANGELOG.md updated[1.3.76] Added entry
  • Breaking changes flagged — N/A
  • No new runtime dependencies — N/A; ADR explicitly defers Node-install dep until playwright: bootstrap — run npx playwright init-agents --loop=claude, commit scaffold #464
  • No real session data — N/A
  • No machine-specific paths — N/A
  • Docs updated — this IS the docs update
  • Release notes drafted — see CHANGELOG.md [1.3.76]
  • UI verified — N/A
  • A11y verified — N/A
  • Commits GPG-signed — yes
  • Reviewer has read every changed line — diff is 5 files, 150 insertions, 3 deletions; all in one ADR file plus version-bump

Bundle

  • docs/maintainers/ADR-001-playwright-stack.md — new (150 lines): Context / Three paths / Decision: A / Constraints / Layout / Out of scope / References
  • llmwiki/__init__.py, pyproject.toml — version 1.3.75 → 1.3.76
  • README.md — version badge bump
  • CHANGELOG.md[1.3.76] Added entry

Out of scope / follow-ups

Next

After merge: tag v1.3.76, then ship the pragmatic-deliverable PR that lands specs/*.md for the 8 page types (closes the docs half of #465) + tests/e2e/features/regression/*.feature for UI bugs #452#460 (closes the regression-lock half of #466) — both achievable today on the Python harness without unblocking #464.

…#463) — v1.3.76

Closes #463 (parent: #462).

Records the Path-A decision: keep the existing Python tests/e2e/ suite
(pytest-playwright + pytest-bdd + pytest-html, 60+ scenarios, gating
contract) and add @playwright/test Test Agents alongside under a new
tests/agents/ once #464 unblocks.

Path A vs B vs C trade-offs documented; reasons for picking A:
- Zero migration risk
- 2,651-test contract stays gating
- Path-B (full migration) stays open for re-evaluation
- Path-C (skip agents) stays as fallback if agents don't pay off

Constraints: #464 needs Node-install OK in this repo's sandbox.
While #464 is blocked, most of #465 (specs/*.md) and #466 (regression
Gherkin scenarios for UI bugs #452-#460) can ship via the existing
Python harness — the agents bootstrap is only strictly required for
#467 (healer-in-CI auto-patch comments).
@Pratiyush Pratiyush merged commit 458cfd2 into master Apr 27, 2026
10 checks passed
@Pratiyush Pratiyush deleted the docs/463-playwright-decision branch April 27, 2026 19:01
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

playwright: decide pytest-playwright (Python) vs npx @playwright/test (TS) for the agents workflow

1 participant