docs: README claim audit + badge-drift regression test (#682) — v1.3.75#688
Merged
Conversation
Closes #682. 7 stale or wrong claims fixed: - tests-2363 → 2651 badge (pytest actually collects 2651) - pip install -e '.[pdf]' (extra was removed in simplification sweep) → replaced with real extras [graph]/[dev]/[e2e]/[all] - "pypdf is an optional extra for PDF ingestion" → real extras list - "the unit suite (472 tests)" → 2,651 tests - "every command in 60 seconds" → 90 seconds + link to VHS recording - TODO re-record demo GIF for v1.3 → embed docs/demo.gif (shipped 1.3.67) - "~300 MB for Chromium" → "several hundred MB" New regression test: test_test_count_badge_within_window_of_actual runs pytest --collect-only and fails when the badge drifts more than ±15% from the actually- collected count. Catches the exact rot mode that triggered this audit (badge silently ~290 tests behind reality through several PR cycles).
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Walks every numeric/factual claim in `README.md` against the current code + data and fixes the seven that drifted. Pairs the audit with a regression test that fails CI when the test-count badge silently rots more than ±15% behind reality.
Closes #682.
What changed
Verified, no change needed:
What's new
Behavioural delta
How to test it
```bash
python3 -m pytest tests/test_readme_badges.py -v # 10 pass, including the new test
python3 -m pytest tests/ -q -m "not slow" # full suite green
python3 -m pytest tests/ --collect-only 2>&1 | grep "tests collected"
```
Pre-merge checklist
Bundle
Out of scope / follow-ups
Next
After merge: tag `v1.3.75`, then move to the Playwright Test Agents epic (#462–#467). Per memory: open #462 with a phased plan first, then #463 (decide pytest-playwright vs `npx @playwright/test`) before any code PRs in the family.