Skip to content

Commit 3dc5044

Browse files
aviadr1claude
andauthored
feat: L1+L2 verification discipline — path tracing, invariant statements, test coverage warnings (#23)
Add mandatory verification policies (kaizen #11, #15, #17): - CLAUDE.md: path-tracing checklist, invariant statement requirement, runtime artifact verification - Pre-commit hook: advisory warning when source files are staged without corresponding test changes - Global CLAUDE.md: path tracing and test invariant requirements for container agents Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
1 parent c6f7e02 commit 3dc5044

File tree

3 files changed

+74
-0
lines changed

3 files changed

+74
-0
lines changed

.husky/pre-commit

Lines changed: 17 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -9,3 +9,20 @@ npm run format:fix
99
if [ -n "$STAGED_TS" ]; then
1010
echo "$STAGED_TS" | xargs git add
1111
fi
12+
13+
# Advisory: warn if staged source files have no corresponding test changes (kaizen #15)
14+
STAGED_SRC=$(git diff --cached --name-only --diff-filter=d | \
15+
grep -E '\.(ts|js|tsx|jsx)$' | \
16+
grep -vE '(\.test\.|\.spec\.|__tests__|\.config\.|vitest\.|CLAUDE\.md|\.claude/|\.husky/)' || true)
17+
18+
STAGED_TESTS=$(git diff --cached --name-only --diff-filter=d | \
19+
grep -E '\.(test|spec)\.(ts|js|tsx|jsx)$' || true)
20+
21+
if [ -n "$STAGED_SRC" ] && [ -z "$STAGED_TESTS" ]; then
22+
SRC_COUNT=$(echo "$STAGED_SRC" | wc -l | tr -d ' ')
23+
echo "" >&2
24+
echo "⚠️ $SRC_COUNT source file(s) staged with no test changes:" >&2
25+
echo "$STAGED_SRC" | sed 's/^/ - /' >&2
26+
echo " Consider adding tests before this reaches PR review." >&2
27+
echo "" >&2
28+
fi

CLAUDE.md

Lines changed: 39 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -105,6 +105,45 @@ These policies were learned from past mistakes. Follow them strictly.
105105
- **Affects humans directly?** → Must be Level 3 (humans should never wait on agent mistakes)
106106
- CLAUDE.md instructions are Level 1 — necessary but not sufficient. When they fail, escalate to hooks (Level 2) or architectural enforcement (Level 3).
107107

108+
## Verification Discipline (Kaizen #11, #15, #17)
109+
110+
### Path Tracing — MANDATORY before any fix
111+
112+
Before writing ANY fix, map the full execution path from trigger to user-visible outcome:
113+
114+
```
115+
1. MAP the chain: input → layer 1 → layer 2 → ... → user-visible outcome
116+
2. For each link: how to verify it works, what artifact/log/query proves it
117+
3. After the fix: verify EVERY link, not just the one you changed
118+
4. Self-review must trace the path — "I changed layer N, what happens at N+1...?"
119+
```
120+
121+
**Never fix a single layer and declare done.** The fix isn't complete until the final outcome is verified end-to-end.
122+
123+
### Invariant Statement — MANDATORY before writing tests
124+
125+
Before writing ANY test, state explicitly:
126+
127+
```
128+
INVARIANT: [what must be true]
129+
SUT: [exact system/function/artifact under test]
130+
VERIFICATION: [how the test proves the invariant holds]
131+
```
132+
133+
**Anti-patterns to avoid:**
134+
- Testing mocks instead of real code (you're proving your mocks work, not your code)
135+
- Testing the wrong artifact (e.g., `/app/dist/` when runtime uses `/tmp/dist/`)
136+
- "All 275 tests pass" when none cover the actual change
137+
- Verifying implementation details (`cpSync was called`) instead of outcomes (`agent has the tool`)
138+
139+
### Runtime Artifact Verification
140+
141+
Always test the **actual deployed artifact**, not just source presence:
142+
- If code is compiled, test the compiled output
143+
- If code runs in a container, verify inside the container
144+
- If a mount provides a file, verify the mount exists AND the consumer reads it
145+
- "The file exists in the repo" is not verification — "the agent receives it at runtime" is
146+
108147
## Kaizen Backlog
109148

110149
Future work, process improvements, and cross-repo engineering proposals are tracked as GitHub Issues in [`Garsson-io/kaizen`](https://github.com/Garsson-io/kaizen). When a dev agent identifies an improvement that's out of scope for the current PR, file it there with the `kaizen` label. Include: what, why, when, how, reproduction steps, and verification criteria.

groups/global/CLAUDE.md

Lines changed: 18 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -111,6 +111,24 @@ When marking a case done, reflect on:
111111
- What would make this type of work faster next time?
112112
- Suggest dev cases for any improvements needed
113113

114+
## Verification Discipline
115+
116+
### Path Tracing (dev cases)
117+
118+
Before fixing anything, map the full execution path from trigger to outcome:
119+
1. Write down every link in the chain (e.g., policy file → mount → agent-runner reads it → SDK receives it → agent acts)
120+
2. After the fix, verify EVERY link — not just the one you changed
121+
3. The fix isn't done until the final user-visible outcome works
122+
123+
### Test Invariants (dev cases)
124+
125+
Before writing ANY test, state:
126+
- *INVARIANT:* what must be true
127+
- *SUT:* the exact system/function/artifact under test
128+
- *VERIFICATION:* how the test proves the invariant
129+
130+
Never test mocks instead of real code. Never test the wrong artifact. Never declare "tests pass" when none cover your change.
131+
114132
## Your Workspace
115133

116134
Files you create are saved in `/workspace/group/`. Use this for notes, research, or anything that should persist across cases.

0 commit comments

Comments
 (0)