Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
197 changes: 142 additions & 55 deletions .claude/skills/review-pr/SKILL.md

Large diffs are not rendered by default.

384 changes: 384 additions & 0 deletions .github/scripts/review_gate.py

Large diffs are not rendered by default.

82 changes: 75 additions & 7 deletions .github/workflows/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -4,7 +4,7 @@
|---|---|---|---|
| `custom-memory-engine.yaml` | Every PR (a `changes` job skips the gates when neither `custom-memory-engine/` nor `.github/` changed, so required checks still report; `.github/` counts because the engine suite tests the CI configuration); pushes to `main` touching those paths; manual | `changes`, then `static`, `verify`, `default-install` (parallel), plus advisory `dual-profile` | Engine quality gates (below) |
| `ci.yaml` | Every PR; pushes to `main`; manual | `checks`, `workflows` | `checks`: root application lint, format, type check and tests (`tests/`, no services). The engine is excluded from the root Ruff config and verified only by its own workflow. `workflows`: actionlint over every workflow, pinned by digest |
| `claude-code-review.yml` | Same-repository, non-draft, non-Dependabot PRs, on open, push, ready and reopen | `claude-review` | Advisory automated review, through the repository's own `review-pr` skill (below), not the stock `code-review` plugin. A second pair of eyes, not the review gate: the gate is the AI review on the Project Owner's behalf (`V2-D57`, `PLAN/CONTRIBUTING.md` §1.1), and no human peer review is a prerequisite |
| `claude-code-review.yml` | Same-repository, non-draft, non-Dependabot PRs, on open, push, ready and reopen | `claude-review` | Advisory automated review, through the repository's own `review-pr` skill (below), not the stock `code-review` plugin. A gate step decides the tier from the PR's history and skips a push that changed no tree before the action is paid for. A second pair of eyes, not the review gate: the gate is the AI review on the Project Owner's behalf (`V2-D57`, `PLAN/CONTRIBUTING.md` §1.1), and no human peer review is a prerequisite |
| `claude.yml` | `@chronamind` in an issue or PR comment, or `@claude review` on a PR, by an owner, member or collaborator | `claude` (assistant), `review` (reviewer) | The interactive assistant, and an on-demand review through the same `review-pr` skill |
| `app-image.yaml` | PRs and `main` pushes touching anything that goes into the root image (`Dockerfile`, lock, `app/`, ...); manual | `image` | Builds the root image, imports the app inside it, checks it runs as `appuser` with read-only code, reports its size. Not required, so its path filter is safe |
| `deploy.yaml` | Manual (`workflow_dispatch`) only | `Build and push to Docker Hub` | Root application image. Automatic publication on `main` is an owner decision and is not enabled |
Expand Down Expand Up @@ -53,17 +53,60 @@ The replacement differs in four ways that matter here:
| A second push | skipped if Claude has already commented | reviewed, strictly: new regressions and unfixed earlier findings |
| Findings | high-signal only | evidence bar of six parts, severity-ranked, nit-capped, deduplicated |

Four things it decides before the model is paid for, or verifies after GitHub answers — the four the canary got wrong:

| | Mechanism |
|---|---|
| **Identity** | `REPOSITORY` is passed in and is the only source of a URL (§1 of the skill). The canary cited a different owner, recalled rather than read. |
| **Tier** | `REVIEW_TIER` is read from the pull request's review history by the gate step, not inferred from the diff. A base branch that advances moves the diff range; the canary's second review then called itself a first review and left its own thread open. |
| **Resolution** | the skill fetches threads by GraphQL, matches them by comment content, and resolves one only after verifying the fix in the code — and only when `isResolved: true` comes back. The canary verified the fix and left the thread open. |
| **Cost** | the gate compares the pushed tree with the one the last review read and skips the run when they are identical. The canary's empty commit bought a 15-turn run that posted nothing. |

Each is a line of `.github/scripts/review_gate.py` or of the skill, and each has a test in
`custom-memory-engine/tests/test_review_gate.py` or `tests/test_claude_integration.py`.

What it can and cannot do: it reads the diff and the surrounding code, and it does not build or run the repository. It
never approves, never merges and never pushes — those tools are denied in `claude_args` — so it cannot satisfy a branch
protection requirement or substitute for a human decision. It complements the deterministic gates; it does not replace
them, and a clean review is not proof of correctness.

**The one write it performs** is resolving its own review threads, and only after verifying the fix and getting
`isResolved: true` back. No `permissions:` change is involved: the action mints an App token by OIDC exchange, and that
installation — not the job's `GITHUB_TOKEN`, which stays read-only — is the identity that already posts the comments.

What that identity needs is more specific than it looks: `resolveReviewThread` is gated on **Contents: write**, not on
`pull_requests: write` (a token holding only the latter is refused with `Resource not accessible by integration`; see
`github/gh-aw#35726`, `oven-sh/bun#36959` and the note in the `zuke-build/zuke#634` thread). The action's default token
request already asks for `contents: write`, `pull_requests: write` and `issues: write`
(`src/github/token.ts` at the pinned SHA), so this repository's installation should already qualify. The skill does not
rely on that: it reads `viewerCanResolve` first and, when it is false, reports `thread open: not permitted (no Contents:
write)` and leaves the thread alone. **If resolution fails in practice**, the manual action is on the App installation,
not in this repository: the Claude GitHub App (app settings → Permissions & events) must grant
*Contents: Read and write* for the installing account, and the installation must include this repository.

**A pull request that edits `claude-code-review.yml` (or `claude.yml`) gets no automatic review, and its
`claude-review` check is green.** That is not a policy here but a platform rule: the action's OIDC exchange is refused
unless the running workflow file is byte-identical to the default branch's, so the action exits "skipped" and the job
passes. Nothing in this repository can lift it, and buying around it by weakening the check is not an option. So the
gate prints a warning and a step-summary notice naming the workflow, and the review is requested explicitly with a
`@claude review` comment: that job runs on `issue_comment`, whose workflow *is* the default branch's, and reviews any
pull request including the one that changes the workflow.

**Cost.** Each run is billed to the subscription behind `CLAUDE_CODE_OAUTH_TOKEN` and consumes GitHub Actions
minutes. The two levers are repository variables, not edits to the workflow: `CLAUDE_REVIEW_MODEL` (default
`claude-sonnet-5-5`; `claude-opus-5-5` buys depth at a materially higher cost per review) and, indirectly, how often it runs — it
reviews every push, because a PR here is rebased several times and the merged tree is what ships. The run log's last
line reports `num_turns` and `total_cost_usd` for each review, so the per-PR cost is measured, not guessed. A concurrency
group keyed on the PR cancels a superseded review, and every job has a timeout, so a stuck run cannot bill for long.
minutes. Three things bound it, none of them a model downgrade:

- **A push that changes no tree is not reviewed.** The gate step compares the pushed commit with the head the last review
recorded in its summary, and skips before the action mints a token. It does not filter by path: a change to `PLAN/V2/`,
`.github/` or an instruction file is reviewable here, so any tree change is reviewed and an allow list would be the way
a governance change gets skipped as a docs change. A skipped run says so in the step summary, with the sha it compared.
- **A superseded review is cancelled**, by a concurrency group keyed on the PR, and every job has a timeout. A run
started by a bot lands in a group of its own: the action posts an acknowledgement comment as it starts, and that
comment used to re-enter the same concurrency group and cancel the run that had just posted it.
- **The model is a repository variable**, `CLAUDE_REVIEW_MODEL` (default `claude-sonnet-5-5`; `claude-opus-5-5` buys
depth at a materially higher cost per review).

The run log's last line reports `num_turns` and `total_cost_usd` for every review, so the per-PR cost is measured rather
than guessed — and those are API-equivalent figures for a subscription, not a quota or a bill.

**Duplicate reviews.** The managed **Claude Code Review** service (Team/Enterprise, enabled by a Claude organization
Owner) is a different, deeper product with a check run, inline findings and its own spend limit. If it is enabled for
Expand All @@ -72,7 +115,32 @@ token. Nothing in this repository can read or change that setting, so it is an a

**Verifying the reviewer.** A green run proves nothing about detection. After any change to `REVIEW.md`, the skill or
the workflow, open a throwaway PR with a planted defect (delete a guard from a migration trigger, bypass `assemble` on a
read path) and confirm the review reports it inline and cites the mechanism. The instruction files' own references are
read path) and confirm the review reports it inline and cites the mechanism. The gate can be replayed read-only against
any past PR, which is how its tier and skip decisions were checked without a live run:

```bash
REPOSITORY=eMinders/ChronaMind PR_NUMBER=<n> HEAD_SHA=<sha> BASE_SHA=<sha> ACTION=synchronize \
python3 .github/scripts/review_gate.py
```

**A controlled canary, and the rules that keep it safe.** Detection is the one property a green run cannot show, so it is
tested by planting a defect. This is the procedure the reviewer was last validated with, and it is the only thing that
proves a change to `REVIEW.md`, the skill or this workflow has not cost a finding:

1. Branch from `main` and plant **one** semantic defect tied to a real invariant, of the kind CI cannot see — no syntax
error, no failing test, no lint. Verify that by mutation first: with the defect in place the full suite and every
static gate must still pass, or the canary tests CI rather than the reviewer.
2. Write the expected finding down **before** pushing — location, trigger, wrong behaviour, consequence, the safeguard
that misses it, the fix, and the severity you expect — and keep it outside the repository. A record written
afterwards only describes what happened.
3. Push the defect only, and open a normal pull request marked do-not-merge. It must not touch `.github/workflows/`,
`REVIEW.md` or `.claude/`: a change to those is exactly what the platform will not review (above), so such a canary
would test nothing.
4. Compare against the record: was it detected, at the right location, with the right mechanism and a calibrated
severity? Then push the fix and check the *re-review* — `Tier: re-review`, the finding reported fixed, its thread
resolved, and nothing repeated.
5. Close it unmerged and delete the branch. A canary is evidence about the reviewer and is never a change: it must not
reach `main`, and it must not leave a defect behind. The instruction files' own references are
checked on every run: `tests/test_claude_integration.py` holds them to `scripts/check_evidence.py`, and pins the
workflow's triggers, permissions, deny list and base-branch checkout.

Expand Down
50 changes: 48 additions & 2 deletions .github/workflows/claude-code-review.yml
Original file line number Diff line number Diff line change
Expand Up @@ -65,7 +65,44 @@ jobs:
fetch-depth: 0
persist-credentials: false

- name: Decide whether this push needs a review, and at which tier
# Two decisions, taken before the model is paid for (`.github/scripts/review_gate.py`, unit-tested in
# `custom-memory-engine/tests/test_review_gate.py`):
# tier read from the review history on the PR, not inferred from the shape of the diff. A base branch that
# advances moves the diff range, and a reviewer that infers its tier from the diff then calls itself a
# first review, re-reports nothing and leaves its own threads open (the canary did exactly this).
# cost a push that changes no tree (an empty commit, a rebase onto the same tree) is skipped before the
# action mints a token. `synchronize` fires on the push, so the reviewed head is the only baseline that
# answers this, and it is read from the previous summary comment.
# The gate is a decision, not a filter: any tree change is reviewable here, including PLAN/, .github/ and the
# instruction files. An allow list of "code" paths is how a governance change gets skipped as a docs change.
id: gate
env:
GH_TOKEN: ${{ secrets.GITHUB_TOKEN }}
REPOSITORY: ${{ github.repository }}
PR_NUMBER: ${{ github.event.pull_request.number }}
HEAD_SHA: ${{ github.event.pull_request.head.sha }}
BASE_SHA: ${{ github.event.pull_request.base.sha }}
ACTION: ${{ github.event.action }}
run: |
# The gate decides whether a pull request gets reviewed, so it is policy code and comes from the base branch
# — a pull request must not be able to edit its own gate into skipping. The one exception is the pull request
# that introduces it, where the base has no copy; that copy is then part of the diff, like the skill's own
# bootstrap above. Neither copy present is not a reason to skip: it is a reason to review.
if ! git cat-file -e "HEAD:.github/scripts/review_gate.py" 2>/dev/null; then
{ git -C pr-head archive HEAD .github/scripts | tar -x -C .; } || true
echo "gate: introduced by this pull request (:warning: its change is part of the diff)"
fi
if [ -f .github/scripts/review_gate.py ]; then
python3 .github/scripts/review_gate.py
else
echo "::warning::no review gate in the base branch or the pull request; reviewing without a tier"
printf 'review=true\ntier=unknown\nreason=gate_failed\nprevious_head_sha=\nprevious_base_sha=\n' \
>> "$GITHUB_OUTPUT"
fi

- name: Resolve the review standard from the base branch, not from the pull request
if: steps.gate.outputs.review == 'true'
# Three copies coexist, and each has one job:
# workspace root the base branch's instructions, trusted. The action itself restores `CLAUDE.md`, `AGENTS.md`
# and `.claude/` here, and a `/skill-name` prompt resolves its SKILL.md from here too.
Expand Down Expand Up @@ -100,6 +137,7 @@ jobs:

- name: Run the review
id: claude-review
if: steps.gate.outputs.review == 'true'
uses: anthropics/claude-code-action@4d75e5f390ee76b2e09c1980ecc99ff3a2e088c8 # v1.0.275
env:
# `--model` reads the environment, so the depth/cost trade-off is a repository variable rather than an edit to
Expand All @@ -111,11 +149,19 @@ jobs:
prompt: >-
/review-pr
PR_NUMBER=${{ github.event.pull_request.number }}
REPOSITORY=${{ github.repository }}
BASE_REF=${{ github.event.pull_request.base.ref }}
BASE_SHA=${{ github.event.pull_request.base.sha }}
HEAD_SHA=${{ github.event.pull_request.head.sha }}
REVIEW_TIER=${{ steps.gate.outputs.tier }}
REVIEW_REASON=${{ steps.gate.outputs.reason }}
PREVIOUS_BASE_SHA=${{ steps.gate.outputs.previous_base_sha }}
PREVIOUS_HEAD_SHA=${{ steps.gate.outputs.previous_head_sha }}
The base branch is checked out at the workspace root and the pull request at ./pr-head.
Follow the skill exactly; it says which files to read from which checkout, and which instructions to read
from the base branch rather than from the pull request.
REPOSITORY is the canonical owner/name: every repository URL you write comes from it, and no other
owner or name may appear in a link.
Follow the skill exactly; it says which files to read from which checkout, which instructions to read from
the base branch rather than from the pull request, and what REVIEW_TIER requires of you.
# `default`, not `auto`: the review job runs with a strict allow list, and every tool it needs is named below.
# The deny list is belt and braces for the one thing a review must never do: change the pull request, approve
# it, or merge it (V2-D57: no GitHub approval is ever fabricated to satisfy branch protection).
Expand Down
Loading
Loading