Repository navigation
docs: add tier-specific agent guides - #13439
glamr-agent wants to merge 18 commits into
Conversation
Agents read every AGENTS.md and CLAUDE.md on the path they descend, so context that lives only in the root file is paid for on every task and still misses the local detail. Add a guide at each major tier and restore the four AGENTS.md symlinks that had gone missing. New tier guides: - lib/AGENTS.md: crate map grouped by role, workspace build and test commands, and the note that the two PyO3 crates declare their own workspaces and so are not built by a root cargo build. - lib/runtime/AGENTS.md: the DistributedRuntime object model, the three communication planes with the environment variable that selects each, and the source layout. - components/src/dynamo/AGENTS.md: the package map, the two source roots the dynamo namespace is assembled from, and the maturin rebuild step Python callers depend on. - deploy/AGENTS.md: the directory map, the custom resources and their generated artifacts, and a hand-off to the nested operator guides. Restored symlinks (AGENTS.md -> CLAUDE.md), matching the convention the rest of the tree already uses: - lib/llm, components/src/dynamo/frontend, lib/mocker/src/replay/online, aisimulate/crates/core/src/replay The root AGENTS.md gains a Component guides table and a Concepts section pointing at the existing knowledge base, and two link labels that read docs/AGENTS.md are corrected to docs/fern/AGENTS.md. Signed-off-by: svc-glamr@nvidia.com <svc-glamr@nvidia.com>
The three new tier guides had no explicit area glob, so build_codeowners.py --strict fell them through to the catch-all and exited 1 at 5578/5581. The catch-all also routed deploy/AGENTS.md to the runtime team, which is the wrong reviewer for that tree. Route each tier file to the team that owns the tree it describes, following the precedent of tests/CLAUDE.md, which the runtime area already claims: - lib/AGENTS.md and components/src/dynamo/AGENTS.md to runtime, which already owns lib/runtime/, lib/bindings/, lib/llm/, and the shared components/src/dynamo/common/ package. - deploy/AGENTS.md to operator, which already owns deploy/CONTRIBUTING.md, deploy/__init__.py, deploy/helm/, and deploy/operator/. CODEOWNERS is regenerated with emit_codeowners.py, not hand-edited. CONTRIBUTORS.md is unchanged. Coverage returns to 5581/5581 (100.00%). Signed-off-by: svc-glamr@nvidia.com <svc-glamr@nvidia.com>
|
👋 Hi glamr-agent! Thank you for contributing to ai-dynamo/dynamo. Just a reminder: The 🚀 |
Automated evidence record — validation completeValidation status: complete Evidence summary: [2/2 validated] AI review assessment: sound — advisory only. An AI agent's judgment of whether the change is logically sound on the recorded evidence; not a merge authorization. CI and a human reviewer hold that. The full review is attached as a separate Validation result: complete — pass. Evidence audit: complete [2/2 validated] — the evidence table is grounded in recorded runs. Evidence [2/2 validated]Generated from validation/registry.jsonl — do not edit by hand.
|
plan.md# Plan — component/crate-level agent markdown for Dynamo
Route: implementation
Template: debug-investigation
Engine: vllm
## User intent
A Slack thread asked for "a good agent.md context which explains key Dynamo
concepts (e.g. request/event planes, router, disagg) and summary of codebase
structure — for warming up a new agent with Dynamo context." The reply pointed
at the root `AGENTS.md` and immediately conceded "we probably should fix it in
so far as it doesn't do a good job." A second participant supplied the concrete
idea the caller told us to implement: "Instead of complex root one I wonder if
we should really get component/crate level mds… As soon as agent descends down
to a dir it'll read it in."
So the ask is not "rewrite the root file." It is to move orientation *down* the
tree, to where an agent lands, and let the root stop trying to be the whole
map. Agent harnesses read every `AGENTS.md` on the path they descend, so a file
at `lib/` and a file at `lib/runtime/` compose: the tier file frames the
neighbourhood, the crate file names the local rules. That composition is the
whole point of the idea, and it is what this change builds.
There is no runtime behaviour change here, no performance claim, and no
hypothesis about a defect in shipped code. The `debug-investigation` overlay is
retained because it is the only template that authorizes a *documentation*
correction while still insisting on a real, reviewable diff, and because it
cites `learnings/discovery-survey-before-planning.md` first — which is exactly
what saved this work item from writing the wrong change (see Discovery). Its
"reproduce the symptom" framing maps to establishing, concretely and from the
tree, what the root guide actually fails to do for a cold agent.
## Non-goals
- Not rewriting the root `AGENTS.md`. It is a competent *contributor* guide —
build, test, lint, PR conventions, DCO, CODEOWNERS. Those sections stay. The
root gains two small things and loses nothing.
- Not authoring new architecture prose. Dynamo already has canonical concept
pages under `docs/fern/pages/developer-guide/knowledge-base/concepts/`. The
new files summarize in a couple of lines and link; they do not re-explain,
because re-explained architecture rots and then lies to agents.
- Not covering every crate and package. The workspace has 39 members and
`components/src/dynamo/` has 17 packages. A 30-file dump is not reviewable.
The deferred set is named explicitly below.
- Not converting the existing `CLAUDE.md`-canonical files, and not relocating
content other teams authored (the `lib/llm` N-2 compatibility policy from
#12800/#12894, the operator semantics from #13244). Churn on those files buys
nothing and costs reviewer goodwill.
- Not touching `.github/codeowners/areas.yaml`. Discovery proved no edit is
needed (below).
- No GPU work, no engine smoke test, no benchmark. Nothing in this change can
be defended by running a model.
## Discovery
Everything in this section was read from the checkout at commit `bce85e29a` on
`main`, or from the GitHub API. Paths are repo-relative.
### The premise was wrong, and that changed the plan
The work item is phrased as if Dynamo has no component-level agent files. It
has 64 of them. `git ls-files | grep -iE 'agents?\.md|claude\.md'` returns
`AGENTS.md`/`CLAUDE.md` pairs across `lib/backend-common/`, `lib/bench/`,
`lib/data-gen/`, `lib/kv-router/` (plus five nested `src/` subdirectories),
`lib/kvbm-engine/`, `lib/kvbm-kernels/`, `lib/kvbm-logical/`, `lib/llm/src/`
(four subdirectories), `components/src/dynamo/{common/backend,router,sglang}/`,
`deploy/operator/` (six files), `docs/fern/`, `recipes/`, `tests/`,
`examples/router/custom-policy-example/`, and `aisimulate/`.
So the correct change is not "invent a pattern." It is "extend an established
pattern into the gaps that matter, and make the root point at it." Writing this
plan from the work item's wording alone would have produced a duplicate
convention layered on top of a working one.
### What the existing files actually are
Reading them showed they are not orientation documents. `lib/llm/CLAUDE.md` is
a 56-line N-2 worker/frontend wire-compatibility policy.
`components/src/dynamo/frontend/CLAUDE.md` is a 16-line rule that
`FrontendConfig` must not be written back to `os.environ`.
`deploy/operator/internal/controller/AGENTS.md` is a list of reconciliation
invariants. These are *guardrails* — "do not do X here" — authored as a
by-product of feature PRs. Not one of them tells a cold agent what the
component is or how it fits.
That is the real shape of the gap, and it is why the new files are additive
rather than replacements: leaf files constrain, and nothing exists yet to
orient. The new tier files orient. The two roles compose on the descent path
instead of competing.
### Three conventions are in play, and the newest one wins
`git ls-files -s` on the mode bits separates them cleanly:
- **Symlink twins** (24 files, mode `120000`): `CLAUDE.md` is the real file and
`AGENTS.md` is a symlink to it. This is the dominant `lib/` and
`components/` pattern, established wholesale by merged PR #10984
("docs(agents): add missing AGENTS symlinks", merged 2026-06-29), which added
18 such symlinks in one go.
- **`AGENTS.md`-canonical, no twin**: `deploy/operator/` (six files),
`docs/fern/`, `recipes/`. This is what recent merges do — #13244
("docs(operator): codify implementation semantics", merged 2026-08-14) added
`AGENTS.md` only.
- **`AGENTS.md`-canonical with an import pointer**: the repository root, whose
`CLAUDE.md` is the single line `@AGENTS.md`; and
`deploy/operator/internal/controller/CLAUDE.md`, which is `@"AGENTS.md"`.
New files in this change follow the root's convention: `AGENTS.md` is the real
file and no `CLAUDE.md` twin is created. That matches the most recent merged
precedent, makes the cross-agent open filename canonical, and adds the fewest
files. The older symlink direction is left exactly as it is — flipping 24 files
would be pure churn.
### PR #10984's invariant has silently regressed
#10984 existed to guarantee that every `CLAUDE.md` has an `AGENTS.md` sibling,
so harnesses that read only the open filename are not blind. Its own Validation
section records the check it used — a `find`-based sweep for `CLAUDE.md`
directories lacking `AGENTS.md`. Running the same sweep today returns four
directories where the invariant has since broken:
- `lib/llm/` — the largest crate in the tree
- `components/src/dynamo/frontend/`
- `lib/mocker/src/replay/online/`
- `aisimulate/crates/core/src/replay/`
An agent that reads `AGENTS.md` and descends into `lib/llm/` currently gets
nothing, while an agent that reads `CLAUDE.md` gets a compatibility policy. That
is a concrete, verifiable defect in exactly the mechanism this work item is
about, and closing it costs four symlinks.
### What the root guide does badly — named, not asserted
Reading `AGENTS.md` (159 lines) against the Slack ask:
1. **It never explains a single concept the thread asked for.** "Request plane"
and "event plane" do not appear. The Overview names disaggregation and
KV-aware routing in one sentence each and moves on. Roughly 100 of its 159
lines are process: the skills index, ecosystem table, build/test/lint
commands, PR and DCO conventions.
2. **The concepts are already written down, and the root does not link them.**
`docs/fern/pages/developer-guide/knowledge-base/concepts/communication-planes/`
contains `request-plane.md`, `event-plane.md`, and `discovery-plane.md`;
`.../concepts/system-architecture/` contains `architecture-flow.md`,
`distributed-runtime.md`, and `disaggregated-serving.md`. The root guide
references none of them. The person in the thread was told to read a file
that pointed away from the material they wanted.
3. **Two links are labelled wrong.** Lines 83 and 84 render as
`` `docs/AGENTS.md` `` while targeting `docs/fern/AGENTS.md`. `ls
docs/AGENTS.md` confirms no such file exists. An agent that trusts the
label and tries to open the path fails. (Open PR #12405, "docs: remove stale
agent guide paths", fixes two *different* stale references — in
`.agents/skills/dynamo-docs/SKILL.md` and `lib/kvbm-kernels/CLAUDE.md` — and
does not touch these two labels. No overlap.)
4. **The Repository Map is one row per top-level directory and stops there.**
`lib/` is a single table cell reading "Rust workspace crates: `runtime`,
`llm`, `kv-router`, `kvbm-*`, `mocker`, and more (see the root `Cargo.toml`)."
An agent landing in `lib/kv-hashing/` is told to go read a manifest.
### Component inventory, from source rather than memory
Root `Cargo.toml` `[workspace] members` lists 39 entries. Under `lib/` the
crate directories are: `backend-common`, `bench`, `bindings`, `data-gen`,
`gpu_memory_service`, `kv-hashing`, `kv-router`, `kvbm-common`, `kvbm-config`,
`kvbm-consolidator`, `kvbm-engine`, `kvbm-kernels`, `kvbm-logical`,
`kvbm-physical`, `llm`, `memory`, `mocker`, `rl`, `router-plugins`, `runtime`,
`sidecar`, `tokens`, `truthy`. `components/src/dynamo/` holds `common`,
`frontend`, `global_planner`, `global_router`, `kv_dc_relay`, `kv_state_agent`,
`mocker`, `planner`, `profiler`, `replay`, `router`, `sglang`, `squeeze_evolve`,
`thunderagent_router`, `tokenspeed`, `trtllm`, `vllm`. `deploy/` holds `helm`,
`inference-gateway`, `observability`, `operator`, `power-agent`,
`pre-deployment`, `snapshot`, `utils`.
`lib/runtime/` is the standout gap: it is the crate the concept question is
about, it has no `README.md` and no agent file, and
`lib/runtime/src/transports/` contains `etcd.rs`, `event_plane/`, `nats.rs`,
`tcp.rs`, and `zmq.rs` — the plane vocabulary is literally the directory
layout. `lib/llm/src/` similarly holds `entrypoint/`, `http/`, `grpc/`,
`preprocessor.rs`, `protocols/`, `kv_router/`, `local_model/`, `migration.rs`,
and `block_manager/`.
### Conventions any new markdown must satisfy
- `.pre-commit-config.yaml` applies `codespell`, `end-of-file-fixer`,
`trailing-whitespace`, `mixed-line-ending`, and `check-case-conflict` to all
files. Markdown is not excluded by the top-level `exclude:` regex.
- The `validate-skills` hook has `files: ^(\.agents/skills/|AGENTS\.md$)`.
pre-commit matches with `re.search`, so **every nested `AGENTS.md` triggers
this hook** — adding files under `lib/` will run it. Reading
`scripts/validate_skills.py` shows it is invoked with `pass_filenames: false`
and only ever reads the repository-root `AGENTS.md`, checking that its
`## Skills` section matches `.agents/skills/` exactly. Consequence: the new
nested files cannot break it, but the root edit must not disturb the Skills
index. Any `## Skills` heading inside a nested file is ignored; even so, the
new files will not use that heading.
- `docs/fern/AGENTS.md` scopes itself in its own first line to "files under
`docs/`, `examples/`, or `recipes/`". New files under `lib/`,
`components/`, and `deploy/` are therefore **not** bound by Fern frontmatter
or the documentation style guide's page rules. The house conventions that do
carry over and will be honoured: the `<!-- SPDX -->` comment header (as on the
root `AGENTS.md`, `recipes/AGENTS.md`, and the `deploy/operator/` set),
GitHub-style `> [!NOTE]` admonitions in `.md`, language-tagged code fences,
and backend casing vLLM / SGLang / TensorRT-LLM.
- CODEOWNERS: `.github/workflows/codeowners.yml` runs
`pytest .github/codeowners/test_codeowners.py` plus a diff-aware `--strict`
coverage gate and a drift check. Reading `CODEOWNERS` shows the agent-file
rules are **root-anchored** — `/AGENTS.md` and `/CLAUDE.md` at lines 311-312
— so nested files do not get pulled to the process team. Running
`.github/codeowners/who_owns.py --codeowners CODEOWNERS --repo .` over the
proposed paths resolves every one to a real team:
`lib/AGENTS.md`, `components/src/dynamo/AGENTS.md`, `deploy/AGENTS.md`,
`lib/runtime/AGENTS.md`, `lib/llm/AGENTS.md` → `dynamo-runtime-codeowners`;
`components/src/dynamo/frontend/AGENTS.md` → `dynamo-frontend-codeowners`.
`areas.yaml` also carries `meta.catch_all`. No `areas.yaml` edit is required
and the coverage gate will not red-X.
### Naming
Standardizing on `AGENTS.md`. The thread says "agent.md" colloquially, but
every one of the 64 in-tree files is `AGENTS.md` or `CLAUDE.md`,
`areas.yaml` routes those two names, `scripts/validate_skills.py` keys off
`AGENTS.md`, and `AGENTS.md` is the cross-harness open convention. Introducing
`agent.md` would create a third filename that no tooling reads.
### Prior-art survey
All-state searches via `gh pr list --repo ai-dynamo/dynamo --state all --search`
on `AGENTS in:title`, `CLAUDE.md in:title`, `crate-level in:title`,
`agent guide in:title`, and free-text `AGENTS.md` / `agents.md`. Plus
`git log` over every agent-markdown path.
Nothing implements this. The closest results, each checked against `main`:
- **#10984 MERGED** — added 18 `AGENTS.md` symlinks. Present on `main`; four of
its guarantees have since regressed (above). Complementary, not competing.
- **#13244 MERGED 2026-08-14** — `deploy/operator/**/AGENTS.md`. Present on
`main`. Sets the `AGENTS.md`-canonical precedent this change follows.
- **#11231 MERGED** — "chore: rewrite AGENTS.md" — produced today's root file.
A *root* rewrite, i.e. the approach the thread explicitly moved away from.
- **#11230 MERGED** — added the `validate-skills` hook.
- **#12405 OPEN** — fixes two stale refs in `SKILL.md` and
`lib/kvbm-kernels/CLAUDE.md`; disjoint from the two root labels here.
- **#12528 OPEN** — "docs: realign agent guides with the tab structure and add a
docs lint job" — scoped to `docs/fern/`, no component-level files.
- **#11378 / #13023 / #11313 / #9596** — Cursor Cloud dev-environment setup
notes; open or closed, unrelated content.
`git log` over the agent-markdown paths shows the pattern grows opportunistically
— #13110, #12897, #12800, #12894, #12525 each dropped a guardrail file beside a
feature. Nobody has done a deliberate structural pass. No merged PR resolves this
request, so there is no `Disposition` line and no competing open PR to help land
instead of duplicating.
### Existing tests
`grep`/`git ls-files` across `tests/` found no test asserting anything about
`AGENTS.md` or `CLAUDE.md` placement, content, or symlink parity. The only
executable gates touching these paths are the pre-commit hooks and the
CODEOWNERS workflow, both above. Per `learnings/no-tautological-tests.md` there
is nothing to duplicate, and a markdown-content unit test would assert the
implementation rather than behaviour — so none is planned. The invariant that
*is* worth asserting (every `CLAUDE.md` has an `AGENTS.md` sibling) is checked
as a recorded command, matching how #10984 itself validated.
## Chosen approach
Add tier-level orientation where an agent descends, close the four regressed
symlink gaps, and turn the root from a wall into an index. Nine paths, eight of
them new files.
**Tier and crate orientation — four new `AGENTS.md`:**
1. **`lib/AGENTS.md`** — the Rust workspace tier. A crate map grouped by role
rather than alphabetically: runtime and transport (`runtime`, `memory`,
`tokens`, `truthy`); LLM serving (`llm`, `backend-common`, `sidecar`,
`router-plugins`); routing (`kv-router`, `kv-hashing`); KV block management
(the `kvbm-*` family); simulation and benchmarking (`mocker`, `bench`,
`data-gen`); bindings (`bindings/python`, `bindings/c`). Plus the
per-crate build and test invocation, and the note that
`lib/bindings/python` is deliberately outside the workspace and built with
maturin. This is the file that covers every crate not given its own, which
is what makes the scoped subset defensible.
2. **`lib/runtime/AGENTS.md`** — the concept file the thread asked for, sited on
the crate that owns the concepts. Request plane, event plane, and discovery
plane, each one sentence and a link to its canonical page under
`docs/fern/pages/developer-guide/knowledge-base/concepts/communication-planes/`;
the `DistributedRuntime` → namespace → component → endpoint model, linked to
`.../system-architecture/distributed-runtime.md`; and a pointer to
`lib/runtime/src/transports/` as where the vocabulary is realized in code.
3. **`components/src/dynamo/AGENTS.md`** — the Python tier. Package map, the
PyO3/maturin boundary to the Rust core, the `python -m dynamo.<pkg>`
entrypoint convention, where per-package `README.md` files already live, and
the pointer to `.ai/python-guidelines.md` and `.ai/pytest-guidelines.md`.
4. **`deploy/AGENTS.md`** — the Kubernetes tier. What `operator`, `helm`,
`inference-gateway`, `observability`, `snapshot`, and `pre-deployment` each
are, and a hand-off to the existing `deploy/operator/AGENTS.md` set rather
than restating it.
**Restore the #10984 invariant — four `AGENTS.md` symlinks** at `lib/llm/`,
`components/src/dynamo/frontend/`, `lib/mocker/src/replay/online/`, and
`aisimulate/crates/core/src/replay/`, each pointing at the `CLAUDE.md` already
there. Symlinks, not copies, and not content rewrites: these four directories
are inside the symlink-twin neighbourhood, so a symlink is the locally correct
form and it guarantees the two filenames cannot drift. No existing content
moves. After this, the sweep from #10984's own Validation section returns empty
across the whole repository.
**Root `AGENTS.md` — three surgical edits, nothing removed:**
- Fix the two `` `docs/AGENTS.md` `` labels on lines 83-84 to
`docs/fern/AGENTS.md`.
- Add a short **Concepts** section linking the existing knowledge-base pages —
request/event/discovery planes, architecture flow, distributed runtime,
disaggregated serving — so the guide finally answers the question that was
asked in the thread instead of pointing away from it. Links only, a line of
gloss each; the pages stay canonical.
- Add a **Component guides** pointer naming the tier files, so the root
advertises the descend-and-read mechanism rather than trying to be the map.
The Skills section is left byte-identical so `validate_skills.py` keeps passing.
**Deliberately deferred, and why.** Individual files for `tokens`, `truthy`,
`rl`, `kv-hashing`, `memory`, `mocker`, `kvbm-common`, `kvbm-config`,
`kvbm-consolidator`, `kvbm-physical`, `sidecar`, `bindings`, `router-plugins`,
`gpu_memory_service`; and for `vllm`, `trtllm`, `planner`, `profiler`,
`mocker`, `global_planner`, `global_router`, `kv_dc_relay`, `kv_state_agent`,
`replay`, `squeeze_evolve`, `thunderagent_router`, `tokenspeed`. Each is
covered by its tier file, most already have a `README.md`, and none carries
concept vocabulary a cold agent is missing. Writing 27 more files in this PR
would make it unreviewable and would mean inventing per-crate prose without a
domain owner to check it. The tier files establish the pattern; owning teams
extend it when they have something specific to say — which is precisely how
`lib/kv-router/` and `deploy/operator/` got theirs.
## Rejected alternatives
**Rewrite the root `AGENTS.md` to explain everything.** This is what the thread
explicitly rejected ("Instead of complex root one…"), and it is what #11231
already did once to produce today's file. A single root file cannot be both
short enough to read on every invocation and complete enough to orient in
`kvbm-physical`. It also concentrates every future edit into one file the
process team owns, whereas nested files route to the team that owns the code —
proven above with `who_owns.py`.
**One `AGENTS.md` per workspace member (39 files).** Faithful to a literal
reading of "crate level mds", and the reason it loses is reviewability: a
39-file markdown PR gets skimmed and rubber-stamped or stalls. It would also
require inventing prose for crates like `truthy` and `kvbm-config` with no
owner to check it, producing exactly the confidently-wrong context that makes
agent guides harmful. The tier files give every crate coverage at a fraction of
the surface.
**Name the files `agent.md`, as the thread's wording suggests.** Rejected on
tooling grounds: `scripts/validate_skills.py`, `areas.yaml`, and every one of
the 64 existing files key on `AGENTS.md`/`CLAUDE.md`. A third name would be read
by nothing.
**Create `CLAUDE.md` twins alongside each new `AGENTS.md`.** Doubles the file
count for no gain. Claude Code reads `AGENTS.md`, and the newest merged
precedent (#13244, `deploy/operator/`) ships `AGENTS.md` alone. Adding twins
would also create four more pairs that can drift.
**Convert `lib/llm/CLAUDE.md` and `frontend/CLAUDE.md` into real `AGENTS.md`
files carrying both orientation and the existing rules.** Considered, and it is
the tidier end state. Rejected for this PR because it relocates content other
teams authored (#12800, #12894) into a file they did not write, which invites a
review objection orthogonal to the change's purpose. Since agents read every
`AGENTS.md` on the descent path, `lib/AGENTS.md` is read on the way into
`lib/llm/` anyway — the orientation lands without moving anyone's text. The
symlink gets the parity fix at zero content risk.
**Add a pre-commit hook enforcing the `CLAUDE.md` ⇒ `AGENTS.md` invariant.**
Genuinely tempting, since the invariant regressed once already and would
otherwise regress again. Rejected as scope: it is a `.pre-commit-config.yaml`
plus a new script plus a CODEOWNERS-routing change to the process team, and it
would bury a docs PR in tooling review. Worth its own follow-up; called out in
the PR description as a known follow-up rather than smuggled in here.
**`02-rust-cargo-check`.** No Rust source, manifest, or feature changes —
`lib/AGENTS.md` and `lib/runtime/AGENTS.md` are inert to the build. The
"optional vs required parameter" reasoning in the planner brief does not apply:
this change adds no parameter and no caller anywhere needs updating, so there is
no caller-discovery claim for `cargo check --workspace` to defend.
## Validation strategy
Markdown-only on a single-A100 sandbox with no Docker daemon (per
`compute-env.md`). There is no runtime behaviour, no engine invocation, and no
performance number, so recipes 06 through 10 and 13 through 17 are all
inapplicable — not blocked, simply not warranted. `00-dynamo-editable-install`
is not triggered either: nothing selected here imports `dynamo.*`.
**`01-python-lint`** — the recipe's own "When to use" names `*.md` with no
behavioural change, so this is the primary gate and it genuinely *runs*.
Section 1 is `N/A — no Python files touched (files: *.md, symlinks)`. Section 2
runs `pre-commit run --files <each changed path> --hook-stage manual` from the
checkout; `pre-commit` is present at `/opt/dynamo/venv/bin/pre-commit`. This is
not a formality: because the hook's `files` regex matches with `re.search`,
every new nested `AGENTS.md` fires `validate-skills`, which re-validates the
root Skills index against `.agents/skills/`. It also runs `codespell`,
`end-of-file-fixer`, `trailing-whitespace`, `mixed-line-ending`, and
`check-case-conflict` over the new prose. Evidence: every hook reports
`Passed`, exit 0. This defends the claim that the new files conform to the
repository's enforced markdown conventions and that the root edit did not break
the skills index.
**`05-code-inspection`** — nominated for the two claims no linter can decide,
not as a substitute for one. Section 1 is `N/A` pre-publication (no PR number
yet); if the run happens after publication, record the diff scan.
Section 2 is the load-bearing part and is where the recipe earns its place. A
documentation file that describes architecture is a set of factual assertions
about source, and the only way to prove them is to read the source. Every
architectural claim in the four new files gets a `file:line` citation in the
recorded packet — for instance "Claim: the event plane is a `lib/runtime`
transport concern. Evidence: `lib/runtime/src/transports/event_plane/` and
`lib/runtime/src/transports/nats.rs` confirm"; "Claim: `lib/bindings/python` is
excluded from the workspace. Evidence: root `Cargo.toml` `[workspace] members`
lists `lib/bindings/python/codegen` only." Any sentence in a new file without a
citation is a blocking finding and must be deleted or corrected. Every
`docs/fern/...` link target is confirmed to exist with `git ls-files`. This is
the check that stops the PR from shipping confidently-wrong agent context, which
is the single worst failure mode available to this change.
Section 3 is the parallel-pattern slot, and here it runs three real commands
rather than grepping:
- The `CLAUDE.md` ⇒ `AGENTS.md` parity sweep — the same `find`-based check
recorded in merged PR #10984's Validation section. Expected: empty output
across the whole repository, where `main` today returns four directories.
This is the recorded proof that the regressed invariant is closed.
- `python3 .github/codeowners/build_codeowners.py --areas
.github/codeowners/areas.yaml --repo . --strict` (with `--changed-only`
against the base as the workflow does). Expected: coverage gate passes with
no unowned path. The planner's routing claim was established with
`who_owns.py`; this runs the gate that CI actually enforces.
- `python3 -m pytest .github/codeowners/test_codeowners.py --override-ini="addopts="`
— the matcher/drift unit tests the codeowners workflow gates on. Expected:
all pass, confirming `CODEOWNERS` has not drifted from `areas.yaml`.
Section 4 (CI and review status) is recorded after publication if reachable,
and marked `blocked / pre-publication` otherwise.
Between them: Recipe 01 proves the files are well-formed and the enforced hooks
are green; Recipe 05 proves the files are *true*, the CODEOWNERS gate is
satisfied, and the parity invariant is restored. Nothing in the ladder is
padding, and nothing in it reads where it could run.
```validation-recipes
01-python-lint
05-code-inspection
```
## Required deliverables
The standard agency packet set: `plan.md` (this file), `change.md` and
`change.diff` from the printer, `change-validation.md` from the validator, and
`review.md` from the reviewer. The publisher pushes
`feat/crate-level-agent-mds--c80ee3b2d603` against base `main` and opens the
pull request on `ai-dynamo/dynamo`.
Diff contents, nine paths:
| Path | Action |
|------|--------|
| `lib/AGENTS.md` | new — Rust workspace tier, crate map by role |
| `lib/runtime/AGENTS.md` | new — request/event/discovery planes, runtime model |
| `components/src/dynamo/AGENTS.md` | new — Python tier, PyO3 boundary, entrypoints |
| `deploy/AGENTS.md` | new — Kubernetes tier, hands off to `deploy/operator/` |
| `lib/llm/AGENTS.md` | new symlink → `CLAUDE.md` |
| `components/src/dynamo/frontend/AGENTS.md` | new symlink → `CLAUDE.md` |
| `lib/mocker/src/replay/online/AGENTS.md` | new symlink → `CLAUDE.md` |
| `aisimulate/crates/core/src/replay/AGENTS.md` | new symlink → `CLAUDE.md` |
| `AGENTS.md` | edited — fix two link labels, add Concepts + Component guides |
Every new file carries the `<!-- SPDX -->` header in the root file's form, uses
GitHub-style admonitions, language-tagged fences, and vLLM / SGLang /
TensorRT-LLM casing. No new file uses a `## Skills` heading. Commits are signed
with DCO (`git commit -s`). PR title follows Conventional Commits —
`docs(agents): add component-level agent guides` — with `Summary` and
`Validation` sections in the description, and a `Notes` section recording the
deferred per-crate set and the un-enforced parity invariant as follow-ups. |
change.md# Change: component and crate-level agent guides
- **Branch:** `feat/crate-level-agent-mds--c80ee3b2d603` (based on `main` at
`bce85e29a70ba77720f2ce2815d760c2ef58d487`)
- **Commits (two, both signed off with `git commit -s`):**
- `34791cf6ab8c7b412da7cdcc409e05ad4f340349` — `docs: add component and
crate-level agent guides`
- `fa4adc13a204fb5319a073b78b3caa6abf3894da` — `ci: route the new tier agent
guides in areas.yaml`
(the CODEOWNERS correction described under
[The areas.yaml correction](#the-areasyaml-correction))
- **Diff:** `change.diff` in this directory, produced with
`git diff bce85e29a…..HEAD`, so it covers both commits against the branch point
- **Not pushed.** Nothing was pushed to any remote.
- **MR URL: https://github.com/ai-dynamo/dynamo/pull/13439**
Eleven paths changed, 451 insertions, 2 deletions. Four new prose files, four new
symlinks, one edited root file, plus `areas.yaml` and the regenerated
`CODEOWNERS`.
## What the change does
Agent harnesses read every `AGENTS.md` and `CLAUDE.md` on the path they descend.
Before this change the repository had exactly one broad orientation file at the
root plus a scattering of narrow leaf rules; there was nothing at the tier level,
so an agent that landed in `lib/kvbm-physical/` or `deploy/helm/` had the whole
repository's conventions and none of its neighbourhood's. This adds one guide per
major tier and restores four symlinks that had gone missing.
### 1. `lib/AGENTS.md` (new, 124 lines)
The Rust workspace tier. A crate map grouped by role — runtime and transport, LLM
serving, routing, KVBM, simulation and benchmarking, bindings — with the package
name for each directory. Per-crate build/test/clippy commands, the `cargo fmt --
<file>` habit over `cargo fmt --all`, and the maturin rebuild step. A `> [!NOTE]`
records that `bindings/python/` and `bindings/kvbm/` declare their own
`[workspace]` tables and so are not built by a root `cargo build`. Closes with a
list of which crates carry nested agent files and the precedence rule.
Every row was taken from that crate's own `Cargo.toml` `description` or its
`src/lib.rs` module docs; the grouping is editorial, the descriptions are not.
### 2. `lib/runtime/AGENTS.md` (new, 104 lines)
The `DistributedRuntime` → `Namespace` → `Component` → `Endpoint` hierarchy as a
diagram, with `Runtime` and `Worker` underneath it. A `> [!NOTE]` that
`DistributedRuntime` is not a process singleton. A table of the three
communication planes: what each carries, the environment variable that selects it,
the options with the default marked, and where the code lives. A source-layout
table. The closing paragraph flags that changes here are cross-cutting and points
at the wire-compatibility policy in `lib/llm/CLAUDE.md`.
### 3. `components/src/dynamo/AGENTS.md` (new, 110 lines)
The Python tier. Its centre of gravity is the boundary with Rust: a table showing
that the `dynamo` namespace is assembled from two source roots
(`components/src/dynamo/` → `ai-dynamo` via hatchling,
`lib/bindings/python/src/dynamo/` → `ai-dynamo-runtime` via maturin), that
`dynamo.runtime` is a re-export layer over the compiled `dynamo._core`, and a
`> [!NOTE]` that editing Rust and then running Python changes nothing until
`maturin develop --uv` runs. Then the `python3 -m dynamo.<package>` convention,
and a package map in three groups (serving path, routing, planning and analysis).
### 4. `deploy/AGENTS.md` (new, 72 lines)
The Kubernetes tier. Directory map, the nine custom resources and where their
generated schemas live, a `> [!NOTE]` that the generated artifacts are not
hand-editable and `make manifests` is the regeneration step, then an explicit
hand-off to the seven nested operator agent files with the precedence rule.
### 5-8. Four symlinks
`lib/llm/`, `components/src/dynamo/frontend/`, `lib/mocker/src/replay/online/`,
and `aisimulate/crates/core/src/replay/` each had a `CLAUDE.md` with no
`AGENTS.md` sibling — the only four such directories in the tree. Created with
`ln -s CLAUDE.md AGENTS.md` from inside each directory, so the target is relative.
All four are staged as mode `120000` with blob
`681311eb9cf453d0faddf3aacaec7357e97ba8e9`, which is the same blob the twenty
existing `AGENTS.md` symlinks already share.
### 9. `AGENTS.md` (root, edited, +33/-2)
Three surgical edits, no rewrite:
- Two link labels in the Repository Map read ``[`docs/AGENTS.md`](docs/fern/AGENTS.md)``
— label and target disagreed, and no `docs/AGENTS.md` exists. Labels corrected
to `docs/fern/AGENTS.md`; targets unchanged.
- A new **Component guides** section after the Repository Map: a four-row table
pointing at the tier guides, plus the statement of the descent and precedence
rule.
- A new **Concepts** section linking the five existing knowledge-base pages, so
the root file points at the architecture rather than restating it.
The `## Skills` section is byte-identical; `git diff AGENTS.md | grep -i skill`
returns nothing.
### 10-11. `.github/codeowners/areas.yaml` and `CODEOWNERS` (second commit, +6)
Three explicit area globs, and the regenerated `CODEOWNERS` that follows from
them. See the next section for why.
## The areas.yaml correction
The first commit failed validation. The prose held up, but
`build_codeowners.py --strict` exited 1 on the branch at `5578/5581 (99.95%)`,
with `lib/AGENTS.md`, `components/src/dynamo/AGENTS.md`, and `deploy/AGENTS.md`
listed as catch-all-only. The same failure reproduced under
`--strict --changed-only --base main`, which is the form the PR gate actually
runs. A clean clone of `main` exits 0 at 100.00%, so this change alone caused it.
**The plan's reasoning was wrong, and I inherited it.** The plan listed "no
`areas.yaml` edit" as a non-goal, justified by `who_owns.py` returning a team for
each new file. That check does not answer the question the gate asks.
`who_owns.py` reports the effective reviewer and is happy to reach it through the
`*` catch-all; `--strict` distinguishes explicit-glob coverage from catch-all
coverage and fails on the latter. `lib/runtime/AGENTS.md` escaped only because
`/lib/runtime/` is already an explicit glob — an accident of the existing policy,
not a property of the new files. I treated the plan's non-goal as settled and did
not run the gate itself before committing, which is the mistake; the gate is
cheap and would have caught this in seconds.
The catch-all resolution was also substantively wrong for one file: it routed
`deploy/AGENTS.md` to the runtime team, who do not own that tree.
**Owners chosen.** Each tier file goes to the team that owns the tree it
describes, which is the shape the existing policy already uses for an agent file
— the runtime area claims `tests/CLAUDE.md` directly rather than leaving agent
files to the `process` area:
| Path | Area | Why |
|------|------|-----|
| `lib/AGENTS.md` | `runtime` → `@ai-dynamo/dynamo-runtime-codeowners` | Already owns `lib/runtime/`, `lib/bindings/`, `lib/llm/`, `lib/backend-common/`, `lib/truthy/` — the largest explicit claim on `lib/` |
| `components/src/dynamo/AGENTS.md` | `runtime` → `@ai-dynamo/dynamo-runtime-codeowners` | Already owns `components/src/dynamo/common/`, the shared library every other package in that tier builds on; no other area spans the Python tier |
| `deploy/AGENTS.md` | `operator` → `@ai-dynamo/dynamo-operator-codeowners` | Already owns `deploy/CONTRIBUTING.md`, `deploy/__init__.py`, `deploy/helm/`, `deploy/utils/`, `deploy/operator/` — this corrects the wrong catch-all routing |
For the first two the resulting team matches what the catch-all produced, but it
is now asserted rather than defaulted, which is what `--strict` requires and what
keeps it stable if the catch-all team ever changes.
`CODEOWNERS` was regenerated with `emit_codeowners.py`, never hand-edited, per
the root `AGENTS.md` rule. It gained exactly three lines. `CONTRIBUTORS.md` is
also produced by that generator and came out byte-identical, so it is not in the
commit.
`lib/runtime/AGENTS.md` needed no glob — `/lib/runtime/` already covers it — so
no glob was added for it.
### Gate outputs after the fix
All run from the checkout root, after the second commit:
| Gate | Command | Result |
|------|---------|--------|
| Coverage, full tree | `build_codeowners.py --areas … --repo . --strict` | **exit 0**, `explicitly owned: 5581/5581 (100.00%) \| catch-all only: 0` |
| Coverage, PR form | `build_codeowners.py --areas … --repo . --strict --changed-only --base main` | **exit 0**, same 5581/5581 (100.00%) |
| Drift | `emit_codeowners.py --areas … --out CODEOWNERS --external … --contributors-out CONTRIBUTORS.md` then `git diff --exit-code CODEOWNERS CONTRIBUTORS.md` | **exit 0** — no drift |
| Unit tests | `python3 -m pytest .github/codeowners/test_codeowners.py --override-ini="addopts="` | **93 passed** |
The `--changed-only` run reports whole-tree numbers rather than a changed-file
subset. That is the tool's documented behaviour, not a mis-invocation: the
workflow comment states a PR that edits ownership policy is judged full-tree,
because a policy edit can orphan any path. This PR edits `areas.yaml`, so it gets
the strictest form.
Per-file resolution now, via
`who_owns.py --codeowners CODEOWNERS <the four tier files>`:
```text
lib/AGENTS.md @ai-dynamo/dynamo-runtime-codeowners
lib/runtime/AGENTS.md @ai-dynamo/dynamo-runtime-codeowners
components/src/dynamo/AGENTS.md @ai-dynamo/dynamo-runtime-codeowners
deploy/AGENTS.md @ai-dynamo/dynamo-operator-codeowners
```
`pre-commit run --files .github/codeowners/areas.yaml CODEOWNERS` was re-run for
the second commit: `codespell`, `check yaml`, `check for case conflicts`, `check
for merge conflicts`, `mixed line ending`, `trim trailing whitespace`, `check that
scripts with shebangs are executable`, `Check canonical AI Simulate docs are
synced to Fern`, and `Report pytest markers` all passed; the rest reported no
matching files. `validate-skills` does not match either path and was skipped here
— it was already green over the first commit's nine paths.
## Verification
### Every architectural claim was read out of the tree first
No sentence in the four new files was written from memory. Representative
examples:
| Claim | Source checked |
|-------|----------------|
| Crate directory → package name mapping | root `Cargo.toml` `[workspace] members`, then each crate's `Cargo.toml` `[package] name` |
| Each crate's one-line role | that crate's `Cargo.toml` `description` or `src/lib.rs` module docs |
| The two PyO3 crates are outside the root workspace | `[workspace]` table present in `lib/bindings/python/Cargo.toml` and `lib/bindings/kvbm/Cargo.toml` |
| `kvbm-py3` / wheel `kvbm` | `lib/bindings/kvbm/Cargo.toml` `name`, `lib/bindings/kvbm/pyproject.toml` `name` |
| Object model and singleton caveat | module docs of `lib/runtime/src/component.rs`, doc comment on the struct in `lib/runtime/src/distributed.rs` |
| Plane env vars, options, defaults | `RequestPlaneMode::from_env` and `DistributedConfig::from_settings` in `lib/runtime/src/distributed.rs` |
| `src/component/` contents row | `ls lib/runtime/src/component/` → client.rs, component.rs, endpoint.rs, namespace.rs, registry.rs, service.rs |
| `dynamo.runtime` re-exports from `dynamo._core` | `lib/bindings/python/src/dynamo/runtime/__init__.py` |
| `ai-dynamo` wheel packages `components/src/dynamo` | root `pyproject.toml` `[tool.hatch.build.targets.wheel] packages` |
| Which Python packages are runnable | presence of `__main__.py` per package; `common/` has none |
| Per-package roles | each package's `README.md` or `__init__.py` docstring |
| `deploy/` directory roles | the structure block in `deploy/CONTRIBUTING.md` |
| The nine CRD kinds | filenames in `deploy/operator/config/crd/bases/` |
| `make manifests` regenerates CRDs and RBAC | the `manifests:` target in `deploy/operator/Makefile`, cross-checked against `deploy/operator/AGENTS.md` |
| `epp` is Go, `ext-proc` is Rust | `go.mod` and `Cargo.toml` respectively |
| Which directories carry nested agent files | `git ls-files '**/AGENTS.md' '**/CLAUDE.md'` |
Two counts in the files were recounted rather than trusted: the `kv-router/`
subdirectory count (six, not five as first written) and the bindings table, where
a placeholder for the KVBM package name was replaced with the verified value.
Every relative link target in all five files was confirmed to exist on disk —
including all five `docs/fern/pages/developer-guide/knowledge-base/` pages, both
`.ai/` guideline files, and each nested `AGENTS.md` referenced from the root table
and from `deploy/AGENTS.md`. One malformed relative path
(`…/concepts/../../modular-components/…`) was caught during review and rewritten.
### Where I omitted a claim rather than guess it
- `plan.md` states the workspace has 39 members. Parsing the root `Cargo.toml`
with `tomllib` returns **37**. Rather than pick a side, none of the new files
states a member count — the files say the `[workspace] members` array is
authoritative and link to it. Flagging the discrepancy here because the plan is
the artefact that is wrong, not the tree.
- `lib/runtime/AGENTS.md` describes the plane-selection functions by name and
says the logic lives in `src/distributed.rs`, but does not quote line numbers,
which would rot on the next edit.
### Hooks actually run
`/opt/dynamo/venv/bin/pre-commit run --files <the nine paths>`, from the checkout
root. Result: **all applicable hooks passed, none failed.**
Passed: `codespell`, `check for case conflicts`, `check for merge conflicts`,
`check that scripts with shebangs are executable`, `mixed line ending`,
`trim trailing whitespace`, `Check canonical AI Simulate docs are synced to Fern`,
`Report pytest markers (static + inherited)`, and
`Validate .agents/skills SKILL.md frontmatter and AGENTS.md index`.
The last of these is the one that mattered: its `files` regex matches any path
containing `AGENTS.md`, so all nine paths fire it even though it only reads the
root `AGENTS.md` and `.agents/skills/`. It is green, confirming the root edit did
not desynchronise the skills index — as expected, since the `## Skills` section
was not touched.
Skipped as not-applicable (no matching files in the nine): the Python hooks
(`isort`, `black`, `flake8`, `ruff`), `clang-format`, the JSON/TOML/YAML checks,
`fix requirements.txt`, and the Fern component/asset self-tests. `fix end of
files` also reported no files to check for this set; the four new prose files
each end with exactly one newline regardless.
No Rust, Python, or Go source changed, so `cargo` and `go test` were not run —
there is nothing in either commit they could exercise. The one test suite that is
relevant, `.github/codeowners/test_codeowners.py`, was run for the second commit
and is reported above.
### Safety constraints inspected
- **Branch:** created from `main`, named exactly
`feat/crate-level-agent-mds--c80ee3b2d603`. `git rev-parse --abbrev-ref HEAD`
confirms. No other branch created, no branch renamed.
- **No push.** No remote operation of any kind was performed.
- **`lib/bindings/python/Cargo.lock`** was modified before I started and is
deliberately excluded. It is absent from `git show --stat HEAD` and from
`change.diff`; `git status --short` still shows it as an unstaged ` M`, which is
the intended end state.
- **Scope:** the first commit is exactly the nine authorised paths and no more.
The second commit adds `.github/codeowners/areas.yaml` and the generated
`CODEOWNERS` — outside the original scope, and taken deliberately on the
coordinator's instruction after the CODEOWNERS gate failed; see
[The areas.yaml correction](#the-areasyaml-correction). It touches none of the
nine paths and changes no prose. No pre-commit hook added, no existing
`CLAUDE.md` converted or relocated, no `CLAUDE.md` twin created for any new
file.
- **Writes** are confined to the checkout and this work-item root.
- **No internal infrastructure** — no host, URL, API path, proxy address, IP, or
invocation ID appears in any file in this change or in these artefacts.
- **History:** two new commits, the second appended on top of the first; no
commit amended, rebased, or rewritten.
### Diff integrity
`change.diff` was checked in both directions: `git apply --check` succeeds against
a temporary worktree checked out at `main` (forward), and `git apply --check
--reverse` succeeds against the working tree (confirming it describes exactly the
committed state). The temporary worktree was removed. Symlink creations appear in
the diff as `new file mode 120000` with a one-line body, which is the form
`git apply` restores as a symlink.
## What I could not do
- CI was not run; only local pre-commit hooks were available.
- The `validate-skills` hook confirms the skills index is intact but nothing in
the repository validates nested `AGENTS.md` files — no hook checks their
frontmatter, links, or existence. The link checking reported above was done by
hand, path by path.
- The twenty pre-existing nested agent files were left alone. Several of them are
covered only because a broader directory glob happens to include them; that is
the status quo and this change does not audit or alter it. Only the paths this
change introduces were routed. |
change-validation.md# Validation: component and crate-level agent guides
Branch `feat/crate-level-agent-mds--c80ee3b2d603`, now two commits on top of `main`
(`bce85e29a`):
- `34791cf6a` — `docs: add component and crate-level agent guides`. Nine paths, +445/−2:
four new prose files (`lib/AGENTS.md`, `lib/runtime/AGENTS.md`,
`components/src/dynamo/AGENTS.md`, `deploy/AGENTS.md`), four `AGENTS.md` → `CLAUDE.md`
symlinks, and an edit to the root `AGENTS.md`.
- `fa4adc13a` — `ci: route the new tier agent guides in areas.yaml`. Two paths, +6:
three globs in `.github/codeowners/areas.yaml` and the three `CODEOWNERS` lines they
generate. Appended, not amended. Signed off.
The second commit exists because my first pass failed this change. That history is part
of the record and I am not going to present the current green run as though it were the
first one.
The plan named two recipes, `01-python-lint` and `05-code-inspection`. Both were run at
the new tip; neither was skipped. Every command went through the recorder, and the logs
it produced are the only thing I am asking you to believe. Engine, GPU, Docker, and
Kubernetes recipes were not run and are not blocked — a change that adds markdown and
six lines of ownership policy gives them nothing to exercise. Recipe 0 is not triggered:
no shipped file in this change imports `dynamo.*`.
## What the first pass found, and why
On `34791cf6a` the CODEOWNERS coverage gate failed. `build_codeowners.py --strict` exited
1 at `explicitly owned: 5578/5581 (99.95%) | catch-all only: 3`, naming
`lib/AGENTS.md`, `components/src/dynamo/AGENTS.md`, and `deploy/AGENTS.md`. The same
failure appeared under `--strict --changed-only --base <main>`, which is how the
repository's CODEOWNERS workflow gates a pull request. A negative control on base `main`
passed at 100.00%, so the regression was caused by that commit and nothing else.
The interesting part is why it got past planning. The plan asserted that `who_owns.py`
resolves every proposed path to a real team, so no `areas.yaml` edit was needed.
`who_owns.py` does return a team for all four new prose files — but it returns it via the
`*` catch-all rule at the bottom of the generated `CODEOWNERS`, not via an explicit area
glob. `build_codeowners.py --strict` distinguishes those two cases and fails on the
second. A tool that answers "who reviews this?" was used to answer "is this explicitly
owned?", and those are different questions. `lib/runtime/AGENTS.md` escaped only because
`/lib/runtime/` was already an explicit glob.
That failure is fixed. What follows is what I observe now.
## The correction commit
I read the commit rather than the printer's summary of it. It adds exactly three globs —
`components/src/dynamo/AGENTS.md` and `lib/AGENTS.md` to the `runtime` area,
`deploy/AGENTS.md` to the `operator` area — and the three corresponding `CODEOWNERS`
lines. `deploy/AGENTS.md` → `@ai-dynamo/dynamo-operator-codeowners` is the routing the
catch-all previously got substantively wrong, and it now sits beside the existing
`/deploy/helm/`, `/deploy/utils/`, and `/deploy/operator/` rules, which is the right
neighbourhood for it.
`CODEOWNERS` was generated, not hand-edited, and I proved that rather than accepting it:
in an isolated clone of the tip I re-ran `emit_codeowners.py` and then
`git status --porcelain` plus `git diff --exit-code CODEOWNERS CONTRIBUTORS.md`. The
status output was empty and the diff returned 0 — the committed `CODEOWNERS` is
byte-identical to what the generator produces from the edited `areas.yaml`.
`CONTRIBUTORS.md` genuinely did come out byte-identical, which is why it is absent from
the commit. Two independent confirmations: its blob is `371e010c…` at both `bce85e29a`
and the tip, unchanged by either commit; and the regeneration above rewrote it and
produced no diff.
## The gate, re-run
All four commands run through the recorder with exit codes captured bare — the earlier
swallowed-exit problem, where a trailing `echo` masked a failure as exit 0, is not
repeated here.
- `build_codeowners.py --areas … --repo . --strict` → **exit 0**,
`explicitly owned: 5581/5581 (100.00%) | catch-all only: 0`. No `!! strict:` line.
- The same command with `--strict --changed-only --base <main>` → **exit 0**, same
numbers.
- Drift check (`emit_codeowners.py` then `git diff --exit-code CODEOWNERS
CONTRIBUTORS.md`), run in an isolated clone so nothing was written into the printer's
checkout → **exit 0**, no drift.
- `pytest .github/codeowners/test_codeowners.py --override-ini="addopts="` → **93
passed**, exit 0.
The printer's reported numbers match what I observed, but I re-derived them; I did not
take its table.
### The whole-tree caveat: confirmed, not a mis-invocation
The printer flagged that the `--changed-only` run reports whole-tree numbers. That is
documented behaviour, and it is documented in three places I read directly:
- `build_codeowners.py:66`, `def is_policy_change(...)`, whose docstring reads "True if
the PR touches ownership policy -> judge coverage whole-tree", with the reasoning that
a policy edit can orphan untouched paths.
- The `--changed-only` CLI help at `build_codeowners.py:110`: "PR that edits ownership
policy is still judged full-tree."
- The workflow's own comment block at `.github/workflows/codeowners.yml:45-53`, which
says the diff-aware PR gate falls back to full-tree when the PR edits policy.
The tool also announces it at runtime. My `--changed-only` log carries the line
`note: PR touches ownership policy (areas/scripts/CODEOWNERS); evaluating full-tree
coverage instead of the changed surface`. So: **confirmed**. The invocation is correct
and the whole-tree numbers are the intended, self-announced behaviour for a PR of this
shape — and in this case it is the stricter reading, not a looser one.
## 01-python-lint, re-run over the grown path set
`pre-commit run --files $(git diff --name-only <main> HEAD) --hook-stage manual` — eleven
paths now, the original nine plus `.github/codeowners/areas.yaml` and `CODEOWNERS`.
Exit 0. Ten hooks matched and all ten `Passed`; nothing failed. The newly matched one is
`check yaml`, which the first pass never exercised because there was no YAML in the
change — it passes on the edited `areas.yaml`. `codespell`, `check for case conflicts`,
`check for merge conflicts`, `check that scripts with shebangs are executable`,
`mixed line ending`, `trim trailing whitespace`, the AI Simulate docs sync check, the
pytest-marker report, and `Validate .agents/skills SKILL.md frontmatter and AGENTS.md
index` all pass as before.
The skills hook is the one worth naming: its `files` pattern matches any path containing
`AGENTS.md`, so all four new prose files and all four symlinks fire it, and it re-reads
the root `AGENTS.md` against `.agents/skills/`. Green here is real evidence that the root
edit did not desynchronise the skills index.
One discrepancy against the plan persists, non-blocking: `fix end of files` still reports
`(no files to check) Skipped`, so the plan's expectation that `end-of-file-fixer` would
exercise the new prose is still not met by the hook run. I checked the files directly —
each ends with exactly one newline — but the hook is not what establishes that.
## The correction did not disturb the first commit's work
I did not take this on trust either. Every one of the nine original paths has a
byte-identical blob at `34791cf6a` and at the tip — same object hash, all nine. The
correction commit touches only `.github/codeowners/areas.yaml` and `CODEOWNERS`. The four
symlinks are still mode `120000` carrying blob `681311eb…`, the same blob the twenty
pre-existing `AGENTS.md` symlinks share.
Because the prose blobs are unchanged, the Section 2 findings below carry over
unchanged. I re-ran the claim verifier at the new tip anyway: **failed claims 0**.
## Section 2 — every architectural claim against a real `file:line`
A documentation file that describes architecture is a set of factual assertions about
source. I did not accept the printer's narrative for any of them; I re-derived each one
from the tree, mechanically, through the recorder. Roughly a hundred discrete claims
across the four new files were checked. **Failed claims: 0.**
`lib/AGENTS.md`. All twenty-six directory → package rows verified against each crate's
own `Cargo.toml` `[package] name`, including the two that are easy to get wrong:
`bindings/c/` → `libdynamo_llm` and `bindings/kvbm/` → `kvbm-py3` with wheel `kvbm`. The
`> [!NOTE]` claiming the two PyO3 crates sit outside the root workspace is correct on
both halves — each declares its own `[workspace]` table, and neither appears in the root
`[workspace] members`. The `bindings/python/` row's `module dynamo._core` is backed by
`[lib] name = "_core"`. The opening sentence that everything under `lib/` is Rust except
`lib/gpu_memory_service/` holds: that directory contains zero `.rs` files and a
`pyproject.toml`. The counts in the closing "Nested agent files" paragraph were recounted
rather than trusted — `kv-router/` has exactly six subdirectories carrying an agent file
and `llm/` exactly four. The `rl/` row's `DYN_ENABLE_RL` / `--enable-rl` opt-in is
referenced in `lib/rl/src/lib.rs` and in the vLLM backend.
`lib/runtime/AGENTS.md`. `DistributedRuntime` at `lib/runtime/src/distributed.rs:53`; the
namespace/component/endpoint hierarchy in the module docs of
`lib/runtime/src/component.rs:4`; `Runtime` at `src/runtime.rs:55` and `Worker` at
`src/worker.rs:49`, both re-exported from `src/lib.rs` (lines 55 and 61). The
not-a-process-singleton `> [!NOTE]` is a fair reading of the doc comment at
`src/distributed.rs:45`. Every cell of the three-plane table was checked against the
selection code, not against prose: request plane via `RequestPlaneMode::from_env` at
`distributed.rs:834` reading `DYN_REQUEST_PLANE` at `:835`, with `enum RequestPlaneMode`
at `:799` carrying `#[default] Tcp`; event plane via `resolve_event_transport_kind` at
`:663`, whose own comment calls itself the single authoritative mapping and which
defaults to Zmq when the variable is unset; discovery via
`DistributedConfig::from_settings` at `:696`, whose in-source comment enumerates exactly
`"kubernetes", "etcd" (default), "file", "mem"` and whose default is `etcd`. Every row of
the source-layout table resolves to a real path, and the two rows that summarise
directory contents were verified by listing them: `src/component/` holds
client/component/endpoint/namespace/registry (plus `service.rs`), and `src/transports/`
holds `tcp.rs`, `nats.rs`, `zmq.rs`, `etcd/`, `event_plane/`.
`components/src/dynamo/AGENTS.md`. The two-source-root table is exact: the root
`pyproject.toml` declares `name = "ai-dynamo"` with the hatchling backend and packages
`components/src/dynamo`, and `lib/bindings/python/pyproject.toml` declares
`name = "ai-dynamo-runtime"` with the maturin backend. The re-export claim is literal —
`lib/bindings/python/src/dynamo/runtime/__init__.py` imports `Client`, `Context`,
`DistributedRuntime`, and `Endpoint` from `dynamo._core`. Every package named in the
three-group map exists on disk; every one of them except `common/` has a `__main__.py`,
and `common/` does not, so the stated exception is exactly right. The cross-references
hold too: `frontend/CLAUDE.md` does carry a configuration-boundary rule, `sglang/` and
`common/backend/` do carry their own agent files, `tokenspeed/` really does implement
`LLMEngine` from `dynamo.common.backend.engine`, `common/backend/` really does define
`Worker` and `BaseEngine`, and `replay/` really does import its offline implementation
from `aisimulate`.
`deploy/AGENTS.md`. Every directory-map row is a real directory. `epp/` has a `go.mod` and
`ext-proc/` has a `Cargo.toml`, as claimed. The nine custom resources match the nine files
under `deploy/operator/config/crd/bases/` one-to-one, with no tenth file. The
`make manifests` `> [!NOTE]` is the strongest claim in the file and it survives reading:
the target at `deploy/operator/Makefile:238` invokes the generator with
`rbac:roleName=manager-role` and `output:crd:artifacts:config=config/crd/bases`, and the
same target ends at line 296 by copying the generated role into the platform chart — so
"regenerates both the CRD bases and the RBAC role vendored into the platform chart" is
accurate, including the part about the chart. All seven nested operator agent files
referenced in the hand-off exist.
Links. Independently of the printer, I resolved all 68 relative markdown link targets
across the five changed files, requiring each to exist on disk *and* be tracked by
`git ls-files`. Missing: **0**.
## The parity sweep, with its negative control
This is the invariant the change introduces, and it is proven with a real pair. On `main`,
exactly four directories hold a `CLAUDE.md` with no `AGENTS.md` sibling:
`lib/llm`, `components/src/dynamo/frontend`, `lib/mocker/src/replay/online`, and
`aisimulate/crates/core/src/replay`. At the tip the same sweep returns none of them. The
scan is not vacuous — it walks 30 `CLAUDE.md` files — and the `AGENTS.md` count moves
34 → 42, which is the four new prose files plus the four new symlinks and nothing else.
One transparency note on my own instrument: the `git ls-tree` form of the sweep also
reports the repository root as a miss, because `dirname` yields `.` and the sibling is
listed as `AGENTS.md` rather than `./AGENTS.md`. That is an artefact of my path
normalisation, it appears identically on both refs, and the root `AGENTS.md` plainly
exists. The real delta is four gaps closed and none opened.
## Section 4
Unreachable pre-publication: the branch is unpushed, there is no PR, and remote
operations are not permitted from this role, so real CI results cannot be read. The local
surrogate for the gate CI would run is the `--strict --changed-only` invocation above,
and it is green.
## The plan's "39 workspace members"
Parsing the root `Cargo.toml` returns **37**. The plan is wrong and the printer was right
to flag it. It matters that this error never reached the diff: no shipped file states a
member count, and `lib/AGENTS.md` instead points at the `[workspace] members` array as
authoritative. The defect belongs to the plan as an artefact, not to the change under
review, and it is not part of my verdict.
## Investigation outcome:
The gate that failed is now green, and it is green for the right reason. The fix did not
paper over the finding: it added the explicit ownership the `--strict` check was asking
for, routed `deploy/AGENTS.md` to the team that owns the tree it documents rather than
leaving it on the catch-all, and regenerated `CODEOWNERS` through the generator — which I
confirmed byte-for-byte rather than eyeballing. Coverage is back to 5581/5581 (100.00%)
with zero catch-all-only files, on both the full-tree and the PR-shaped invocation, and
the whole-tree behaviour of the latter is documented in the tool, its CLI help, the
workflow, and a runtime note the tool prints itself.
Nothing was disturbed in the process. All nine paths from the first commit are
byte-identical at both commits, the symlinks still share the canonical blob, the parity
invariant still holds with its negative control intact, and re-running the claim
verifier at the new tip still returns zero failures across roughly a hundred
architectural assertions. Recipe 01 is green over the enlarged path set, with `check yaml`
newly exercised on the edited policy file.
Two things remain on the record without blocking. The plan's member count is wrong,
though it never reached a shipped file. And `fix end of files` does not match these paths,
so the plan's expectation of that hook is unmet — the files are fine, but the hook is not
what proves it.
One structural observation for whoever merges: the fix is a follow-up commit rather than
an amendment, so `34791cf6a` in isolation still fails the coverage gate. That is invisible
if the branch merges as a unit or squashed, and it is worth knowing if anything ever
bisects onto that commit. I do not treat it as a defect in the branch as submitted.
## Verdict: pass
The CODEOWNERS coverage gate, which I failed this change on in the first pass, now exits
0 at 5581/5581 (100.00%) with zero catch-all-only files — in the full-tree form, in the
PR-shaped `--changed-only` form, and with a drift check and 93 unit tests green alongside
it. Recipe 01 passes over the enlarged path set with no failing hook. Roughly a hundred
architectural claims across the four new guides are each supported by a real `file:line`,
with zero contradictions, and the parity invariant the change establishes is demonstrated
with a genuine positive/negative pair. Nothing is outstanding. |
review.md> 🤖 **Automated AI review — advisory.** An AI agent's judgment of whether this change is logically sound on the evidence; not a merge authorization. CI, customs, and a human reviewer hold that.
## Assessment: sound
Two commits on `feat/crate-level-agent-mds--c80ee3b2d603` (`34791cf6a`, then the correction `fa4adc13a`), tip verified against the recorded packets. Eleven paths, 451 insertions / 2 deletions: four new tier-level `AGENTS.md` prose files, four `AGENTS.md` symlinks restoring the `CLAUDE.md`⇄`AGENTS.md` twin invariant, three surgical root `AGENTS.md` edits, three `areas.yaml` globs and the regenerated `CODEOWNERS` lines.
I re-derived the load-bearing claims myself rather than reading the validator's conclusions. Summary of what I checked and found.
### Does it answer the ask
The Slack ask was for agent context explaining key concepts (request/event planes, router, disagg) plus codebase structure, and explicitly preferred component/crate-level files over a more complex root one. Judged on the delivered files, not the plan's self-description:
- `lib/runtime/AGENTS.md` is the direct hit. A cold agent landing in `lib/runtime/` previously had no agent file and no README; it now gets the `DistributedRuntime → Namespace → Component → Endpoint` object model, a plane table (request / event / discovery) with env var, options, default and the code location for each, the not-a-singleton caveat, and a source-layout table. That is precisely the "request/event planes" orientation asked for, placed where the agent lands.
- `deploy/AGENTS.md` is the second real improvement: a directory map, the nine CRDs enumerated against their generated bases, the `make manifests` regeneration rule, and an explicit handoff to the seven nested operator files.
- Router and disaggregation are covered by a table row plus links rather than new prose. That is a narrower answer than the Slack thread's phrasing invites, but it is consistent with the plan's stated non-goal of not authoring new architecture prose that would immediately drift, and the links resolve. I do not treat it as a defect.
Net: yes, a cold agent descending into `lib/runtime/` or `deploy/` is genuinely better oriented than before.
### Is the prose true
I sampled roughly forty architectural assertions across all four new files and re-derived each against source, independently of the validator's sweep: the crate→package-name map; the deliberate workspace exclusion of the two PyO3 crates (`lib/bindings/python`, `lib/bindings/kvbm` each carry their own `[workspace]`); the plane table's env vars, option sets and defaults (`RequestPlaneMode` with `#[default] Tcp`; `resolve_event_transport_kind()` documenting ZMQ-default / NATS-opt-in; the `DYN_DISCOVERY_BACKEND` value list); the not-a-singleton caveat; the `__main__.py` entrypoint convention (only `common/` lacks one); the two-source-root wheel table and the `dynamo._core` re-exports; nine CRD kinds against exactly nine generated bases; `epp/` Go and `ext-proc/` Rust; `make manifests` including the RBAC-into-chart step; the `power-agent` (Python, power-cap) and `snapshot` (Go, CRIU) DaemonSets; the "experimental, not a released component" labelling of `thunderagent_router` and `squeeze_evolve`; the `DYN_ENABLE_RL` / `--enable-rl` opt-in; the kv-router-six / llm-four nested-file counts. **Zero false assertions.** A link resolver over all five changed markdown files found zero broken relative targets.
### Findings
- **nit — `lib/AGENTS.md:119`** (`## Nested agent files`). The enumeration of crates and modules carrying their own agent file lists `kv-router/` and six subdirectories, `llm/` and four subdirectories, `backend-common/`, `bench/`, `data-gen/`, the three `kvbm-*` crates and `runtime/` — but omits `mocker/src/replay/online/`, which carries a `CLAUDE.md` and to which *this change itself* adds the `AGENTS.md` symlink. Every other entry is exactly right. The sentence is hedged ("Some crates and modules carry their own…"), so this is incomplete rather than false, and the omission costs nothing at read time because an agent descending into that directory reads the file regardless. Worth one word to close, not worth blocking.
- **non-blocking structural note.** The `areas.yaml` correction was appended rather than amended, so `34791cf6a` in isolation still fails the CODEOWNERS coverage gate. Invisible on a squash merge, which is how this lands; it would only surface to someone bisecting across the pair. The packets disclose it rather than papering over it.
### Scope discipline — the `areas.yaml` edit
The plan listed "not touching `.github/codeowners/areas.yaml`; discovery proved no edit is needed" as a non-goal, and the change crosses it. I judge this the right call, and honestly reported. I reproduced both ends: in an isolated worktree at `34791cf6a`, `build_codeowners.py --strict` exits **1** at `5578/5581 (99.95%) | catch-all only: 3`; at tip it exits **0** at `5581/5581 (100.00%) | catch-all only: 0`. So the first commit really did break a gate that CI runs on every PR, and the second commit really repairs it. This is a fix, not symptom suppression — the three new files are given explicit owners rather than the gate being relaxed.
`change.md` states plainly that "the plan's reasoning was wrong, and I inherited it", and names the root cause correctly: `who_owns.py` answers "who is the effective reviewer" and is satisfied by the `*` catch-all, whereas `--strict` distinguishes explicit-glob coverage from catch-all coverage. That is a real diagnosis of a discovery-stage error, not a retrofit.
The owners chosen are defensible: routing `deploy/AGENTS.md` to the operator area is a substantive improvement over the catch-all, which would otherwise have sent Kubernetes-layer guidance to the runtime owners. `emit_codeowners.py` drift is clean and `pytest .github/codeowners/test_codeowners.py` passes.
### Evidence and honesty
Both recipes named in the plan's `validation-recipes` fence are present and validated; neither is missing. I read both cited logs end to end and each shows the command genuinely executing and passing — the pre-commit run shows ten hooks `Passed` with none failed (including `check yaml` and the skills validator, which every nested `AGENTS.md` fires), and the codeowners run shows `explicitly owned: 5581/5581 (100.00%) | catch-all only: 0`. The one `Skipped` hook (`fix end of files`, no files to check) is disclosed in the validation packet rather than glossed.
Two `failing` dispositions from the first pass remain on record and are *not* superseded by a later note. That is harmless here — the deciding runs are later than the dispositions, and a stale disposition cannot downgrade them — and I prefer the failures staying visible to them being edited away. The validation packet likewise refuses to present the green run as the first one.
The plan's "39 workspace members" is wrong; the real count is 37. Both `change.md` and the validation packet flag it, and I confirmed the error never reached a shipped file: no delivered `AGENTS.md` states a member count. It is a planning-artefact error, correctly quarantined.
The deferred work — roughly 27 per-crate files, and the un-enforced `CLAUDE.md` ⇒ `AGENTS.md` parity pre-commit hook — is named explicitly with rationale rather than left implicit. I independently ran the parity sweep: base `bce85e29a` has exactly the four claimed missing siblings, tip has zero. The invariant this change restores is real and currently unguarded, which is stated rather than hidden.
Nothing here suppresses a symptom, asserts the implementation back at itself in place of behaviour, or draws a conclusion the recorded evidence does not carry.
## Assessment: sound |
|
CI result: passed All required checks on
Full CI beyond the pre-merge set runs only after a maintainer comments |
|
@coderabbitai full review\n\n@devin-ai-integration review this PR at current head fa4adc1. |
Signed-off-by: GLAMR <svc-glamr@nvidia.com>
|
nursery: @coderabbitai full review |
|
nursery: /devin review |
|
✅ Action performedFull review finished. |
|
nursery: @dynamo-ops please run full CI for 94ec7e9 |
|
/ok to test 94ec7e9 |
Withdrawing my approval because new commits landed after it. I will review the new commits before I approve again.
dmitry-tokarev-nv
left a comment
There was a problem hiding this comment.
Approving 94ec7e9fb0e190b991941a49b69678bbdfb9c22a. No finding is open.
The new commit fixes the last P3 finding, at components/src/dynamo/AGENTS.md:64-65. It applies my suggestion as written, so for these two lines I am partly reviewing my own text. I authored no commits on this PR. This approval replaces my approval of 3c06f12900, which I dismissed after the push.
What I tested at this head and on a merge with main.
The merge is with main at 9979c32f53, and it has no conflicts.
- Only
tokenspeed/, the two sample engines and three test doubles subclass the engine classes incommon/backend/engine.py.vllm/,sglang/,trtllm/andtriton/import only helper functions fromcommon/backend. - The package map names all 19 packages under
components/src/dynamo/. - All 70 relative links in the five changed guides resolve, and so do their 5 heading anchors.
scripts/validate_agent_instructions.pypasses with 46 scopes.CODEOWNERSregenerates fromareas.yamlwith no difference.build_codeowners.py --strictreports 5934 of 5934 files owned, and the 157 unit tests of the codeowners tools pass.
Deferred on purpose: I did not build the Rust workspace or run the python3 -m commands again, because this push changes no code.
Signed-off-by: GLAMR <svc-glamr@nvidia.com>
|
nursery: @dynamo-ops please run full CI for 33e8fab |
|
/ok to test 33e8fab |
✅ Dynamo PR CI passed — run 41104863085 (attempt 1) on
|
Signed-off-by: GLAMR <svc-glamr@nvidia.com>
|
nursery: @dynamo-ops please run full CI for 04775b4 |
|
/ok to test 04775b4 |
Withdrawing my approval because new commits landed after it. I will review the new commits before I approve again.
Signed-off-by: GLAMR <svc-glamr@nvidia.com>
|
nursery: @dynamo-ops please run full CI for 078b2ab |
|
/ok to test 078b2ab |
|
nursery: Updated the PR description at head |
Summary
Add tier-specific agent guides for the Rust workspace, runtime crate, Python components, and Kubernetes deployment, and link them from the root guide.
Overview
Agents descending into each tier get its source map, local development conventions, and links to canonical architecture documentation.
Key Changes
Core logic added: 0 lines. This PR changes agent guidance and ownership routing.
Each new
AGENTS.mdhas a siblingCLAUDE.mdcontaining@AGENTS.md. Ownership policy explicitly routes the tier guides to runtime or operator owners, andCODEOWNERSis regenerated from that policy. Nested-guide discovery follows the root instruction-file convention.The component map includes Triton and the experimental unary APIs. Policy references link directly to canonical
AGENTS.mdfiles. The deployment and CRD inventory reflects the current source tree. Runtime behavior is unchanged.Where should the reviewer start?
Start with the Component Guides section in
AGENTS.md, then readlib/AGENTS.md,lib/runtime/AGENTS.md,components/src/dynamo/AGENTS.md, anddeploy/AGENTS.md. Review routing is defined in.github/codeowners/areas.yaml.Issues and Ongoing Work
Closes #16469
mainshortened the root guide. The merge preserves that content and adds the tier-guide and concept links from this PR.Validation
mainat5f3c7e3323ebce2c615b8b9c895f2f1da23b7af6and resolved the root guide conflict in signed-off commit078b2abe0632f9a8db805c6d4da91cc2e86c9847. The root retains the tier and concept links alongside the updated upstream content.CODEOWNERSwas regenerated from the merged policy, and whitespace checks passed.pre-commitis unavailable; the repository linters were used for the edited guide. No local tests, builds, or benchmarks were run. The merged head will be validated through remote CI.Pull Request Checklist
/pr-checklistskill.Summary by CodeRabbit
Documentation
Chores