Skip to content

refactor(router): move RouterHintSourceMetadata to kv-indexer and WorkerAffinityTarget to session-affinity - #196

Closed
ishandhanani wants to merge 2 commits into
idhanani/rs2-18-trim-guardrailsfrom
idhanani/rs2-19-protocol-narrowing
Closed

ishandhanani wants to merge 2 commits into
idhanani/rs2-18-trim-guardrailsfrom
idhanani/rs2-19-protocol-narrowing

Conversation

@ishandhanani

Copy link
Copy Markdown
Owner

Router split v2, unit 19 of 19. Stacked on #194 (unit 18).

Summary

Protocol narrows to the types every crate names (DYNO-195). Owner decision 2026-11-03: protocol reads as a composition of types each with one or a few consumers; attempt stays whole. Commit 1 moves RouterHintSourceMetadata, the cache-owner hint-source record hosts publish in runtime configuration and the KV indexer's residency snapshot resolves, whole with its serde test from lib/router/protocol/src/metadata.rs (deleted) to lib/router/kv-indexer/src/residency.rs, next to ResolvedRouterHintSource, which wraps it; it goes to kv-indexer and not kv-hints because kv-hints already depends on kv-indexer for the transfer candidate vocabulary, so the reverse would be a cycle; kv-indexer exports it at the root and in host, the six lib/llm and bindings importers go through dynamo_kv_indexer::host, the kernel tests and the bench through the crate; no manifest changes. Commit 2 moves WorkerAffinityTarget (struct, new, From<WorkerWithDpRank>) whole from lib/router/protocol/src/worker.rs to lib/router/session-affinity/src/lib.rs, where the alias line becomes the definition plus pub type AffinityTarget = WorkerAffinityTarget;; session affinity writes it, admission's classifier and worker selection's context read it, the kernel carries it, hosts set it; dynamo-admission and dynamo-worker-selection gain the dynamo-session-affinity edge the owner accepted (worker-selection's tree gains tokio, tokio-util, dashmap, blake3: grep -c tokio over cargo tree -e normal 4 -> 6), the prelude re-exports the type from session-affinity under the same name, and the gate rows and the three crates' docs state the new edges. Rule applied: a type with one writer and one or two readers lives with its writer, and a stage may depend on another stage for its output types; protocol keeps the counterparty wire and identity only (attempt, columns, identity, session, worker, worker_type). Replaces fork PR #195 (two commits).

What moves

Dependency edges

lib/router/admission/Cargo.toml and lib/router/worker-selection/Cargo.toml: dynamo-session-affinity = { workspace = true } added; the root, Python bindings, and KVBM bindings lockfiles gain the two entries each. scripts/router_layering_check.sh: dynamo-worker-selection direct set dynamo-router-protocol dynamo-session-affinity dynamo-slot-tracker and dynamo-admission direct set dynamo-router-protocol dynamo-session-affinity dynamo-slot-tracker dynamo-tokens, each dropping dynamo-session-affinity from its forbidden list; dynamo-session-affinity's forbidden list gains dynamo-worker-selection dynamo-admission. 40 checks plain, 59 with --host, 0 failures. Planted dynamo-worker-selection in session-affinity's [dependencies]: cargo rejects the cycle and the gate exits 2 with ERROR (fail closed); output in #195.

Validation

Unit-level move-diff from 0ebe1e5139 over the union (--from lib/router/protocol/src/metadata.rs lib/router/kv-indexer/src/residency.rs lib/router/protocol/src/worker.rs lib/router/session-affinity/src/lib.rs --to lib/router/kv-indexer/src/residency.rs lib/router/protocol/src/worker.rs lib/router/session-affinity/src/lib.rs): common=1442 new=1 dropped=0, the new line being the alias (artifact /tmp/rr-split/unit-prs/19-move-diff.txt on the build host). Every other .rs change is a use re-point (16 + 15 files, +64 -68 and +47 -36 including the moved text).

Chain v2, full mode, on the head 55f356872d (log /tmp/rr-split/s195-validation.log): stage 1, the 13 router crates (clippy under default and all features, cargo test 1,247 passed / 5 ignored with the moved serde test now counted under kv-indexer, rustdoc with all features), 1 min 52 s; stage 2, kv-router (clippy, bench check, 229 tests including the prelude test, unchanged), 1 min 04 s; stage 3, workspace clippy, 3 min 56 s; stage 4, hosts (dynamo-llm lib alone, its kv_router 492 / discovery 275 / pool_relay 261 tests, 13 host crates clippy, bindings clippy; the pre-existing kvbm-consolidator manifest failure is the one non-zero exit), 22 min 32 s; stage 5, gates (every kv-router feature alone and together, the layering gate 40 checks and --host 59 checks with 0 failures, codeowners 7586/7586, fmt, 3 lockfiles --locked), 2 min 15 s; stage 6, the two per-commit move-diffs; 31 min 39 s total, exit 0. Explicitly on each commit's tree before the chain: the moved serde test green in kv-indexer; cargo test -p dynamo-kv-router --test plugin_api_prelude 1 passed; cargo tree -p dynamo-worker-selection -e normal | grep -c tokio 4 at the base, 6 at the head.

The replaced fork PR carries the per-commit move-diffs and the planted-violation output; see its body.

Tickets

Part of DYNO-195

Owner decision 2026-11-03: protocol reads as a composition of types
each with one or a few consumers. RouterHintSourceMetadata is the
cache-owner hint-source record hosts publish in runtime configuration
and the KV indexer's residency snapshot resolves
(ResolvedRouterHintSource wraps it); no other stage names it. It moves
whole, with its serde test, from lib/router/protocol/src/metadata.rs
(deleted, with its module line) to
lib/router/kv-indexer/src/residency.rs, just before the struct that
wraps it. It belongs to the indexer and not to dynamo-kv-hints because
kv-hints depends on the indexer for the transfer candidate vocabulary;
the indexer naming a kv-hints type would close a cycle.

kv-indexer exports it at the crate root and in host (hosts publish
it). Re-pointed: lib/llm discovery/kv_state_agent.rs,
kv_router/publisher/{attachment_owner,state_agent,state_agent_host}.rs,
kv_router/indexer/recovery/state_agent.rs, and the Python bindings'
kv_state_agent.rs import dynamo_kv_indexer::host; the kernel's
kv_hints and selection core tests, the bench's mooncake_lower_tier,
and the indexer's own kv_hints and lower_tier tests import the
residency path or the crate root. No Cargo.toml change: every importer
already depends on dynamo-kv-indexer. Protocol's crate doc and README
list the type under dynamo-kv-indexer; its AGENTS.md module list drops
metadata. Gate rows unchanged (40 checks, 0 failures).

Move-diff from 0ebe1e5 over metadata.rs and residency.rs into
residency.rs: common 175, new 0, dropped 0, MOVE.

Signed-off-by: Ishan Dhanani <ishandhanani@gmail.com>
Owner decision 2026-11-03: protocol reads as a composition of types
each with one or a few consumers. WorkerAffinityTarget is the target a
session binds to: session affinity writes it (the table, the lease,
the replication wire's target()), admission's classifier and worker
selection's context read it as the request's requested target, the
kernel carries it between them, and hosts set it on requests. It moves
whole (struct, derives, doc, new, From<WorkerWithDpRank>) from
lib/router/protocol/src/worker.rs to
lib/router/session-affinity/src/lib.rs, where the alias line
`pub use ...::WorkerAffinityTarget as AffinityTarget` becomes the
definition plus `pub type AffinityTarget = WorkerAffinityTarget;`, so
both names keep resolving; the host module exports it too.
WorkerWithDpRank and its constructors stay in protocol.

New stage edges, accepted by the owner: dynamo-admission and
dynamo-worker-selection depend on dynamo-session-affinity (three
lockfiles updated). The cost is session-affinity's tree entering
worker-selection's: tokio, tokio-util, dashmap, and blake3
(`cargo tree -p dynamo-worker-selection -e normal | grep -c tokio`:
4 before, 6 after). The direction stays one way: session-affinity
names neither stage.

Re-pointed: worker-selection context.rs, admission classifier/mod.rs,
the kernel's filter.rs, types.rs, selection/{types,catalog}.rs,
selection/core/{mod,operation}.rs (run.rs and the core tests reach it
through super::*), selector/reference.rs; lib/llm kv_router.rs and
routing_host/kv_selection.rs import dynamo_session_affinity::host; the
plugins prelude re-exports it from dynamo_session_affinity under the
same name (plugin_api_prelude passes unchanged; the soft-pin-repin
example compiles through it).

Gate: dynamo-worker-selection's direct set becomes {protocol,
session-affinity, slot-tracker}, dynamo-admission's gains
session-affinity, each drops session-affinity from its forbidden list;
dynamo-session-affinity's forbidden list gains dynamo-worker-selection
and dynamo-admission. 40 checks plain, 59 with --host, 0 failures.
Planted dynamo-worker-selection in session-affinity's [dependencies]:
cargo rejects the cycle (session-affinity -> worker-selection ->
session-affinity) and the gate exits 2 with ERROR on its first cargo
tree, fail closed; reverted. lib/router/README.md and the three
AGENTS.md and README files state the new edges.

Move-diff from 2d8fa36 over worker.rs and session-affinity lib.rs:
common 1267, new 1 (the alias line), dropped 0.

Signed-off-by: Ishan Dhanani <ishandhanani@gmail.com>
@ishandhanani
ishandhanani force-pushed the idhanani/rs2-19-protocol-narrowing branch from 55f3568 to dc1537a Compare November 4, 2026 09:30
@ishandhanani

Copy link
Copy Markdown
Owner Author

Landed upstream as ai-dynamo#18055 (squash commit 79692cf on main, 2026-11-11). Closing this fork review unit.

This branch had an error being deployed

1 failed deployment
external_collaborator — dc1537af Deployed Nov 4, 2026 by ishandhanani via ok-to-test #537
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

documentation Improvements or additions to documentation refactor router

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant