Repository navigation
refactor(router): move RouterHintSourceMetadata to kv-indexer and WorkerAffinityTarget to session-affinity - #195
Closed
ishandhanani wants to merge 2 commits into
Conversation
ishandhanani
had a problem deploying
to
external_collaborator
November 3, 2026 04:53 — with
GitHub Actions
Failure
ishandhanani
had a problem deploying
to
external_collaborator
November 3, 2026 04:54 — with
GitHub Actions
Failure
Owner decision 2026-11-03: protocol reads as a composition of types
each with one or a few consumers. RouterHintSourceMetadata is the
cache-owner hint-source record hosts publish in runtime configuration
and the KV indexer's residency snapshot resolves
(ResolvedRouterHintSource wraps it); no other stage names it. It moves
whole, with its serde test, from lib/router/protocol/src/metadata.rs
(deleted, with its module line) to
lib/router/kv-indexer/src/residency.rs, just before the struct that
wraps it. It belongs to the indexer and not to dynamo-kv-hints because
kv-hints depends on the indexer for the transfer candidate vocabulary;
the indexer naming a kv-hints type would close a cycle.
kv-indexer exports it at the crate root and in host (hosts publish
it). Re-pointed: lib/llm discovery/kv_state_agent.rs,
kv_router/publisher/{attachment_owner,state_agent,state_agent_host}.rs,
kv_router/indexer/recovery/state_agent.rs, and the Python bindings'
kv_state_agent.rs import dynamo_kv_indexer::host; the kernel's
kv_hints and selection core tests, the bench's mooncake_lower_tier,
and the indexer's own kv_hints and lower_tier tests import the
residency path or the crate root. No Cargo.toml change: every importer
already depends on dynamo-kv-indexer. Protocol's crate doc and README
list the type under dynamo-kv-indexer; its AGENTS.md module list drops
metadata. Gate rows unchanged (40 checks, 0 failures).
Move-diff from 0ebe1e5 over metadata.rs and residency.rs into
residency.rs: common 175, new 0, dropped 0, MOVE.
Signed-off-by: Ishan Dhanani <ishandhanani@gmail.com>
Owner decision 2026-11-03: protocol reads as a composition of types
each with one or a few consumers. WorkerAffinityTarget is the target a
session binds to: session affinity writes it (the table, the lease,
the replication wire's target()), admission's classifier and worker
selection's context read it as the request's requested target, the
kernel carries it between them, and hosts set it on requests. It moves
whole (struct, derives, doc, new, From<WorkerWithDpRank>) from
lib/router/protocol/src/worker.rs to
lib/router/session-affinity/src/lib.rs, where the alias line
`pub use ...::WorkerAffinityTarget as AffinityTarget` becomes the
definition plus `pub type AffinityTarget = WorkerAffinityTarget;`, so
both names keep resolving; the host module exports it too.
WorkerWithDpRank and its constructors stay in protocol.
New stage edges, accepted by the owner: dynamo-admission and
dynamo-worker-selection depend on dynamo-session-affinity (three
lockfiles updated). The cost is session-affinity's tree entering
worker-selection's: tokio, tokio-util, dashmap, and blake3
(`cargo tree -p dynamo-worker-selection -e normal | grep -c tokio`:
4 before, 6 after). The direction stays one way: session-affinity
names neither stage.
Re-pointed: worker-selection context.rs, admission classifier/mod.rs,
the kernel's filter.rs, types.rs, selection/{types,catalog}.rs,
selection/core/{mod,operation}.rs (run.rs and the core tests reach it
through super::*), selector/reference.rs; lib/llm kv_router.rs and
routing_host/kv_selection.rs import dynamo_session_affinity::host; the
plugins prelude re-exports it from dynamo_session_affinity under the
same name (plugin_api_prelude passes unchanged; the soft-pin-repin
example compiles through it).
Gate: dynamo-worker-selection's direct set becomes {protocol,
session-affinity, slot-tracker}, dynamo-admission's gains
session-affinity, each drops session-affinity from its forbidden list;
dynamo-session-affinity's forbidden list gains dynamo-worker-selection
and dynamo-admission. 40 checks plain, 59 with --host, 0 failures.
Planted dynamo-worker-selection in session-affinity's [dependencies]:
cargo rejects the cycle (session-affinity -> worker-selection ->
session-affinity) and the gate exits 2 with ERROR on its first cargo
tree, fail closed; reverted. lib/router/README.md and the three
AGENTS.md and README files state the new edges.
Move-diff from 2d8fa36 over worker.rs and session-affinity lib.rs:
common 1267, new 1 (the alias line), dropped 0.
Signed-off-by: Ishan Dhanani <ishandhanani@gmail.com>
ishandhanani
force-pushed
the
idhanani/dyn-195-metadata-and-affinity-target
branch
from
November 4, 2026 09:30
55f3568 to
dc1537a
Compare
ishandhanani
had a problem deploying
to
external_collaborator
November 4, 2026 09:30 — with
GitHub Actions
Failure
ishandhanani
had a problem deploying
to
external_collaborator
November 4, 2026 09:30 — with
GitHub Actions
Failure
Owner
Author
|
Landed upstream as ai-dynamo#18055 (squash commit 79692cf on main, 2026-11-11). Closing this fork review unit. |
This branch had an error being deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Router split v2 follow-up (DYNO-195). Stacked on #193.
Summary
Two move-only commits, each with its own move-diff; no behavior change. Owner decisions (2026-11-03): protocol should read as a composition of types each with one or a few consumers;
attemptstays whole (its minter is the only writer of the private id field, and the id is named by two stages that must not depend on each other); the new stage edges admission -> session-affinity and worker-selection -> session-affinity are accepted, including tokio, tokio-util, dashmap, and blake3 entering worker-selection's tree.refactor(router): move RouterHintSourceMetadata to kv-indexer(2d8fa36). The struct, its derives and docs, and its serde test move whole fromlib/router/protocol/src/metadata.rs(deleted, with itspub mod metadata;line) tolib/router/kv-indexer/src/residency.rs, just beforeResolvedRouterHintSource, which wraps it. Who writes it: hosts, when they publish a cache owner's hint source in runtime configuration (lib/llmdiscovery/kv_state_agent.rs, the state-agent publishers and the attachment owner, the Python bindings' state-agent config). Who reads it: the indexer's residency snapshot, which resolves it intoResolvedRouterHintSourceand the transfer keys the kernel hands worker selection; no other stage names it. Why kv-indexer and not kv-hints: kv-hints depends on kv-indexer for the transfer candidate vocabulary (DYNO-191) andresidency.rsembeds the metadata, so a kv-hints home would make the indexer name a kv-hints type and close a cycle. kv-indexer exports it at the crate root and inhost(the host convention);kv_hints.rsimportscrate::residency::RouterHintSourceMetadata; the crate's AGENTS.md "owns" sentence names it. Re-pointed, compiler-driven: lib/llmdiscovery/kv_state_agent.rs,kv_router/publisher/{attachment_owner,state_agent,state_agent_host}.rs,kv_router/indexer/recovery/state_agent.rs, andlib/bindings/python/rust/llm/kv_state_agent.rsthroughdynamo_kv_indexer::host; the kernel'skv_hints.rsandselection/core/tests.rs(tests) through theresidencypath; the bench'smooncake_lower_tier.rsthrough the crate root; the indexer's ownlower_tier.rstests throughcrate::residency. No Cargo.toml change: every importer already depends ondynamo-kv-indexer. Protocol's crate doc lists the type under "Deliberately elsewhere"; the README's "deliberately not here" table names it in the kv-indexer row (the README has no module table, so no row is removed); protocol's AGENTS.md module list dropsmetadata. Gate rows unchanged. 19 files, +67 -71 (16.rsfiles, +64 -68).refactor(router): move WorkerAffinityTarget to session-affinity(55f3568).pub struct WorkerAffinityTargetwith its derives and doc,impl WorkerAffinityTarget { new }, andimpl From<WorkerWithDpRank> for WorkerAffinityTargetmove whole fromlib/router/protocol/src/worker.rstolib/router/session-affinity/src/lib.rs, where the alias linepub use dynamo_router_protocol::worker::WorkerAffinityTarget as AffinityTarget;becomes the definition pluspub type AffinityTarget = WorkerAffinityTarget;, so both names keep resolving; the crate'shostmodule exports it (hosts set it on requests).WorkerWithDpRankand its constructors stay in protocol. Who writes it: session affinity (the binding table,AffinityLease::target,wire::AffinityBindingEvent::target(),rebound_target) and hosts (lib/llm sets the request'saffinity_targetfrom its session coordinator). Who reads it: admission's classifier (the classify request's affinity target) and worker selection's context (WorkerSelectionContext, the soft-pin filters), with the kernel carrying it between them (filter.rs,selection/*,types.rs). Edge cost, stated plainly:lib/router/admission/Cargo.tomlandlib/router/worker-selection/Cargo.tomlgaindynamo-session-affinity = { workspace = true }(the root, Python bindings, and KVBM bindings lockfiles each gain the two entries), so session-affinity's tree enters worker-selection's:cargo tree -p dynamo-worker-selection -e normal | grep -c tokioreads 4 before and 6 after (tokio and tokio-util; dashmap and blake3 arrive with them). The direction stays one way: session-affinity names neither stage, and a reverse edge is now a cargo cycle. Re-pointed, compiler-driven: worker-selectioncontext.rs; admissionclassifier/mod.rs; the kernel'sfilter.rs,types.rs,selection/{types,catalog}.rs,selection/core/{mod,operation}.rs(run.rsand the core tests reach it throughsuper::*, so they need no edit),selector/reference.rs(test); lib/llmkv_router.rsandrouting_host/kv_selection.rsthroughdynamo_session_affinity::host;plugins::preludere-exports it fromdynamo_session_affinityunder the same name (plugin_api_preludepasses unchanged; the soft-pin-repin example compiles through the prelude).lib/router/README.mdand the three AGENTS.md and README files state the new edges. 28 files, +71 -52 (15.rsfiles, +47 -36).Move-diffs
Commit 1,
python3 -I scripts/router_move_diff.py 0ebe1e5139 WORKTREE --from lib/router/protocol/src/metadata.rs lib/router/kv-indexer/src/residency.rs --to lib/router/kv-indexer/src/residency.rs:Commit 2,
python3 -I scripts/router_move_diff.py 2d8fa3618e WORKTREE --from lib/router/protocol/src/worker.rs lib/router/session-affinity/src/lib.rs --to lib/router/protocol/src/worker.rs lib/router/session-affinity/src/lib.rs:The one new line is the alias the brief asked for; the old
pub use ... as AffinityTarget;is auseline and normalization drops it on both sides. Over the union from0ebe1e5139(both--fromsets into both--tosets):common=1442 new=1 dropped=0, the same line.Gate rows and the planted violation
Rows changed in
scripts/router_layering_check.sh(exact direct Dynamo set | forbidden anywhere in the--all-featuresclosure):dynamo-session-affinity: directdynamo-router-protocol(unchanged); forbidden gainsdynamo-worker-selection dynamo-admission(the direction).dynamo-worker-selection: directdynamo-router-protocol dynamo-session-affinity dynamo-slot-tracker(was without session-affinity); forbidden dropsdynamo-session-affinity.dynamo-admission: directdynamo-router-protocol dynamo-session-affinity dynamo-slot-tracker dynamo-tokens; forbidden dropsdynamo-session-affinity.40 checks plain, 59 with
--host, 0 failures (the counts #193 left).Planted
dynamo-worker-selection = { workspace = true }inlib/router/session-affinity/Cargo.toml, alone, then reverted (git statusclean andcargo metadata --lockedpassing afterwards):With the forward edges in place, session-affinity -> worker-selection (or -> admission) is a cycle by construction: cargo refuses the graph before the gate's table check can run, the gate's first
cargo treefails, and the gate exits 2 (ERROR, fail closed) rather than printing aFAILrow. The forbidden entries document the direction for the day a forward edge goes away.Validation
Validation chain v2 in
MODE=full(mandatory: two stage Cargo.toml edges change),TAG=195 LLM_EXTRA_TESTS="discovery pool_relay"with both move specs inMOVE_SPECS_FILE, run once on the final head55f356872dafter commit 2 (log/tmp/rr-split/s195-validation.logon the build host). Every stage exit 0 except the pre-existing kvbm-consolidator manifest failure (unrelated, untouched); chain exit 0.target23 GB after the run.cargo clippy --all-targets -- -D warnings(default and--all-features);cargo test --all-features: admission 22, error 29, kv-events 64, kv-hashing 19 + 17 + 3, kv-hints 4, kv-indexer 600 + 2 ignored (599 before: the moved serde test), multimodal 4, protocol 8 (9 before), scheduler 289 + 1 ignored, session-affinity 10, slot-tracker 131, worker-selection 43, zmq 2, doc-tests 2 + 2 ignored (1,247 passed, 5 ignored, the same total as refactor(router): plugins use the prelude; drop the API snapshots; slim the gate and AGENTS files #193);RUSTDOCFLAGS="-D warnings" cargo doc --no-deps --all-features.cargo clippy -p dynamo-kv-router --features http,ingest-zmq,replica-sync-zmq,peer-recovery,metrics,testing --all-targets -- -D warnings;cargo check --features bench --benches; tests 220 + 1 + 7;plugin_api_prelude1 passed, the test unchanged.cargo clippy --workspace --all-targets -- -D warningsclean.cargo check -p dynamo-llm --libalone;cargo test -p dynamo-llm --libforkv_router(492),discovery(275),pool_relay(261); per-hostcargo clippy -p <host> --tests -- -D warningsexit 0 for dynamo-mocker, dynamo-bench, dynamo-ext-proc, dynamo-custom-policy-builtin, dynamo-worker-selection-policy-catalog, dynamo-sglang-sidecar, dynamo-vllm-sidecar, dynamo-backend-common, dynamo-sglang-mocker, dynamo-vllm-mocker, dynamo-custom-policy-example-epp, dynamo-custom-policy-example-soft-pin-repin, libdynamo_llm; kvbm-consolidator exit 101 (manifest, pre-existing);lib/bindings/pythoncargo clippy --features kv-indexer,slot-tracker,select-service -- -D warningsexit 0.scripts/router_features_check.sh(every kv-router feature alone and together);scripts/router_layering_check.sh40 checks, 0 failures and--host59 checks, 0 failures;python3 .github/codeowners/build_codeowners.py --strict7586/7586 explicitly owned (7587 before:metadata.rsis gone);cargo fmt --all -- --check;cargo metadata --lockedfor the root,lib/bindings/python, andlib/bindings/kvbmlockfiles.common=175 new=0 dropped=0MOVE; move-diff 2common=1267 new=1 dropped=0(the alias line), as above.s195-c1-check.log):cargo check --testsfor protocol, kv-indexer, scheduler, bench;cargo check -p dynamo-llm --tests; bindings check; the moved serde test green in kv-indexer and protocol down to 8 tests; gate 40/0; move-diff MOVE. Commit 2 (logs195-c2-check.log):cargo check --all-features --testsfor the four stage crates, protocol, and the kernel; kv-router with the chain's feature set, the mocker, builtin, the soft-pin-repin and EPP examples, the bench;plugin_api_prelude1; session-affinity 10 and protocol 8 tests;cargo check -p dynamo-llm --tests; bindings check; gate 40/0 and--host59/0; fmt; three lockfiles--locked;cargo tree -p dynamo-worker-selection -e normal | grep -c tokio6 (4 at the base).target23 GB; 293 GB free.