Repository navigation
refactor(router): protocol cleanup: load, hints, residency, candidates, multimodal, and ActiveLoad to their owners - #189
Closed
ishandhanani wants to merge 7 commits into
Conversation
…he slot tracker Dissolve `dynamo_router_protocol::load`. The protocol crate keeps only what a worker publishes about itself and what every crate names; a type with one producing stage now lives in that stage, and a stage may depend on another stage for that stage's output types instead of hoisting them into protocol. Moves: - `RankLoad` and `BusyThresholds` (with `is_configured` and `has_room`) move to `dynamo_slot_tracker::busy`, replacing the re-export that was there. The fullness rule is the slot tracker's output: its projection implements `RankLoad`, and `FullRankCounts` applies the thresholds. - `SchedulerLoad` and `SchedulerGroup` move to `dynamo_slot_tracker::wire` with their wire test. One tracker publishes the scheduler-load report about the ranks it books; it is the tracker's replica wire, not a worker-published fact. - `ActiveLoad`, `WorkerCapacity`, and `DEFAULT_MAX_BATCHED_TOKENS` stay in protocol and move to `dynamo_router_protocol::worker`, beside the worker identities. The constant stays with `WorkerCapacity::prefill_budget`, which reads it; protocol cannot depend on the slot tracker, so the ticket's placement of the constant under slot-tracker does not hold. New edges: `dynamo-admission` and `dynamo-worker-selection` now depend on `dynamo-slot-tracker` for the fullness rule. The layering gate rows for both crates name the edge (router deps become protocol plus slot-tracker; slot-tracker leaves the forbidden list and joins the direct deps). Hosts import `BusyThresholds`, `RankLoad`, `SchedulerLoad`, and `SchedulerGroup` through `dynamo_slot_tracker::host`; the kv-router and scheduler `protocols` shims keep every existing public name and re-point. Protocol drops its `rmp-serde` dev-dependency, which served only the moved test; the slot tracker gains `rmp-serde` and `serde_json` as dev-dependencies. Crate docs for protocol, slot-tracker, admission, and worker-selection name the new owner and the new edges. No behavior change. Validation: Stage crates (dynamo-router-protocol, dynamo-slot-tracker, dynamo-admission, dynamo-worker-selection, dynamo-scheduler): clippy --all-targets -D warnings with default and --all-features, tests (11, 131, 22, 44, 291 passed), rustdoc -D warnings. dynamo-kv-router: clippy with http,ingest-zmq,replica-sync-zmq,peer-recovery,metrics, testing --all-targets, bench check, 229 tests passed. Workspace clippy --all-targets -D warnings. Hosts: dynamo-llm check, clippy, 485 kv_router and 261 pool_relay tests; clippy --tests for dynamo-mocker, dynamo-bench, dynamo-ext-proc, dynamo-custom-policy-builtin, dynamo-sglang-sidecar, dynamo-sglang-mocker, dynamo-vllm-mocker; the Python bindings with kv-indexer,slot-tracker,select-service. Gates: router_features_check (15/15), router_public_api_check (protocol snapshot regenerated, kv-router snapshot changes only the path of `BusyThresholds` in its `BusyThresholdsExt` impl, dynamo-error unchanged), router_layering_check 41 checks and --host 60 checks with 0 failures, codeowners 7542/7542, cargo fmt --check, the three lockfiles under --locked. router_move_diff from 11d495f over load.rs, busy.rs, wire.rs, worker.rs: 296 normalized lines on each side, new 0, dropped 0, result MOVE. Signed-off-by: Ishan Dhanani <ishandhanani@gmail.com>
…ines KV transfer hints are what the router hands an engine so the selected worker can fetch a KV prefix from another worker or cache owner. They are the counterpart of `dynamo-kv-events` (what engines publish to the router), and engines consume them: backend-common, the SGLang and vLLM sidecars, and the Python engine components. Until now the feature was spread over four crates (protocol, kv-indexer, worker-selection, the scheduler kernel) and engines reached the wire through lib/llm. The new tier-1 crate `lib/kv-hints` (`dynamo-kv-hints`) gives it one owner, one AGENTS.md, one public-API snapshot, and a crate engine-side code can depend on directly later. Among router crates it depends on `dynamo-router-protocol` only; its direct dependencies are exactly protocol, rustc-hash, serde, and serde_json. Moves (git mv where a whole file moved): - `lib/router/protocol/src/kv_hints.rs` -> `lib/kv-hints/src/wire.rs` whole: `KvHint`, `KvHintAction`, `KvSourceLocationsPayload`, the three `KV_HINT_TRANSFER_*` runtime keys, `KvTransferCandidateSource`. The two wire serde tests move with the types from the kernel. Protocol loses the module and with it its only reference to `ResidencyOwnerKey` outside `residency`. - `lib/router/worker-selection/src/kv_transfer.rs` -> `lib/kv-hints/src/transfer.rs` whole: `WorkerKvTransferKeys`, `KvTransferTarget`, `KvTransferScope`, `ResolvedTransferSource`, `KvTransferSourceChoice`, `KvTransferSourceSummary`. - From `dynamo_kv_indexer::kv_hints`: `label_key` and `topology_keys` (pure hashing) -> `lib/kv-hints/src/keys.rs`; the indexer re-exports them at their old path. - From `dynamo_scheduler::kv_hints`: `WorkerKvTransferKeysCache` and the body of `WorkerKvTransferKeysExt::of`, now `WorkerKvTransferKeys::from_worker(start_rank, dp_size, per_rank_metadata, topology, transfer_domain, enforcement)` in `keys`. - From `dynamo_scheduler::protocols`: `KvHintTransferWorkerMetadata` and `KvTransferEnforcement` with the enforcement serde test -> `keys`. Stays, and why: - `KvTransferCandidates` and `CacheOwnerKvTransferKeys` stay in `dynamo-kv-indexer`: they carry `Arc<ResidencyRoutingSnapshot>`, the indexer's projection. The producer owns its output; moving them would make the indexer and the new crate name each other. - `resolve_transfer_sources` and its tests stay in the kernel: it joins the indexer's candidates with worker keys under a selection scope, which is the kernel's stage-to-stage conversion. The attach point in `selection/core/run.rs` is untouched. - `WorkerKvTransferKeysExt` stays in the kernel as a thin adapter: `of` reads the fields off `WorkerConfigLike` and calls `from_worker`. One new kernel test asserts the adapter equals the direct call for a config with two ranks, one source endpoint, a topology map, a transfer domain, and `Preferred` enforcement. Signature changes (no behavior change): the key builder takes extracted fields instead of a `WorkerConfigLike`, and `WorkerKvTransferKeysCache::get_or_init` takes the initializer closure (`|| WorkerKvTransferKeys::of(self)`) instead of the config, because the cache no longer lives beside the config trait. Its two callers (`SelectionWorkerConfig` and the builtin plugin's selection bench) are updated. New edges: kv-indexer, scheduler, kv-router, and lib/llm depend on `dynamo-kv-hints`. worker-selection does not: nothing in it consumed the moved types, so the scheduler kernel and the prelude re-point to the new crate instead. The layering gate gains a row for `dynamo-kv-hints` (router deps exactly protocol; every stage, the kernel, kv-events, kv-hashing, kv-router, runtime, tokio, and the transport crates forbidden; direct deps exact) and `dynamo-kv-hints` joins `ROUTER_FAMILY`, `ROUTER_CRATE_DIRS`, the composition crate's router and direct dependency sets, and the kv-indexer and scheduler rows. `scripts/router_public_api_check.sh` gains the fourth snapshot `lib/kv-hints/public-api.txt`; the pre-commit trigger and the router codeowners area claim `lib/kv-hints/`. Public API: every existing path resolves. `dynamo_scheduler::kv_hints`, `dynamo_scheduler::protocols`, and `dynamo_scheduler::host` re-export the moved items (host also exposes `WorkerKvTransferKeysExt` beside the cache, which the closure form of `get_or_init` needs); the kv-router `kv_hints` and `protocols` shims and `plugins::prelude` re-point with every name unchanged, so `lib/kv-router/public-api.txt` is byte-for-byte identical. `lib/router/protocol/public-api.txt` loses its 28 `kv_hints::` items; the new snapshot has 142 items. Hosts in lib/llm import the wire and keys from `dynamo_kv_hints` directly; backend-common, the sidecars, the bindings, and the mocker keep their current paths through lib/llm and the kernel. `label_key` hashes with `DefaultHasher::new()`: deterministic within one toolchain, not a cross-version contract. Both sides of every comparison are computed in one process today; the crate doc records it as a follow-up if keys ever cross a process boundary. Validation: dynamo-kv-hints: clippy --all-targets -D warnings (default and --all-features), 4 tests, rustdoc -D warnings. Stage crates (dynamo-router-protocol, dynamo-kv-indexer, dynamo-worker-selection, dynamo-scheduler): clippy --all-targets -D warnings with default and --all-features, tests with --all-features (11, 597, 43, 289 passed; worker-selection 44 -> 43 and scheduler 291 -> 289 are the moved tests plus the one new adapter test), rustdoc -D warnings (default and --all-features). dynamo-kv-router: clippy with http,ingest-zmq, replica-sync-zmq,peer-recovery,metrics,testing --all-targets, bench check, 220 + 1 + 1 + 7 tests, the plugin_api_prelude and plugin_api_compat tests. Workspace clippy --all-targets -D warnings. Hosts: dynamo-llm check, clippy, 485 kv_router and 38 local_model tests; clippy --tests for dynamo-mocker, dynamo-bench, dynamo-ext-proc, dynamo-custom-policy-builtin, dynamo-worker-selection-policy-catalog, dynamo-sglang-sidecar, dynamo-vllm-sidecar, dynamo-backend-common, dynamo-sglang-mocker, dynamo-vllm-mocker, dynamo-custom-policy-example-epp, dynamo-custom-policy-example-soft-pin-repin; the Python bindings with kv-indexer,slot-tracker,select-service. Gates: router_features_check, router_public_api_check (kv-router snapshot identical, protocol snapshot -28 kv_hints items, new kv-hints snapshot 142 items, dynamo-error unchanged at 134), router_layering_check 44 checks and --host 63 checks with 0 failures, codeowners 7541/7541, cargo fmt --check, the three lockfiles under --locked. router_move_diff from 4ff5815 over protocol kv_hints.rs, worker-selection kv_transfer.rs, kv-indexer kv_hints.rs, and the kernel's kv_hints.rs and protocols.rs to the new crate and the two kernel files: common 1138, new 91, dropped 11. The dropped lines are the old adapter body that read the config and the config-taking get_or_init; the new lines are from_worker, the six-argument adapter call, the closure-taking get_or_init, and the new kernel test. Signed-off-by: Ishan Dhanani <ishandhanani@gmail.com>
The producer owns the type. `StorageTier` is a field of the KV event
payload engines publish (`RouterEvent::storage_tier`), the residency
domain rides the same wire (`residency_domain`), and the kvbm
consolidator publishes residency too. The owner is the KV-events crate,
not protocol and not a router stage: protocol keeps the counterparty
nouns every crate names (identity, worker, block_hash, mm, session,
attempt, columns, metadata, worker_type). After DYNO-185 removed
`kv_hints.rs`, no protocol module referenced `residency` any more, so
the module moves whole.
Move (git mv, body unchanged): `lib/router/protocol/src/residency.rs`
-> `lib/kv-events/src/residency.rs`: `StorageTier`, `ResidencyDomain`,
`ResidencyOwner`, `ResidencyOwnerKey`, `ResetScope`,
`WireResidencyDomain` with its manual serde impls, and
`UnsupportedResidencyDomain`. Its two `crate::` imports become
`dynamo_router_protocol::{identity, worker}`. The file carries no test
module of its own; the residency-domain compatibility tests already
live beside the event types in `lib/kv-events/src/event.rs`.
`dynamo-kv-events` gains `pub mod residency` and its crate doc, README,
and AGENTS name the vocabulary; protocol drops the module, and its
crate doc and README list the vocabulary under `dynamo-kv-events`.
Re-pointed, compiler-driven, 46 files. kv-events' own files
(`event.rs`, `placement.rs`, `zmq_wire/{mod,convert,tests}.rs`) use
`crate::residency`. Everyone else imports
`dynamo_kv_events::residency` directly (vocabulary crate, no host
module): lib/llm (16 files: the publisher, indexer, recovery, and
kv_consolidator modules, `mocker.rs`, the `kv_router.rs` test),
lib/mocker (10), lib/bench (4), kvbm-consolidator
(`ingress/zmq_subscriber.rs`), kv-router
(`services/indexer/registry.rs`, `services/selection/tests.rs`,
`benches/selection_core.rs`, `tests/standalone_indexer_http.rs`),
kv-indexer (`prelude.rs`, `residency.rs`, `cuckoo/dc.rs`), and
`lib/kv-hints/src/wire.rs`. The deprecated shims
`dynamo_kv_router::protocols` and `dynamo_scheduler::protocols`
re-point their `pub use` with every name unchanged.
One new edge the brief did not anticipate: `dynamo-kv-hints` names
`ResidencyOwnerKey` (`KvTransferCandidateSource::CacheOwner`), so it
now depends on `dynamo-kv-events`. The crate was documented as
depending on protocol only among router crates; its crate doc, README,
and AGENTS now say protocol and kv-events, and its layering gate row
changes with it: router deps exactly protocol, kv-events, kv-hashing;
direct deps gain kv-events; kv-events and kv-hashing leave its
forbidden list. No cycle: kv-events does not depend on kv-hints. The
root, Python-bindings, and kvbm-bindings lockfiles each gain the one
dependency line. No other gate row changes.
Snapshots: `scripts/router_public_api_check.sh` gains the fifth
snapshot `lib/kv-events/public-api.txt` (261 items: 55 `event::`, 127
`zmq_wire::`, 23 `placement::`, 18 `cursor::`, 53 `residency::`, plus
root impls). `lib/router/protocol/public-api.txt` loses its 42
`residency::` lines (36 items and 6 impls) and gains none, 293 -> 251.
`lib/kv-hints/public-api.txt` changes two defining-path lines, both
`CacheOwner(dynamo_kv_events::residency::ResidencyOwnerKey)`.
`lib/kv-router/public-api.txt` (1398) and `lib/error/public-api.txt`
(134) are byte-for-byte identical. `lib/router/README.md` and the
kv-router README deprecated-paths table name the new owner.
Validation:
dynamo-kv-events: clippy --all-targets -D warnings (default and
--all-features), 65 tests, rustdoc -D warnings (default and
--all-features). Stage crates dynamo-router-protocol, dynamo-kv-hints,
dynamo-kv-indexer, dynamo-scheduler: clippy --all-targets -D warnings
with default and --all-features, tests with --all-features (11, 4, 597 +
2 ignored, 289 + 1 ignored passed), rustdoc -D warnings (default and
--all-features). dynamo-kv-router: clippy with http,ingest-zmq,
replica-sync-zmq,peer-recovery,metrics,testing --all-targets, bench
check, 220 + 1 + 1 + 7 tests, the plugin_api_prelude and
plugin_api_compat tests. Workspace clippy --all-targets -D warnings.
Hosts: dynamo-llm check, clippy, 485 kv_router and 21
block_manager::kv_consolidator tests; clippy --tests for dynamo-mocker,
dynamo-bench, dynamo-ext-proc, dynamo-custom-policy-builtin,
dynamo-worker-selection-policy-catalog, dynamo-sglang-sidecar,
dynamo-vllm-sidecar, dynamo-backend-common, dynamo-sglang-mocker,
dynamo-vllm-mocker, dynamo-custom-policy-example-epp,
dynamo-custom-policy-example-soft-pin-repin; the Python bindings with
kv-indexer,slot-tracker,select-service. kvbm-consolidator does not build
in this workspace for a pre-existing reason unrelated to this change
(its manifest inherits `kvbm-logical` from a root workspace that does
not define it); its one import is re-pointed. Gates:
router_features_check, router_public_api_check (five snapshots match),
router_layering_check 44 checks and --host 63 checks with 0 failures,
codeowners 7548/7548, cargo fmt --check, the three lockfiles under
--locked. router_move_diff from e21776c over
`lib/router/protocol/src/residency.rs` to
`lib/kv-events/src/residency.rs`: 190 normalized lines on each side,
common 190, new 0, dropped 0, result MOVE.
Signed-off-by: Ishan Dhanani <ishandhanani@gmail.com>
KvTransferCandidateSource (who holds a prefix a target could fetch), label_key and topology_keys (the label hashing both sides of a source match use), and ResidencyOwnerKey (the compact owner identity the indexer computes in compact_key) are what the indexer emits with a match. #165 placed the first three in dynamo-kv-hints and #166 the key in dynamo-kv-events, which left the indexer depending on the feature crate for its own output types. Rule: a feature crate depends on the stage that produces its types, never the reverse; the producer owns its outputs. Moves, text verbatim (move-diff over the five files from e844c15: 573/573 common, new 0, dropped 0): KvTransferCandidateSource and its From<WorkerWithDpRank> from lib/kv-hints/src/wire.rs and label_key and topology_keys from lib/kv-hints/src/keys.rs into lib/router/kv-indexer/src/kv_hints.rs beside KvTransferCandidates; ResidencyOwnerKey from lib/kv-events/src/residency.rs into lib/router/kv-indexer/src/residency.rs just before ResidencyOwnerKeyExt. kv-hints gains dynamo-kv-indexer, drops dynamo-kv-events (nothing else in it used that crate), and re-exports the three names at every path it exposed (root, keys::, wire::), so every existing consumer path resolves. kv-indexer drops dynamo-kv-hints. kv-events drops the key it never used; ResidencyOwner is untouched. Protocol gains nothing. Edges: kv-indexer -> protocol, kv-events, kv-hashing (was also kv-hints); kv-hints -> protocol, kv-indexer (was protocol, kv-events). Gate rows for both crates updated, including kv-hints' forbidden list, which can no longer name tokio, anyhow, or async-trait now that they arrive through the indexer; scheduler and kv-events rows and COMPOSITION_ROUTER_DEPS unchanged. Re-pointed: the kernel and kv-router protocols shims take ResidencyOwnerKey from dynamo_kv_indexer::residency (same public name); the plugin prelude takes KvTransferCandidateSource from the indexer beside KvTransferCandidates. Three lockfiles follow the manifests. Crate docs, AGENTS, and READMEs for kv-indexer, kv-hints, and kv-events, the lib/router and protocol README tables, name the new owner. Snapshots: kv-events 261 -> 257 (the key, its impl, digest, from_digest); kv-hints 142 -> 134 (the three re-exported names render as pub use lines at root, keys::, and wire:: in place of their inlined definitions, and the two source fields at both paths name the indexer's defining path); kv-router, protocol, and dynamo-error byte-identical. No behavior change. DYNO-191. Validation: clippy (default and --all-features, --all-targets, -D warnings), cargo test --all-features, and rustdoc -D warnings for kv-events, kv-hints, kv-indexer, scheduler, and protocol; kv-router clippy with all features, bench check, tests, and prelude tests; workspace clippy; hosts alone (lib/llm check, clippy, kv_router tests; twelve host crates clippy --tests; Python bindings clippy); router_features_check, router_public_api_check, router_layering_check plain and --host, codeowners --strict, cargo fmt --check, and the three lockfiles --locked. The chain ran on top of fork PR #168 (e048753); after #168 was closed the commit was rebased onto #166's head e844c15 and re-verified there: workspace clippy, the five crates' tests --all-features, the gate --host, the five snapshots, fmt --check, and the three lockfiles --locked. Signed-off-by: Ishan Dhanani <ishandhanani@gmail.com>
LocalBlockHash and ExternalSequenceBlockHash are what the hash contract produces: compute_block_hash and compute_block_hash_for_seq return the first, and the second is the engine-side identity the router recomputes with this crate's rules so the two sides agree. Protocol is counterparty wire and identity only, and the producer owns its outputs, so the two newtypes move to dynamo-kv-hashing. Move, text verbatim (git mv lib/router/protocol/src/block_hash.rs to lib/kv-hashing/src/block_hash.rs; move-diff from 94513b3 over the file and the kv-hashing root: 67/67 common, new 0, dropped 0; the mod line and the root re-export are import lines the tool normalizes away): both structs, From<u64> and From<i64>, the four manual serde impls, and the two serialization tests. kv-hashing re-exports both names at the root; routing.rs imports LocalBlockHash from the crate and its doc link names crate::ExternalSequenceBlockHash. Protocol drops the module; its crate doc and README table point at kv-hashing. serde_json stays in protocol's dependencies: the metadata.rs and worker_type.rs tests still use it, so the documented exact set (serde, serde_json, thiserror) is unchanged. Deprecated shims: lib/kv-router/src/protocols.rs and lib/router/scheduler/src/protocols.rs drop ExternalSequenceBlockHash and LocalBlockHash (owner decision: no shim for the block hash names). dynamo_kv_router::LocalBlockHash stays at the crate root, deprecated as before, now sourced from dynamo_kv_hashing. The plugins prelude (lib/kv-router/src/plugins/prelude.rs) is the tier-1 plugin API, not a shim: it keeps ExternalSequenceBlockHash, re-pointed; the kv-indexer prelude likewise, so the builtin plugin tests and bench compile unchanged through the prelude. Re-pointed, compiler-driven: 57 files from dynamo_router_protocol::block_hash:: to dynamo_kv_hashing:: (lib/llm 20, kv-indexer 10 incl. prelude and bench, bench 9, kv-router 7 incl. the prelude and the standalone indexer test, mocker 5, kv-events 3, kv-hints wire.rs, the C and Python bindings), plus the kernel's own twelve files that took the names through crate::protocols (kv_hints, local, overlap_refresh, queue, selection/core/{mod,operation,tests}, selection/{input,pending}, selector/policy_tests, tracking_hash, types) and kv-router's core_host_tests, which import from dynamo_kv_hashing directly. Edges: kv-hints gains dynamo-kv-hashing (its wire names ExternalSequenceBlockHash; the crate was already in its closure through the indexer); every other importer already depended on kv-hashing. Gate: the kv-hints row's direct set gains dynamo-kv-hashing; no other row changes (kv-hashing keeps protocol for the mm types until the next commit). Three lockfiles: the kv-hints block gains dynamo-kv-hashing. Docs: kv-hashing crate doc, AGENTS, and README (the LocalBlockHash home path), protocol crate doc and README table, kv-events AGENTS and README, kv-hints crate doc, AGENTS, and README, and the lib/router README rows name the new owner. Snapshots: protocol 251 -> 236 (the block_hash module, both structs, From<u64>, From<i64>, the four serde impls and their fns); kv-router 1398 -> 1396 (protocols::ExternalSequenceBlockHash and protocols::LocalBlockHash gone; the eight compute_seq_hashes_for_ tracking lines now name dynamo_kv_hashing::block_hash::LocalBlockHash); kv-hints 134 and kv-events 257 change defining paths only (2 and 4 lines); dynamo-error byte-identical. No behavior change. DYNO-190. Signed-off-by: Ishan Dhanani <ishandhanani@gmail.com>
… engines and the router share
A multimodal block's KV identity is its tokens plus the mm_hash of every
object it touches, and the frontend and the engine must compute it the
same way. The pieces that make them agree were split across three
crates: the per-block and per-request metadata in protocol (mm.rs),
the pad and identifier rules in kv-hashing (routing.rs), and the vLLM
extra-key marker in kv-events (extra_keys.rs). None of them is a
counterparty wire type or an identity every crate names, so protocol
should not hold them, and the pad rules are not hashing. They become
one leaf crate, dynamo-multimodal (lib/multimodal, tier 1), that
depends on serde and xxhash-rust and on no Dynamo crate.
Moves, text verbatim (move-diff from the previous commit over the four
source files and the seven destinations: 1132/1137 common 1132, new 5, dropped 0; the five
new normalized lines are the crate's #![deny(missing_docs)] and the
mod tests { } wrappers of pad.rs and extra_key.rs, nothing else):
1. lib/router/protocol/src/mm.rs -> lib/multimodal/src/info.rs whole
(git mv): BlockMmObjectInfo, BlockExtraInfo, RequestMmObjectInfo,
RequestExtraInfo and to_block_level.
2. lib/kv-hashing/src/routing.rs -> lib/multimodal/src/pad.rs:
MM_PAD_SHIFT_VALUE and MM_PAD_HASH_MASK with the SGLang drift
comment, pad_value_for_mm_hash, hash_mm_identifier, and their
three tests. routing.rs keeps BlockHashOptions and every
compute_* and consumes dynamo_multimodal::BlockExtraInfo.
3. lib/kv-events/src/zmq_wire/extra_keys.rs ->
lib/multimodal/src/extra_key.rs: mark_mm_hash_for_extra_key,
parse_mm_hash_from_extra_key, and only_frontend_padded_mm_hashes_
are_parsed. kv-events keeps DYNAMO_CACHE_SALT_PREFIX and the two
extra_keys_to_* codec functions (they take the ZMQ wire item) and
imports the parser from dynamo-multimodal; zmq_wire::mod.rs drops
the two moved names from its re-export list (no shim).
The crate root is the data path (frontend hash_mm_identifier ->
pad_value_for_mm_hash placeholder tokens for SGLang or the
mark_mm_hash_for_extra_key marker for vLLM; RequestExtraInfo ->
BlockExtraInfo per block -> kv-hashing folds mm_hash into the block
hash; engine events -> kv-events parses the marker back) plus private
modules and root re-exports by name; README, AGENTS, and its
`@AGENTS.md` twin state what it owns, the exact dependency set, and what
stays elsewhere (the kv-events codec and placeholder-run
normalization, kv-hashing's compute_block_hash*, lib/llm's
model-specific mm routing, and kv-hashing's PLH RequestMmObjectInfo, a
different shape with the same name, untouched).
Re-pointed, compiler-driven: kv-hashing routing.rs (import and tests),
kv-events event.rs and zmq_wire/{convert,deserialize,extra_keys,
filter,tests,types}.rs, the kernel's selection/input.rs, kv-router's
scheduling/config.rs and services/selection/tests.rs, lib/llm's
kv_router.rs, prefill_router/query.rs, routing_host/{kv_selection,
request_guard}.rs, publisher/{mod,tests}.rs, preprocessor.rs,
protocols/common.rs, protocols/common/preprocessor.rs, and
http/service/generate.rs, and the Python bindings' llm/kv.rs: every
dynamo_router_protocol::mm::, dynamo_kv_hashing::routing::{pad_value_
for_mm_hash, hash_mm_identifier}, and dynamo_kv_events::zmq_wire::
mark_mm_hash_for_extra_key path now reads dynamo_multimodal::. The
deprecated protocols shims in kv-router and the kernel keep every mm
name (the four structs, the two constants, the two functions),
sourced from dynamo-multimodal; the plugins prelude names none of
them.
Edges: kv-hashing -> multimodal (and drops protocol: nothing left in
the crate names it, verified by grep over src, tests, and benches);
kv-events, scheduler, kv-router, lib/llm, and the Python bindings gain
multimodal; kv-indexer and kv-hints reach it only through kv-hashing;
protocol now has no relation to hashing or to multimodal identity.
Workspace member and workspace dependency added; three lockfiles
follow. Gate: new dynamo-multimodal row (no router deps, every router
crate and transport forbidden, direct deps exactly serde xxhash-rust),
ROUTER_FAMILY, ROUTER_CRATE_DIRS, COMPOSITION_ROUTER_DEPS, and
COMPOSITION_DIRECT_DEPS gain it; kv-hashing row router deps become
dynamo-multimodal with protocol forbidden and the direct set swapping
protocol for multimodal; kv-events and scheduler rows gain it in the
closure and the direct set; kv-indexer and kv-hints rows gain it in
the closure only; protocol, zmq, error, slot-tracker,
session-affinity, worker-selection, and admission rows forbid it.
Codeowners: lib/multimodal/ under the router area plus a router +
multimodal co-ownership glob mirroring extra_keys.rs; CODEOWNERS
regenerated (+2 rules). Docs: protocol crate doc and README table,
kv-hashing AGENTS, kv-events README and AGENTS, lib/router README
(fifteen crates). Sixth public-api snapshot added for the new crate.
Snapshots: multimodal new (19 items: the four structs and their
fields, to_block_level, the two constants, the four functions);
protocol 236 -> 223 (the mm module, the four structs, their fields,
and to_block_level); kv-events 257 -> 255 (mark_mm_hash_for_extra_key
and parse_mm_hash_from_extra_key gone; five lines name
dynamo_multimodal::info::BlockExtraInfo instead of protocol's:
KvCacheStoredBlockData::mm_extra_info, RawKvEvent::BlockStored::
block_mm_infos, StoredBlockOptions::mm_extra_info,
create_stored_blocks, extra_keys_to_block_mm_infos); kv-router
1396, kv-hints 134, and dynamo-error 134 byte-identical (the shims'
mm names render as pub use lines without a defining path). No behavior change. DYNO-190.
Validation: chain /tmp/rr-split/chain-190.sh, log s190-validation.log on
the build host; every stage exit 0 except the pre-existing
kvbm-consolidator manifest failure. clippy default and --all-features
(--all-targets, -D warnings), cargo test --all-features, and rustdoc
-D warnings for multimodal (4), kv-hashing (19 + 17 + 3), protocol (9),
kv-events (64), kv-hints (4), kv-indexer (597 + 2 ignored), scheduler
(289 + 1 ignored); kv-router clippy with all features, bench check,
tests (220 + 1 + 1 + 7), prelude and compat tests (1 + 1); workspace
clippy; hosts alone: lib/llm check, clippy, tests kv_router 485,
preprocessor 383 + 2 ignored, protocols::common 132,
http::service::generate 36; thirteen host packages clippy --tests
including libdynamo_llm; Python bindings clippy with
kv-indexer,slot-tracker,select-service; router_features_check;
router_public_api_check (six snapshots match); router_layering_check
47 checks, 0 failures and --host 66 checks, 0 failures; codeowners
--strict 7549/7549; cargo fmt --check; three lockfiles --locked.
Signed-off-by: Ishan Dhanani <ishandhanani@gmail.com>
ActiveLoad is the load message a worker publishes on the event plane. Its publisher (kv_router::publisher::worker_metrics) and every reader (kv_router::metrics_subscriber, discovery::worker_monitor, pool_relay) live in lib/llm, and no router crate names it: the slot tracker has its own in-process SchedulerLoadSnapshot and only linked to ActiveLoad from a doc comment. Protocol is counterparty wire and identity only, and the producer owns its outputs, so the struct moves to lib/llm/src/kv_router/active_load.rs (new file; pub mod active_load and pub use active_load::ActiveLoad at the kv_router root). The dynamo-router-worker crate idea is dropped: the worker vocabulary (WorkerId, DpRank, WorkerWithDpRank, WorkerAffinityTarget, WorkerCapacity, DEFAULT_MAX_BATCHED_TOKENS) stays in protocol::worker. Move, text verbatim: the struct with its derives, serde attribute, and docs. Move-diff from ecc0595 over protocol worker.rs to worker.rs plus the new file: 66/66 common, new 0, dropped 0, MOVE (the new file's two use lines, for WorkerId/DpRank and serde, are import lines the tool normalizes away). Re-pointed to crate::kv_router::ActiveLoad: kv_router/metrics_subscriber, kv_router/publisher/worker_metrics, kv_router/publisher/tests (2), discovery/worker_monitor (2), pool_relay/{pool_registry,load}, pool_relay/wan/grpc/conformance. pool_relay/host.rs and kv_router/metrics.rs name ActiveLoad in log strings and prose only. slot-tracker multi_worker.rs: the doc link becomes plain text naming dynamo_llm::kv_router::ActiveLoad (slot-tracker cannot link into lib/llm). No Cargo.toml edge changes (lib/llm already depends on protocol and serde), so the three lockfiles are unchanged. Deprecated shims: lib/kv-router/src/protocols.rs and lib/router/scheduler/src/protocols.rs drop ActiveLoad now rather than deprecating it, because neither crate can see lib/llm; the kv-router README "Deprecated paths" table gains a protocols::ActiveLoad row that points at dynamo_llm::kv_router::ActiveLoad. Nothing imported the name through either shim. The Python and C bindings never named it. Docs: protocol worker.rs module doc drops "its active load"; the protocol crate doc and README table name lib/llm as the owner; the kv_router AGENTS module map gains active_load.rs. Gate: no row changes. Snapshots: protocol 223 -> 217 (the struct and its five fields under worker::); kv-router 1396 -> 1395 (protocols::ActiveLoad, the early shim removal); kv-events 255, kv-hints 134, error 134, multimodal 19 byte-identical. No behavior change. DYNO-192. Signed-off-by: Ishan Dhanani <ishandhanani@gmail.com>
ishandhanani
had a problem deploying
to
external_collaborator
November 2, 2026 13:53 — with
GitHub Actions
Failure
Owner
Author
|
Landed upstream as ai-dynamo#18055 (squash commit 79692cf on main, 2026-11-11). Closing this fork review unit. |
This branch had an error being deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Router split v2, unit 16 of 17. Stacked on #188 (unit 15).
Summary
Protocol cleanup under the rule decided on 2026-11-01: a stage may depend on another stage for that stage's output types, and protocol keeps counterparty wire and identity only (stage edges over protocol hoisting). #163 dissolves
protocol::load: the fullness rule (RankLoad,BusyThresholds) and the scheduler-load wire (SchedulerLoad,SchedulerGroup) go to the slot tracker,ActiveLoad,WorkerCapacity, andDEFAULT_MAX_BATCHED_TOKENStoprotocol::worker, and worker-selection and admission gain a slot-tracker edge. #165 addsdynamo-kv-hints(lib/kv-hints, tier 1): the KV transfer hints the router hands engines, theKvHintwire andKV_HINT_TRANSFER_*keys, the transfer source choice, the worker transfer keys. #166 moves the residency vocabulary (tiers, domains, owners) todynamo-kv-events. #169 makes kv-indexer own the transfer candidate vocabulary (KvTransferCandidateSource, label hashes) and the residency owner key, with kv-hints re-exporting from kv-indexer. #170 moves the block hash identitiesLocalBlockHashandExternalSequenceBlockHashto kv-hashing and adds the leafdynamo-multimodal(lib/multimodal, tier 1) for the multimodal identity both ends of the wire compute, so protocol has no relation to hashing or multimodal identity. #172 movesActiveLoadtolib/llm, where its publisher and every reader live. After this unit protocol's modules areattempt columns identity metadata session worker worker_type. The rebase placed upstream ai-dynamo#16983 inside this range: at #170's second commit thegenerate.rsdeletion stands andlib/generate-formats/vllmtakeshash_mm_identifier,pad_value_for_mm_hash, andmark_mm_hash_for_extra_keyfromdynamo_multimodal; the ai-dynamo#17700SelectionLogFieldsre-export and the ai-dynamo#16610StorageTier/ExternalSequenceBlockHashimports in lib/llm follow their owners at #165's, #166's, and #170's commits. Replaces fork PRs #163, #165, #166, #169, #170 (two commits), #172.What moves
0cb7114fe4refactor(router): move the fullness rule and scheduler-load wire to the slot tracker:RankLoadandBusyThresholdstodynamo_slot_tracker::busy,SchedulerLoadandSchedulerGrouptodynamo_slot_tracker::wire,ActiveLoad/WorkerCapacity/DEFAULT_MAX_BATCHED_TOKENStoprotocol::worker;protocol::loadremoved. Move-diff (its body): from_normalized=296 to_normalized=296 common=296 new=0 dropped=0,result MOVE(only comments changed in the moved text).23e5b6d053refactor(router): add dynamo-kv-hints, the hints the router hands engines:wire,keys,transferfrom protocol'skv_hints, worker-selection'skv_transfer.rs, and the kernel'skv_hints/protocols. Move-diff (its body): from_normalized=1149 to_normalized=1229 common=1138 new=91 dropped=11,result DIFFERSby design, every differing line listed with its reason in the body: the config-readingWorkerKvTransferKeysExt::ofbody and the config-taking cache method dropped;WorkerKvTransferKeys::from_workerover fields, the kernel adapter's six-argument call, the closure-takingget_or_init, and the kernel testadapter_equals_the_direct_callnew.28000ec2e6refactor(router): move the residency vocabulary to dynamo-kv-events:git mv lib/router/protocol/src/residency.rs lib/kv-events/src/residency.rs, body unchanged. Move-diff (its body): from_normalized=190 to_normalized=190 common=190 new=0 dropped=0,result MOVE.8122fcdee6refactor(router): kv-indexer owns the transfer candidate vocabulary:KvTransferCandidateSource, the label hashes, andResidencyOwnerKeyinto kv-indexer; kv-hints and kv-events re-point. Move-diff (its body, over the five files): from_normalized=573 to_normalized=573 common=573 new=0 dropped=0,result MOVE.77135c3ee8refactor(router): move the block hash identities to dynamo-kv-hashing:block_hash.rswhole into kv-hashing, re-exported at its root. Move-diff (its body): from_normalized=67 to_normalized=67 common=67 new=0 dropped=0,result MOVE.9e05aaf294feat(router): add dynamo-multimodal, the multimodal identity contract engines and the router share:mm.rstomultimodal/src/info.rs, the pad rules from kv-hashing'srouting.rstopad.rs, the extra-key marker from kv-events'extra_keys.rstoextra_key.rs. Move-diff (its body): from_normalized=1132 to_normalized=1137 common=1132 new=5 dropped=0,result DIFFERSby the tool's strict definition; the five new lines are#![deny(missing_docs)]and themod tests { }wrappers ofpad.rsandextra_key.rs, nothing dropped.3b6c929486refactor(router): move ActiveLoad to lib/llm: the struct cut verbatim fromprotocol/src/worker.rsintolib/llm/src/kv_router/active_load.rs. Move-diff (its body): from_normalized=66 to_normalized=66 common=66 new=0 dropped=0,result MOVE.Dependency edges
dynamo-multimodal-> no router crate; direct: serde, xxhash-rustdynamo-kv-hints-> dynamo-router-protocol, dynamo-kv-events, dynamo-kv-hashing, dynamo-kv-indexer, dynamo-multimodal (router deps); direct: dynamo-kv-hashing, dynamo-kv-indexer, dynamo-router-protocol, rustc-hash, serde, serde_jsondynamo-kv-hashing-> dynamo-multimodal (was dynamo-router-protocol, now forbidden in its tree); direct: derive_builder, dynamo-multimodal, dynamo-tokens, serde, serde_json, thiserror, xxhash-rustdynamo-kv-events-> dynamo-router-protocol, dynamo-kv-hashing, dynamo-multimodal; direct gains dynamo-multimodaldynamo-kv-indexer-> dynamo-router-protocol, dynamo-kv-events, dynamo-kv-hashing, dynamo-multimodal (transitive); direct unchangeddynamo-worker-selection-> dynamo-router-protocol, dynamo-slot-tracker; direct gains dynamo-slot-trackerdynamo-admission-> dynamo-router-protocol, dynamo-slot-tracker; direct gains dynamo-slot-trackerdynamo-scheduler-> adds dynamo-kv-hints, dynamo-multimodal; direct gains dynamo-kv-hints, dynamo-multimodaldynamo-kv-router(no features) -> adds dynamo-kv-hints, dynamo-multimodal;COMPOSITION_DIRECT_DEPSgains bothROUTER_FAMILYgains dynamo-multimodal and dynamo-kv-hints;ROUTER_CRATE_DIRSgainslib/multimodalandlib/kv-hints; the forbidden lists of protocol, zmq, slot-tracker, session-affinity, and error gain dynamo-multimodalValidation
Unit-level move-diff, run from the worktree at the unit head
3b6c929486:python3 -I scripts/router_move_diff.py 168b7658a0 3b6c929486 --from <17 paths> --to <22 paths>, the union of the replaced fork PRs'--from/--tolists filtered to the paths present at each end (full command in the artifact/tmp/rr-split/unit-prs/16-move-diff.txt).96 new / 11 dropped: #165's documented delta (91 new / 11 dropped:
WorkerKvTransferKeys::from_workerbuilding the keys from fields, the kernel adapter's six-argument call, the closure-takingWorkerKvTransferKeysCache::get_or_init, and the kernel testadapter_equals_the_direct_callwith itsWorkerConfigLikefixture, replacing the config-readingofbody and the config-taking cache method) plus #170's five structural lines (#![deny(missing_docs)]and themod testsbraces ofpad.rsandextra_key.rs). #163, #166, #169, #170 commit 1, and #172 are exact moves (296, 190, 573, 67, 66 lines common, nothing new or dropped).New and dropped normalized lines (96 new, 11 dropped).
Chain v2, full mode, on the rebased stack head
ebd31cdf4d(every unit's commits are ancestors of it; the chain ran once on the head, not per unit), log/tmp/rr-split/srebase-validation.log: stage 1, the 13 router crates (clippy under default and all features,cargo test1,247 passed / 5 ignored, rustdoc with all features), 1 min 50 s; stage 2, kv-router (clippy, bench check, 229 tests including the prelude test), 1 min 20 s; stage 3, workspace clippy, 4 min 58 s; stage 4, hosts (dynamo-llmlib alone, its kv_router 492 / discovery 275 / pool_relay 261 tests, 13 host crates clippy, bindings clippy; the pre-existing kvbm-consolidator manifest failure is the one non-zero exit), 29 min 21 s; stage 5, gates (every kv-router feature alone and together, 6 public-API snapshots unchanged with kv-router at 918 items, layering gate 47 checks and--host66 checks with 0 failures, codeowners 7590/7590, fmt, 3 lockfiles--locked), 3 min 06 s; 40 min 35 s total, exit 0.Each replaced fork PR carried its own validation chain; see its body.
Tickets
Part of DYNO-184
Part of DYNO-185
Part of DYNO-186
Part of DYNO-191
Part of DYNO-190
Part of DYNO-192