Skip to content

refactor(router): move ActiveLoad to lib/llm - #172

Closed
ishandhanani wants to merge 1 commit into
idhanani/dyn-190-multimodal-and-block-hashfrom
idhanani/dyn-192-active-load-to-llm
Closed

ishandhanani wants to merge 1 commit into
idhanani/dyn-190-multimodal-and-block-hashfrom
idhanani/dyn-192-active-load-to-llm

Conversation

@ishandhanani

Copy link
Copy Markdown
Owner

Router split v2 follow-up (DYNO-192). Stacked on #170.

Summary

One commit, move-only, no behavior change. Rule applied: the producer owns its output types; dynamo-router-protocol is counterparty wire and identity only.

  • ActiveLoad is the load message a worker publishes on the event plane. Its writer (kv_router::publisher::worker_metrics) and every reader (kv_router::metrics_subscriber, discovery::worker_monitor, pool_relay::{pool_registry,load}, pool_relay::wan::grpc::conformance) are all in lib/llm, and no router crate reads it: the slot tracker has its own in-process SchedulerLoadSnapshot and only linked to ActiveLoad from a doc comment. So the struct moves to a new file lib/llm/src/kv_router/active_load.rs (pub mod active_load; and pub use active_load::ActiveLoad; at the kv_router root). The dynamo-router-worker crate idea is dropped: WorkerId, DpRank, WorkerWithDpRank, WorkerAffinityTarget, WorkerCapacity, and DEFAULT_MAX_BATCHED_TOKENS stay in protocol::worker.
  • Move, text verbatim (the struct, its derives, the #[serde(default)] attribute, and its docs). Move-diff from ecc0595e94 over lib/router/protocol/src/worker.rs to worker.rs plus the new file: summary from_normalized=66 to_normalized=66 common=66 new=0 dropped=0, result MOVE. The new file's two use lines (dynamo_router_protocol::worker::{DpRank, WorkerId} and serde::{Deserialize, Serialize}) are import lines the tool normalizes away.
  • Re-pointed to crate::kv_router::ActiveLoad (nine use lines in eight files): kv_router/metrics_subscriber.rs, kv_router/publisher/worker_metrics.rs, kv_router/publisher/tests.rs (2), discovery/worker_monitor.rs (2), pool_relay/pool_registry.rs, pool_relay/load.rs, pool_relay/wan/grpc/conformance.rs. pool_relay/host.rs and kv_router/metrics.rs name ActiveLoad only in log strings and prose, so they are untouched. lib/router/slot-tracker/src/multi_worker.rs: the SchedulerLoadSnapshot doc link becomes plain text naming dynamo_llm::kv_router::ActiveLoad (slot-tracker cannot link into lib/llm; cargo doc -D warnings would reject the dangling link).
  • Edges: none. lib/llm already depends on dynamo-router-protocol and serde; no Cargo.toml changes, three lockfiles unchanged, gate rows unchanged.

Deprecated shims

  • lib/kv-router/src/protocols.rs and lib/router/scheduler/src/protocols.rs drop ActiveLoad now (removed, not deprecated): neither crate can see lib/llm, so a shim is impossible. Nothing imported the name through either shim (grep -rn 'protocols::ActiveLoad' under lib/ is empty apart from the README row below). The kv-router README "Deprecated paths" table gains a protocols::ActiveLoad -> dynamo_llm::kv_router::ActiveLoad row recording the early removal.
  • The Python and C bindings never named ActiveLoad.

Docs

  • Protocol worker.rs module doc drops "its active load"; the protocol crate doc "Deliberately elsewhere" list and the README "What is deliberately not here" table name lib/llm (kv_router::ActiveLoad) as the owner.
  • lib/llm/src/kv_router/AGENTS.md module map gains active_load.rs.

Snapshots (scripts/router_public_api_check.sh --update, every changed line)

  • lib/router/protocol/public-api.txt 223 -> 217: pub struct worker::ActiveLoad and its five fields (active_decode_blocks, active_prefill_tokens, dp_rank, kv_used_blocks, worker_id).
  • lib/kv-router/public-api.txt 1396 -> 1395: pub use dynamo_kv_router::protocols::ActiveLoad (the early shim removal above).
  • lib/kv-events/public-api.txt 255, lib/kv-hints/public-api.txt 134, lib/error/public-api.txt 134, lib/multimodal/public-api.txt 19 byte-identical.

Validation

First run of validation chain v2 (/tmp/rr-split/chain-v2.sh, MODE=slim: one clippy invocation per (feature set, target kind) over the thirteen router crates, one batched invocation over the thirteen host packages, dynamo-llm alone kept as the one true alone check), log /tmp/rr-split/s192-validation.log on the build host. Slim applies because this PR changes no [features] table and no host Cargo.toml edge. Cold start: target/debug (57 GB of full-debuginfo artifacts) was deleted before the first cargo command and everything was rebuilt under CARGO_PROFILE_DEV_DEBUG=line-tables-only; the precheck (cargo check -p dynamo-llm --lib, --lib --tests, snapshot regeneration) took 11 min of that cold build before the chain. Every stage exit 0 except the pre-existing kvbm-consolidator manifest failure (unrelated, untouched, recorded).

  • Per-stage wall time (chain v2 slim on the cold target; v1 reference from the chain-190 run: 39 min total, 27 min hosts alone): stage 1 router crates 2 min 55 s (two clippy configurations, cargo test --all-features, rustdoc); stage 2 kv-router 1 min 59 s; stage 3 workspace clippy 6 min 11 s; stage 4 hosts 18 min 35 s (dynamo-llm alone check, three dynamo-llm test runs, the one batched thirteen-host clippy, kvbm-consolidator, bindings clippy); stage 5 gates, snapshots, codeowners, fmt, lockfiles 3 min 5 s; stage 6 move-diff under 1 s. Chain total 32 min 45 s (08:50:05 to 09:22:50) against v1's 39 min; hosts 18.6 min against 27 min. The cold precheck before the chain added 11 min 23 s (shim crates 31 s, cargo check -p dynamo-llm --lib 5 min 22 s, --lib --tests 3 min 52 s, snapshot regeneration 1 min 38 s); dependencies came from sccache, so the cold cost is the workspace crates' own codegen under the new profile.

  • Stage 1, thirteen router crates, one invocation per configuration: cargo clippy --all-targets -- -D warnings (default and --all-features); cargo test --all-features: admission 22, error 29, kv-events 64, kv-hashing 19 + 17 + 3, kv-hints 4, kv-indexer 597 + 2 ignored, multimodal 4, protocol 9, scheduler 289 + 1 ignored, session-affinity 10, slot-tracker 131, worker-selection 43, zmq 2, plus one doc-test binary 2 + 2 ignored; RUSTDOCFLAGS="-D warnings" cargo doc --no-deps --all-features.

  • Stage 2: cargo clippy -p dynamo-kv-router --features http,ingest-zmq,replica-sync-zmq,peer-recovery,metrics,testing --all-targets -- -D warnings; cargo check --features bench --benches; tests 220 + 1 + 1 + 7; plugin_api_prelude and plugin_api_compat 1 + 1.

  • Stage 3: cargo clippy --workspace --all-targets -- -D warnings clean.

  • Stage 4 (slim): cargo check -p dynamo-llm --lib alone; cargo test -p dynamo-llm --lib for kv_router (485), discovery (275), pool_relay (261); one batched cargo clippy -p <hosts> --tests -- -D warnings over dynamo-mocker, dynamo-bench, dynamo-ext-proc, dynamo-custom-policy-builtin, dynamo-worker-selection-policy-catalog, dynamo-sglang-sidecar, dynamo-vllm-sidecar, dynamo-backend-common, dynamo-sglang-mocker, dynamo-vllm-mocker, dynamo-custom-policy-example-epp, dynamo-custom-policy-example-soft-pin-repin, libdynamo_llm; lib/bindings/python cargo clippy --features kv-indexer,slot-tracker,select-service -- -D warnings.

  • Stage 5: scripts/router_features_check.sh; scripts/router_public_api_check.sh (all six snapshots match: 1395, 217, 255, 134, 134, 19); scripts/router_layering_check.sh 47 checks, 0 failures and --host 66 checks, 0 failures; python3 .github/codeowners/build_codeowners.py --strict 7557/7557 explicitly owned; cargo fmt --all -- --check; cargo metadata --locked for the root, lib/bindings/python, and lib/bindings/kvbm lockfiles.

  • Stage 6: python3 -I scripts/router_move_diff.py ecc0595e94 WORKTREE --from lib/router/protocol/src/worker.rs --to lib/router/protocol/src/worker.rs lib/llm/src/kv_router/active_load.rs: summary from_normalized=66 to_normalized=66 common=66 new=0 dropped=0, result MOVE.

  • Batching probe (scratch, not committed): H = dynamo-worker-selection-policy-catalog (its own closure enables kv-router default only), F = ingest-zmq (in the union through ext-proc, custom-policy-builtin, and the epp example). Per-host kv-router feature sets from cargo tree -p <host> -e features -i dynamo-kv-router --prefix none: mocker, both sidecars, backend-common, both engine mockers, and libdynamo_llm default metrics; bench bench default metrics; ext-proc and custom-policy-example-epp default ingest-zmq metrics peer-recovery replica-sync-zmq; custom-policy-builtin bench default ingest-zmq peer-recovery replica-sync-zmq; the policy catalog and the soft-pin-repin example default; union bench default ingest-zmq metrics peer-recovery replica-sync-zmq, so eleven of thirteen hosts see less than the union. A scratch #[cfg(test)] test in the catalog naming dynamo_kv_router::services::indexer::registry::ListenerStatus (services::indexer sits behind ingest-zmq): cargo clippy -p dynamo-worker-selection-policy-catalog --tests -- -D warnings FAILS (E0433 cannot find indexer in services), while the batched cargo clippy -p <thirteen hosts> --tests -- -D warnings PASSES (exit 0). So the class exists today: slim's batched host clippy cannot catch a host that uses a feature-gated kv-router item its own closure does not enable. That is the documented trade-off of slim, and the reason MODE=full (per-host, the v1 behavior) is required whenever a PR changes a [features] table in any router crate or adds/removes a dependency edge in a host's Cargo.toml; this PR does neither (no Cargo.toml changes), so slim applies. Probe reverted, git status --short empty, log /tmp/rr-split/s192-probe.log.

  • Disk: target/debug 57 GB before the wipe (58 GB target in all); 14 GB target after the precheck and the full chain.

ActiveLoad is the load message a worker publishes on the event plane.
Its publisher (kv_router::publisher::worker_metrics) and every reader
(kv_router::metrics_subscriber, discovery::worker_monitor, pool_relay)
live in lib/llm, and no router crate names it: the slot tracker has its
own in-process SchedulerLoadSnapshot and only linked to ActiveLoad from
a doc comment. Protocol is counterparty wire and identity only, and the
producer owns its outputs, so the struct moves to
lib/llm/src/kv_router/active_load.rs (new file; pub mod active_load and
pub use active_load::ActiveLoad at the kv_router root). The
dynamo-router-worker crate idea is dropped: the worker vocabulary
(WorkerId, DpRank, WorkerWithDpRank, WorkerAffinityTarget,
WorkerCapacity, DEFAULT_MAX_BATCHED_TOKENS) stays in protocol::worker.

Move, text verbatim: the struct with its derives, serde attribute, and
docs. Move-diff from ecc0595 over protocol worker.rs to worker.rs
plus the new file: 66/66 common, new 0, dropped 0, MOVE (the new file's two use lines, for
WorkerId/DpRank and serde, are import lines the tool normalizes away).
Re-pointed to crate::kv_router::ActiveLoad: kv_router/metrics_subscriber,
kv_router/publisher/worker_metrics, kv_router/publisher/tests (2),
discovery/worker_monitor (2), pool_relay/{pool_registry,load},
pool_relay/wan/grpc/conformance. pool_relay/host.rs and
kv_router/metrics.rs name ActiveLoad in log strings and prose only.
slot-tracker multi_worker.rs: the doc link becomes plain text naming
dynamo_llm::kv_router::ActiveLoad (slot-tracker cannot link into
lib/llm). No Cargo.toml edge changes (lib/llm already depends on
protocol and serde), so the three lockfiles are unchanged.

Deprecated shims: lib/kv-router/src/protocols.rs and
lib/router/scheduler/src/protocols.rs drop ActiveLoad now rather than
deprecating it, because neither crate can see lib/llm; the kv-router
README "Deprecated paths" table gains a protocols::ActiveLoad row that
points at dynamo_llm::kv_router::ActiveLoad. Nothing imported the name
through either shim. The Python and C bindings never named it.

Docs: protocol worker.rs module doc drops "its active load"; the
protocol crate doc and README table name lib/llm as the owner; the
kv_router AGENTS module map gains active_load.rs.

Gate: no row changes. Snapshots: protocol 223 -> 217 (the struct and
its five fields under worker::); kv-router 1396 -> 1395
(protocols::ActiveLoad, the early shim removal); kv-events 255,
kv-hints 134, error 134, multimodal 19 byte-identical. No behavior
change. DYNO-192.

Signed-off-by: Ishan Dhanani <ishandhanani@gmail.com>
@ishandhanani

Copy link
Copy Markdown
Owner Author

Landed upstream as ai-dynamo#18055 (squash commit 79692cf on main, 2026-11-11). Closing this fork review unit.

This branch had an error being deployed

1 failed deployment
external_collaborator — 3b6c9294 Deployed Nov 2, 2026 by ishandhanani via ok-to-test #528
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

documentation Improvements or additions to documentation refactor router

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant