Repository navigation
refactor(router): move ActiveLoad to lib/llm - #172
Closed
ishandhanani wants to merge 1 commit into
Closed
ishandhanani wants to merge 1 commit into
ishandhanani wants to merge 1 commit into
Conversation
ishandhanani
had a problem deploying
to
external_collaborator
November 2, 2026 09:25 — with
GitHub Actions
Failure
ActiveLoad is the load message a worker publishes on the event plane. Its publisher (kv_router::publisher::worker_metrics) and every reader (kv_router::metrics_subscriber, discovery::worker_monitor, pool_relay) live in lib/llm, and no router crate names it: the slot tracker has its own in-process SchedulerLoadSnapshot and only linked to ActiveLoad from a doc comment. Protocol is counterparty wire and identity only, and the producer owns its outputs, so the struct moves to lib/llm/src/kv_router/active_load.rs (new file; pub mod active_load and pub use active_load::ActiveLoad at the kv_router root). The dynamo-router-worker crate idea is dropped: the worker vocabulary (WorkerId, DpRank, WorkerWithDpRank, WorkerAffinityTarget, WorkerCapacity, DEFAULT_MAX_BATCHED_TOKENS) stays in protocol::worker. Move, text verbatim: the struct with its derives, serde attribute, and docs. Move-diff from ecc0595 over protocol worker.rs to worker.rs plus the new file: 66/66 common, new 0, dropped 0, MOVE (the new file's two use lines, for WorkerId/DpRank and serde, are import lines the tool normalizes away). Re-pointed to crate::kv_router::ActiveLoad: kv_router/metrics_subscriber, kv_router/publisher/worker_metrics, kv_router/publisher/tests (2), discovery/worker_monitor (2), pool_relay/{pool_registry,load}, pool_relay/wan/grpc/conformance. pool_relay/host.rs and kv_router/metrics.rs name ActiveLoad in log strings and prose only. slot-tracker multi_worker.rs: the doc link becomes plain text naming dynamo_llm::kv_router::ActiveLoad (slot-tracker cannot link into lib/llm). No Cargo.toml edge changes (lib/llm already depends on protocol and serde), so the three lockfiles are unchanged. Deprecated shims: lib/kv-router/src/protocols.rs and lib/router/scheduler/src/protocols.rs drop ActiveLoad now rather than deprecating it, because neither crate can see lib/llm; the kv-router README "Deprecated paths" table gains a protocols::ActiveLoad row that points at dynamo_llm::kv_router::ActiveLoad. Nothing imported the name through either shim. The Python and C bindings never named it. Docs: protocol worker.rs module doc drops "its active load"; the protocol crate doc and README table name lib/llm as the owner; the kv_router AGENTS module map gains active_load.rs. Gate: no row changes. Snapshots: protocol 223 -> 217 (the struct and its five fields under worker::); kv-router 1396 -> 1395 (protocols::ActiveLoad, the early shim removal); kv-events 255, kv-hints 134, error 134, multimodal 19 byte-identical. No behavior change. DYNO-192. Signed-off-by: Ishan Dhanani <ishandhanani@gmail.com>
ishandhanani
force-pushed
the
idhanani/dyn-190-multimodal-and-block-hash
branch
from
November 2, 2026 13:31
ecc0595 to
9e05aaf
Compare
ishandhanani
force-pushed
the
idhanani/dyn-192-active-load-to-llm
branch
from
November 2, 2026 13:31
f47992d to
3b6c929
Compare
ishandhanani
had a problem deploying
to
external_collaborator
November 2, 2026 13:31 — with
GitHub Actions
Failure
ishandhanani
had a problem deploying
to
external_collaborator
November 2, 2026 13:53 — with
GitHub Actions
Failure
Owner
Author
|
Landed upstream as ai-dynamo#18055 (squash commit 79692cf on main, 2026-11-11). Closing this fork review unit. |
This branch had an error being deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Router split v2 follow-up (DYNO-192). Stacked on #170.
Summary
One commit, move-only, no behavior change. Rule applied: the producer owns its output types;
dynamo-router-protocolis counterparty wire and identity only.ActiveLoadis the load message a worker publishes on the event plane. Its writer (kv_router::publisher::worker_metrics) and every reader (kv_router::metrics_subscriber,discovery::worker_monitor,pool_relay::{pool_registry,load},pool_relay::wan::grpc::conformance) are all in lib/llm, and no router crate reads it: the slot tracker has its own in-processSchedulerLoadSnapshotand only linked toActiveLoadfrom a doc comment. So the struct moves to a new filelib/llm/src/kv_router/active_load.rs(pub mod active_load;andpub use active_load::ActiveLoad;at thekv_routerroot). Thedynamo-router-workercrate idea is dropped:WorkerId,DpRank,WorkerWithDpRank,WorkerAffinityTarget,WorkerCapacity, andDEFAULT_MAX_BATCHED_TOKENSstay inprotocol::worker.#[serde(default)]attribute, and its docs). Move-diff fromecc0595e94overlib/router/protocol/src/worker.rstoworker.rsplus the new file:summary from_normalized=66 to_normalized=66 common=66 new=0 dropped=0,result MOVE. The new file's twouselines (dynamo_router_protocol::worker::{DpRank, WorkerId}andserde::{Deserialize, Serialize}) are import lines the tool normalizes away.crate::kv_router::ActiveLoad(nineuselines in eight files):kv_router/metrics_subscriber.rs,kv_router/publisher/worker_metrics.rs,kv_router/publisher/tests.rs(2),discovery/worker_monitor.rs(2),pool_relay/pool_registry.rs,pool_relay/load.rs,pool_relay/wan/grpc/conformance.rs.pool_relay/host.rsandkv_router/metrics.rsnameActiveLoadonly in log strings and prose, so they are untouched.lib/router/slot-tracker/src/multi_worker.rs: theSchedulerLoadSnapshotdoc link becomes plain text namingdynamo_llm::kv_router::ActiveLoad(slot-tracker cannot link into lib/llm;cargo doc -D warningswould reject the dangling link).dynamo-router-protocolandserde; noCargo.tomlchanges, three lockfiles unchanged, gate rows unchanged.Deprecated shims
lib/kv-router/src/protocols.rsandlib/router/scheduler/src/protocols.rsdropActiveLoadnow (removed, not deprecated): neither crate can see lib/llm, so a shim is impossible. Nothing imported the name through either shim (grep -rn 'protocols::ActiveLoad'underlib/is empty apart from the README row below). The kv-router README "Deprecated paths" table gains aprotocols::ActiveLoad->dynamo_llm::kv_router::ActiveLoadrow recording the early removal.ActiveLoad.Docs
worker.rsmodule doc drops "its active load"; the protocol crate doc "Deliberately elsewhere" list and the README "What is deliberately not here" table name lib/llm (kv_router::ActiveLoad) as the owner.lib/llm/src/kv_router/AGENTS.mdmodule map gainsactive_load.rs.Snapshots (
scripts/router_public_api_check.sh --update, every changed line)lib/router/protocol/public-api.txt223 -> 217:pub struct worker::ActiveLoadand its five fields (active_decode_blocks,active_prefill_tokens,dp_rank,kv_used_blocks,worker_id).lib/kv-router/public-api.txt1396 -> 1395:pub use dynamo_kv_router::protocols::ActiveLoad(the early shim removal above).lib/kv-events/public-api.txt255,lib/kv-hints/public-api.txt134,lib/error/public-api.txt134,lib/multimodal/public-api.txt19 byte-identical.Validation
First run of validation chain v2 (
/tmp/rr-split/chain-v2.sh,MODE=slim: one clippy invocation per (feature set, target kind) over the thirteen router crates, one batched invocation over the thirteen host packages, dynamo-llm alone kept as the one true alone check), log/tmp/rr-split/s192-validation.logon the build host. Slim applies because this PR changes no[features]table and no hostCargo.tomledge. Cold start:target/debug(57 GB of full-debuginfo artifacts) was deleted before the first cargo command and everything was rebuilt underCARGO_PROFILE_DEV_DEBUG=line-tables-only; the precheck (cargo check -p dynamo-llm --lib,--lib --tests, snapshot regeneration) took 11 min of that cold build before the chain. Every stage exit 0 except the pre-existing kvbm-consolidator manifest failure (unrelated, untouched, recorded).Per-stage wall time (chain v2 slim on the cold target; v1 reference from the chain-190 run: 39 min total, 27 min hosts alone): stage 1 router crates 2 min 55 s (two clippy configurations,
cargo test --all-features, rustdoc); stage 2 kv-router 1 min 59 s; stage 3 workspace clippy 6 min 11 s; stage 4 hosts 18 min 35 s (dynamo-llm alone check, three dynamo-llm test runs, the one batched thirteen-host clippy, kvbm-consolidator, bindings clippy); stage 5 gates, snapshots, codeowners, fmt, lockfiles 3 min 5 s; stage 6 move-diff under 1 s. Chain total 32 min 45 s (08:50:05 to 09:22:50) against v1's 39 min; hosts 18.6 min against 27 min. The cold precheck before the chain added 11 min 23 s (shim crates 31 s,cargo check -p dynamo-llm --lib5 min 22 s,--lib --tests3 min 52 s, snapshot regeneration 1 min 38 s); dependencies came from sccache, so the cold cost is the workspace crates' own codegen under the new profile.Stage 1, thirteen router crates, one invocation per configuration:
cargo clippy --all-targets -- -D warnings(default and--all-features);cargo test --all-features: admission 22, error 29, kv-events 64, kv-hashing 19 + 17 + 3, kv-hints 4, kv-indexer 597 + 2 ignored, multimodal 4, protocol 9, scheduler 289 + 1 ignored, session-affinity 10, slot-tracker 131, worker-selection 43, zmq 2, plus one doc-test binary 2 + 2 ignored;RUSTDOCFLAGS="-D warnings" cargo doc --no-deps --all-features.Stage 2:
cargo clippy -p dynamo-kv-router --features http,ingest-zmq,replica-sync-zmq,peer-recovery,metrics,testing --all-targets -- -D warnings;cargo check --features bench --benches; tests 220 + 1 + 1 + 7;plugin_api_preludeandplugin_api_compat1 + 1.Stage 3:
cargo clippy --workspace --all-targets -- -D warningsclean.Stage 4 (slim):
cargo check -p dynamo-llm --libalone;cargo test -p dynamo-llm --libforkv_router(485),discovery(275),pool_relay(261); one batchedcargo clippy -p <hosts> --tests -- -D warningsover dynamo-mocker, dynamo-bench, dynamo-ext-proc, dynamo-custom-policy-builtin, dynamo-worker-selection-policy-catalog, dynamo-sglang-sidecar, dynamo-vllm-sidecar, dynamo-backend-common, dynamo-sglang-mocker, dynamo-vllm-mocker, dynamo-custom-policy-example-epp, dynamo-custom-policy-example-soft-pin-repin, libdynamo_llm;lib/bindings/pythoncargo clippy --features kv-indexer,slot-tracker,select-service -- -D warnings.Stage 5:
scripts/router_features_check.sh;scripts/router_public_api_check.sh(all six snapshots match: 1395, 217, 255, 134, 134, 19);scripts/router_layering_check.sh47 checks, 0 failures and--host66 checks, 0 failures;python3 .github/codeowners/build_codeowners.py --strict7557/7557 explicitly owned;cargo fmt --all -- --check;cargo metadata --lockedfor the root,lib/bindings/python, andlib/bindings/kvbmlockfiles.Stage 6:
python3 -I scripts/router_move_diff.py ecc0595e94 WORKTREE --from lib/router/protocol/src/worker.rs --to lib/router/protocol/src/worker.rs lib/llm/src/kv_router/active_load.rs:summary from_normalized=66 to_normalized=66 common=66 new=0 dropped=0,result MOVE.Batching probe (scratch, not committed): H =
dynamo-worker-selection-policy-catalog(its own closure enables kv-routerdefaultonly), F =ingest-zmq(in the union through ext-proc, custom-policy-builtin, and the epp example). Per-host kv-router feature sets fromcargo tree -p <host> -e features -i dynamo-kv-router --prefix none: mocker, both sidecars, backend-common, both engine mockers, and libdynamo_llmdefault metrics; benchbench default metrics; ext-proc and custom-policy-example-eppdefault ingest-zmq metrics peer-recovery replica-sync-zmq; custom-policy-builtinbench default ingest-zmq peer-recovery replica-sync-zmq; the policy catalog and the soft-pin-repin exampledefault; unionbench default ingest-zmq metrics peer-recovery replica-sync-zmq, so eleven of thirteen hosts see less than the union. A scratch#[cfg(test)]test in the catalog namingdynamo_kv_router::services::indexer::registry::ListenerStatus(services::indexersits behindingest-zmq):cargo clippy -p dynamo-worker-selection-policy-catalog --tests -- -D warningsFAILS (E0433 cannot find indexer in services), while the batchedcargo clippy -p <thirteen hosts> --tests -- -D warningsPASSES (exit 0). So the class exists today: slim's batched host clippy cannot catch a host that uses a feature-gated kv-router item its own closure does not enable. That is the documented trade-off of slim, and the reasonMODE=full(per-host, the v1 behavior) is required whenever a PR changes a[features]table in any router crate or adds/removes a dependency edge in a host's Cargo.toml; this PR does neither (no Cargo.toml changes), so slim applies. Probe reverted,git status --shortempty, log/tmp/rr-split/s192-probe.log.Disk:
target/debug57 GB before the wipe (58 GBtargetin all); 14 GBtargetafter the precheck and the full chain.