Skip to content

refactor(router): split dynamo-kv-router into vocabulary, stage, and kernel crates - #18055

Merged
ishandhanani merged 129 commits into
ai-dynamo:mainfrom
ishandhanani:idhanani/dyn-router-split-v2
Nov 11, 2026
Merged

ishandhanani merged 129 commits into
ai-dynamo:mainfrom
ishandhanani:idhanani/dyn-router-split-v2

Conversation

@ishandhanani

@ishandhanani ishandhanani commented Nov 4, 2026 •

Copy link
Copy Markdown
Contributor

Summary

The composition crate dynamo-kv-router becomes 12 crates under lib/: five vocabulary crates (dynamo-router-protocol, dynamo-kv-events, dynamo-kv-hashing, dynamo-kv-hints, dynamo-error), five stage crates under lib/router/ (dynamo-kv-indexer, dynamo-slot-tracker, dynamo-session-affinity, dynamo-worker-selection, dynamo-admission), the kernel (dynamo-scheduler), and dynamo-kv-router as the composition: the selection service, the plugin registry and plugins::prelude, the flat KvRouterConfig and its parsers, and every adapter (HTTP servers, ZMQ ingest, replica mesh, peer recovery, metrics) behind a feature with default = [].

Why: one producer per type and no common crate, so a change to a wire shape, a stage, or a host has one home; in-tree hosts import a stage through its host module and the vocabulary crates directly, plugins import dynamo_kv_router::plugins::prelude and depend on dynamo-kv-router alone, and scripts/router_layering_check.sh pins every Dynamo-family edge, the forbidden closures, the composition crate's no-features tree, and the hosts' graphs in CI (--host).

Deliberately not in this PR: behavior changes (the step 9 items stay parked: one InputTrigger, worker-only pins with Option<DpRank>, the affinity_target versus pinned_worker pair, folding session affinity into the kernel, the ZMQ helper dedupe) and dynamo-metrics.

Rules

  • The producer owns its output: a type lives in the crate that gives it meaning, and a stage may depend on another stage for that stage's output types.
  • Protocol is counterparty wire and identity only: dynamo-router-protocol carries no executable behavior and no topic modules; what computes stays with its producer.
  • Feature crates depend on stages, never the reverse: dynamo-kv-hints names dynamo-kv-indexer; the indexer never names the hints.
  • No shims: a move is a move; the deprecated paths the series carried for one release are removed in the same series, and the migration map is in lib/router/kv-router/README.md.
  • Plugins are prelude-only: a statically linked policy or classifier crate depends on dynamo-kv-router alone and imports plugins::prelude; the gate's plugin table enforces it for the builtin, catalog, and example crates.

Crates

Crate Path Owns Depends on (Dynamo crates, exact)
dynamo-error lib/error The error contract: DynamoError, ErrorClass::http_status, the reason catalog, PublicDetails none
dynamo-router-protocol lib/router/protocol The identities and worker-facing vocabulary every crate names: worker, rank, partition, pool, owner, and domain ids; WorkerType; WorkerCapacity; the candidate columns (WorkerInputs, WorkerInfo); AttemptId and its minter; SessionContext none
dynamo-kv-hashing lib/kv-hashing The hash contract engines and the router share: LocalBlockHash, ExternalSequenceBlockHash, block and sequence hashes, the tracking-hash context; its multimodal module is the multimodal identity both ends of the wire compute (per-block and per-request metadata, the pad and identifier rules, the extra-key marker) dynamo-tokens
dynamo-kv-events lib/kv-events Everything about KV events: the event types, the engine ZMQ codec with its filters (including the multimodal exclusion and partial-tail filters from #17126 and #17794), placement, the replay cursor, the residency vocabulary dynamo-router-protocol, dynamo-kv-hashing
dynamo-kv-hints lib/router/kv-hints KV transfer hints the router hands engines: the KvHint wire, the transfer source choice (with #17657's fetch continuations), the worker transfer keys dynamo-router-protocol, dynamo-kv-hashing, dynamo-kv-indexer
dynamo-kv-indexer lib/router/kv-indexer KV events in, prefix knowledge out: radix trees, the concurrent compressed tree, the cuckoo pool index, lower tiers, approximate LRU, the session prefix index, the ingest backend, residency projection, the transfer candidate vocabulary, the service DTOs, IndexerConfig dynamo-router-protocol, dynamo-kv-events, dynamo-kv-hashing, dynamo-tokens
dynamo-slot-tracker lib/router/slot-tracker Bookings per rank in, load and fullness out: active sequences, the prompt registry, block and prefill trackers, load projection, the fullness rule, the replication and scheduler-load wire dynamo-router-protocol, dynamo-tokens
dynamo-session-affinity lib/router/session-affinity Session bindings in, constraints out: the replicated session table, leases, holds, weak affinity, WorkerAffinityTarget, its replication wire dynamo-router-protocol
dynamo-worker-selection lib/router/worker-selection Pick one rank from a candidate table under a policy: the rows and views, the filter, scorer, and picker API, the composed policy, runtime updates, the hosted picker (#17712), the default policy, the LoRA filter, conditional disaggregation, SelectionConfig dynamo-router-protocol, dynamo-slot-tracker, dynamo-session-affinity
dynamo-admission lib/router/admission Admit, hold, or reject now: the per-class policy queue, resolved profiles, the classifier runtime and plugin API, AdmissionConfig dynamo-router-protocol, dynamo-slot-tracker, dynamo-session-affinity, dynamo-tokens
dynamo-scheduler lib/router/scheduler Run the stages atomically for one request: SchedulingRequest, eligibility, the hook traits hosts implement, SelectionCore, the queue actor, the stage conversions, Validated<RouterConfig>, KvSchedulerError and SelectionError the five stages, dynamo-kv-events, dynamo-kv-hashing, dynamo-kv-hints, dynamo-router-protocol, dynamo-tokens
dynamo-kv-router lib/router/kv-router Compose a router and be the one crate plugins name: SelectionServiceBuilder, the plugin registry, plugins::prelude, the flat KvRouterConfig and its CLI, env, and policy-file parsers, every adapter behind a feature (http, ingest-zmq, replica-sync-zmq, peer-recovery, metrics), the private ZMQ socket helper dynamo-scheduler, the five stages, the vocabulary crates

dynamo-llm (the frontend host) and dynamo-runtime are on the request path but are not router crates; the runtime depends on no router crate.

Compatibility paths

The split lands in 1.8.0 while 1.7.0 ships main's unsplit dynamo-kv-router, so the 1.7 import paths of dynamo_kv_router keep compiling in 1.8 and 1.9 and are removed in 1.10 (owner decision, 2026-11-10; paths only: no alias features, no feature on the compat module). One file, lib/router/kv-router/src/compat.rs, holds the inventory as named re-exports from the owner crates: 660 pre-split paths (617 distinct lines: 273 plain structs as #[deprecated] type aliases that warn with the new path, 344 pub use lines for traits, functions, constants, enums, generic and tuple types) across 32 pre-split modules, the crate root, and the 9 root module aliases; 14 items and the test_utils block carry the owner's feature cfg. lib.rs mounts the root modules and items by name; scheduling, scheduling::config, services, services::selection, and test_utils still exist and mount their blocks by name from their own files. 207 pre-split paths were already unchanged. Not re-exported: the 9 names of the dummy shared-cache server (services::shared_cache), removed by the stack, and protocols::ActiveLoad, which lives in dynamo_llm::kv_router (this crate cannot depend on dynamo-llm). The schedule, the module-to-owner table, and the removed names are on the deprecations page (docs/fern/pages/reference/general/releases/deprecations.mdx, Upcoming section). Inventory: rustdoc JSON of origin/main's lib/kv-router (hidden items included) mapped to rustdoc JSON of the owner crates at this head, each mapping checked by normalized signature. Proof: an out-of-workspace probe crate with one use per inventory path compiles with 797 paths on default features, 858 with http, ingest-zmq, replica-sync-zmq, peer-recovery, and metrics, and 903 with bench and testing added.

Dependency graph

Dynamo-family edges, from the gate rows at HEAD (dynamo-tokens, a pre-existing leaf, omitted).

graph BT
    error[dynamo-error]
    router_protocol[dynamo-router-protocol]
    kv_hashing[dynamo-kv-hashing]
    kv_events[dynamo-kv-events]
    kv_hints[dynamo-kv-hints]
    kv_indexer[dynamo-kv-indexer]
    slot_tracker[dynamo-slot-tracker]
    session_affinity[dynamo-session-affinity]
    worker_selection[dynamo-worker-selection]
    admission[dynamo-admission]
    scheduler[dynamo-scheduler]
    kv_router[dynamo-kv-router]
    kv_events --> kv_hashing
    kv_events --> router_protocol
    kv_hints --> kv_hashing
    kv_hints --> kv_indexer
    kv_hints --> router_protocol
    kv_indexer --> kv_events
    kv_indexer --> kv_hashing
    kv_indexer --> router_protocol
    slot_tracker --> router_protocol
    session_affinity --> router_protocol
    worker_selection --> router_protocol
    worker_selection --> session_affinity
    worker_selection --> slot_tracker
    admission --> router_protocol
    admission --> session_affinity
    admission --> slot_tracker
    scheduler --> admission
    scheduler --> kv_events
    scheduler --> kv_hashing
    scheduler --> kv_hints
    scheduler --> kv_indexer
    scheduler --> router_protocol
    scheduler --> session_affinity
    scheduler --> slot_tracker
    scheduler --> worker_selection
    kv_router --> admission
    kv_router --> kv_events
    kv_router --> kv_hashing
    kv_router --> kv_hints
    kv_router --> kv_indexer
    kv_router --> router_protocol
    kv_router --> scheduler
    kv_router --> session_affinity
    kv_router --> slot_tracker
    kv_router --> worker_selection
Loading

How to review

The 20 review units on the fork, in order; each unit's body carries its move-diff artifact (what moved verbatim versus what is new). The stack commits are in this PR unsquashed in the same order, followed by one merge commit absorbing upstream main as of 182039dbb5, then the three review follow-up commits listed under Review follow-ups. A second merge commit a30e067f9a then absorbs upstream main as of 53015f9b05 (36 commits): kthui's policy-class batch #17723 flatten policy classes, #17724 EDF ordering, #17725 controlled delay (the policy queue, EDF order, class budgets and head expiry in lib/router/admission; the queue actor's class timers, due-time checks, DRR-on-admission and retained overlap refresh in lib/router/scheduler, which also hosts the restored policy.rs score keys; the flat YAML policy_classes/default_policy_class resolver in lib/router/kv-router; UnknownPolicyClass mapped to 400 in lib/llm and ext-proc), #17437 image-aware KV routing for TRT-LLM sidecars (the trtllm wire module in lib/kv-events, the sidecar importing kv-events, kv-hashing, multimodal and protocol directly), #14775 prefill cancellation, #17716/#18077 pool relay, #17707 NVTX range, #18102 LiveEngine buffering and #17887 the shared replay clock (lib/llm, bindings, mocker, bench; auto-merged). A third merge commit e626076861 absorbs upstream main as of 02331f0d6a (5 commits): #18008 Fx-set worker availability (WorkerIdSet and AvailableWorkerIds in the kernel's types.rs/filter.rs/queue.rs, exported to hosts via dynamo_scheduler::host and to plugins via the prelude), #18084 CRTC sweep trigger (lib/router/kv-indexer), #17900 shared bench lane primitive (lib/bench), #17947 and #17625 outside the router. A fourth merge commit e7b2b4fa04 absorbs upstream main as of 20c5f11f3d (5 commits): #18009 prepared rows kept past the reject thresholds (the kernel's filter.rs/queue.rs/selector/mod.rs drop full workers' prepared rows before the policy adopts them; PreparedRows::retain and the dp_size column in lib/router/worker-selection), #18108 CRTC decode-store append (lib/router/kv-indexer), #16258 vLLM token-limit defaults (lib/llm preprocessor, vLLM sidecar), #18152 and #16426 outside the router. A fifth merge commit 4feb38413d absorbs upstream main as of f580aacddc (4 commits, no conflicts): #18116 plugin selectors for the transcription router (lib/llm discovery watcher), #17691 DynamoDB registration store (global discovery service), #16288 and #16723 outside the router. A sixth merge commit dfd1bba7b1 absorbs upstream main as of dc1fef6951 (1 commit, no conflicts): #15413 operator RollingUpdate, outside the router. A seventh merge commit ee45cf2d1c absorbs upstream main as of 295854c125 (1 commit, no conflicts): #16737 SGLang EAGLE3 request-migration e2e test under tests/ plus three docs pages, outside the router. An eighth merge commit b6bfa517ee absorbs upstream main as of 2fb3b132fc (1 commit, no conflicts): the automated community-events docs refresh (README.md, docs/fern/components/events.generated.ts), outside the router. A ninth merge commit d33927a031 absorbs upstream main as of ed5b75d51e (26 commits, 8 conflicts resolved into the owner crates): #17910 strict boolean worker flags (the kernel's selection/types.rs WorkerRequest/WorkerPatchRequest, carried by rename detection), #18240 configurable session-affinity entry limit (lib/llm reads SessionAffinityConfig and MAX_SESSION_AFFINITY_ENTRIES from dynamo_session_affinity::host, which now exports the limit), #18145 EPP co-ownership (the areas.yaml globs re-pointed to lib/router/kv-router/src/plugins/registry.rs, the kv-router and kernel halves of config.rs, the kernel's selection/*, kv-router's services/selection/* and session-affinity/src/lib.rs, owners kept), #17708 nvtx CI steps next to the stack's feature-build steps, #17623 dependency bumps (mocker keeps fastrand next to zip; lockfiles regenerated), #17675 DynamoDB global session affinity (new lib/global-router-store-dynamodb member), #18278 replay synthetic sessions (mocker, bindings), #16607 use-case docs refactor (the stack's DYN_LOG directive fix carried to the moved multimodal KV routing page), and the lib/llm, runtime, planner, operator, recipes and docs commits outside the router. A tenth merge commit 19e2aa67a9 absorbs upstream main as of e7cb9a9d2b (1 commit, 1 conflict): #17792 (the 1.8 two-tier default notice and the host_default_factory seam) in lib/llm's kv_router/plugins.rs and entrypoint/input/http.rs; the one conflict was the import line where the stack's WorkerInputs and upstream's WorkerSelectionPolicyFactory share a use path, both kept. An eleventh merge commit 86b7bf415b absorbs upstream main as of 4c5706219a (1 commit, no conflicts): the automated community-events docs refresh (README.md, docs/fern/components/events.generated.ts), outside the router. A twelfth merge commit f7a0ec44d9 absorbs upstream main as of c43e48e9c8 (1 commit, 6 conflicts resolved into the owner crates): #17896 standalone dynamo-lora crate (lib/lora). Upstream wires it to dynamo-kv-router for WorkerWithDpRank, ActiveSequenceEvent/ActiveSequenceEventData and LoraWorkerFilter; the stack serves those from dynamo-router-protocol, dynamo-slot-tracker and dynamo-worker-selection, so lib/lora/Cargo.toml names those three in place of dynamo-kv-router (the re-pointing dynamo-vllm-generate received in the first merge; the trait is defined in worker-selection and implemented in lora, so no router crate reaches dynamo-lora). The five conflicted lib/lora files were import blocks where rename detection carried the stack's re-points onto the moved files, resolved as upstream's flattened crate:: paths plus the stack's imports; lib/llm/Cargo.toml keeps the stack's router crate list plus dynamo-lora; three lockfiles regenerated; CODEOWNERS regenerated with upstream's owners on every lib/lora path. A thirteenth merge commit 5dc7edfa94 absorbs upstream main as of fa98520149 (1 commit, no conflicts): #18192 operator LPX agent-runtime sidecar target, outside the router. A fourteenth merge commit 68005ade79 absorbs upstream main as of d9e5845dd7 (1 commit, no conflicts): the automated community-events docs refresh (docs/fern/components/events.generated.ts), outside the router. A fifteenth merge commit dbbe5e42fe absorbs upstream main as of 57b5a57b24 (2 commits, 3 modify/delete conflicts on shims the stack deleted, resolved into the owner crates): #15773 session-affinity hits logged and measured like KV selections (WorkerAffinityTarget::matches in lib/router/session-affinity; SelectionLogFields.session_affinity, the WorkerSelector::applies_worker_filters default, the selection log field that replaces the Selected pinned worker line, and the session-affinity fact in the kernel's protocols.rs and selector/mod.rs; WorkerSelectionPolicy::applies_worker_filters as an inherent method in lib/router/worker-selection with the kernel's trait impl delegating, the uses_exclusive_affinity_target shape; the best-eligible raw-cache widening with three queue tests, the wire-pin affinity filter with two core tests, and the reference-selector override carried into the kernel by rename detection; the replay adapter override in lib/mocker; lib/llm host logging, affinity-target forwarding, metrics docs and two tests), plus #15671 the Qwen3-ASR vLLM example outside the router. A sixteenth merge commit 7e00036163 absorbs upstream main as of d2ceb6a9b8 (2 commits, no conflicts): #17978 release-note contributor credits and #18236 DeepSeek V4 recipes, outside the router. A seventeenth merge commit 61da52673c absorbs upstream main as of aa620b7181 (2 commits, no conflicts): #18334 Nemotron 3.5 Super VL recipe defaults and #17937 multimodal audio-loader client errors, outside the router. An eighteenth merge commit 7ab262c4a0 absorbs upstream main as of 3d477a4129 (1 commit, no conflicts): #16735 planner startup no longer blocked on replicas pending startup capacity (Python under components/src/dynamo/planner), outside the router. A nineteenth merge commit b14856a71e absorbs upstream main as of 197c7e56dd (1 commit, no conflicts): #18255 Kimi-K3.1-Pro recipes aligned with recipe best practices (recipes/kimi-k3.1-pro plus its catalog entry and model-recipes page under docs/fern/pages/recipes), outside the router. A twentieth merge commit bb61c38da3 absorbs upstream main as of 7d8eab2b94 (1 commit, no conflicts): #17575 Kubernetes discovery persistence bounded (lib/runtime/src/discovery/kube.rs and kube/crd.rs, one new runtime test), outside the router. A twenty-first merge commit 0ac05d1d90 absorbs upstream main as of 2680b5064d (1 commit, no conflicts): #18252 the muse parser alias deprecated in favor of muse_glimmer (lib/llm preprocessor.rs and unified_parser.rs alias warning plus one unit test, components/src/dynamo/common runtime args and tests, three tool-calling docs pages), outside the router. A twenty-second merge commit 9bcad0cde3 absorbs upstream main as of ad8ecd2e5d (1 commit, no conflicts): #18293 SGLang MultimodalProcessorOutput imported from req_types (components/src/dynamo/sglang/_compat.py), outside the router. A twenty-third merge commit 791179e68f absorbs upstream main as of 5c17fb8ab6 (1 commit, no conflicts): #15856 lifecycle badges required on recipe targets (the recipe catalog schema, validator and backfill list under docs/fern/pages/recipes/_catalog, docs/tests/test_recipe_badges.py, and one examples glob plus two cases in .github/filters.yaml and test-filters.js next to the stack's router filter lines; the filters self-test passes 92 / 92), outside the router. A twenty-fourth merge commit 6fd5106465 absorbs upstream main as of ef8d3b1a0b (3 commits, two import-only conflicts): #18072 DYN_LORA_S3_TLS_INSECURE for S3 LoRA downloads (lib/lora/src/source.rs plus three unit tests, lib/runtime/src/config/environment_names.rs, two LoRA docs pages), #17250 EPP tokenization moved off the async workers with stage and stall metrics (deploy/inference-gateway/ext-proc: new tokio_runtime.rs and log_budget.rs, metrics.rs, envoy_helpers.rs, server.rs, epp.rs, the tokio schedule-latency feature; gateway and observability docs) taken whole, the two conflicts being adjacent import lines where the stack's owner-crate imports (dynamo_scheduler::host::{RouterConfigOverride, RoutingConstraints} in epp.rs, dynamo_kv_events::zmq_wire::DYNAMO_CACHE_SALT_PREFIX in server.rs) are kept next to upstream's new tokio::sync::Semaphore and crate::tokio_runtime imports, so the merged files differ from upstream only by those re-pointed imports, and #18003 EFA installer, Mooncake 0.3.14 and NIXL bumps (container/ only); validated with the slim chain, cargo test -p dynamo-ext-proc (192 + 1 passed, all 23 new and 2 renamed or rewritten tests by name) and cargo test -p dynamo-lora (157 + 19 passed, the three new S3 TLS tests by name). A twenty-fifth merge commit 58d4775769 absorbs upstream main as of dec6c2200e (1 commit, no conflicts): #17751 planner load_scaling_down_sensitivity 0 semantics corrected (six planner docs pages), outside the router. A twenty-sixth merge commit e361e9607b absorbs upstream main as of 1d6e9e0846 (1 commit, no conflicts): #18338 dynamo-runtime parallel CPU-only test timeout raised to 45m (.github/workflows/dynamo-pipeline.yml), outside the router. A twenty-seventh merge commit 0a5745b148 absorbs upstream main as of baba4997ea (1 commit, no conflicts): #17517 metrics review prompt (.ai/metrics-review.md, applied in .coderabbit.yaml, the pr-checklist skill and the review-prompts README; lib/runtime/src/metrics/prometheus_names.rs module doc comments; one new areas.yaml glob with CODEOWNERS regenerated identical), outside the router. A twenty-eighth merge commit dfe78dab37 absorbs upstream main as of 16c7f5d176 (2 commits, no conflicts): #18198 router replicas registered through global discovery (lib/global-common, lib/global-discovery-client, lib/global-discovery-service with the new backends/routers.rs and tests/router_registration.rs, lib/global-frontend, whose README auto-merged with upstream's new section next to the stack's dynamo_router_protocol::identity::PoolId re-point, and the global-discovery-service configuration page) and #17227 TRT-LLM container images on the upstream NIXL stack (container/, tests/dependencies/test_trtllm_nixl.py, TRT-LLM and release docs), outside the router. A twenty-ninth merge commit e69788f9d2 absorbs upstream main as of 58be22021c (1 commit, no conflicts): #18295 Worker registration fetches HF models without weight files (lib/backend-common/src/worker.rs: LocalModel::fetch(&source, true) in build_local_model plus one new test), outside the router. A thirtieth merge commit 079928f7ed absorbs upstream main as of 403ead2f65 (2 commits, no conflicts): #18311 TRT-LLM guided-decoding grammar rejections returned as 400 (new components/src/dynamo/trtllm/errors.py, handler_base.py raising InvalidArgument before any output token, two new tests) and #18324 Responses API echoes description: null on function tools (echoed_tools_json in lib/llm/src/protocols/openai/responses/template_tools.rs, used by NvResponse serialization and the stream converter's lifecycle snapshots, two new tests), outside the router. A thirty-first merge commit 5e36cd5bd6 absorbs upstream main as of 51a6b3f2f3 (1 commit, no conflicts): #18353 deploy tests ride out 60 s vCluster tunnel restarts (tests/deploy/vcluster_utils.py raises VCLUSTER_CONNECTION_RETRY_LIMIT from 3 to 12, dgdr_utils.py bounds the connection retries inside wait_for_phase by the phase deadline with asyncio.wait_for, test_dgdr_utils.py uses the constant and gains two tests), outside the router. A thirty-second merge commit 8536781e4a absorbs upstream main as of 4948d19a4a (1 commit, no conflicts): the automated docs: refresh community events calendar refresh (docs/fern/components/events.generated.ts, GENERATED_ON 2026-11-06 -> 2026-11-07), outside the router. A thirty-third merge commit eccdcd1ffb absorbs upstream main as of 335d2f6a18 (1 commit, no conflicts): #17730 Harbor and Substrate agents tutorial (new examples/agents/harbor-substrate/ tree, new and edited docs pages under docs/fern/pages/{use-cases/agents,kubernetes/agent-sandboxes} with assets and docs/fern/index.yml; .github/codeowners/areas.yaml gives the agents area the glob examples/agents/ and the generated CODEOWNERS follows, regenerated byte-identical with build_codeowners.py --strict passing), outside the router. A thirty-fourth merge commit bdc8812aab absorbs upstream main as of 1dd1eefd9d (1 commit, no conflicts): #18053 SGLang reasoning gate advertised for multimodal LLM workers (components/src/dynamo/sglang/register.py keys the gate on dynamo_args.multimodal_worker / multimodal_encode_worker instead of the handler type, so frontend-decoded media workers advertise it; tests/test_runtime_metadata.py reparametrized over worker_flags with a reasoning_gate_expected column), Python only, outside the router. A thirty-fifth merge commit 84de392d42 absorbs upstream main as of 05ee045712 (1 commit, no textual conflicts, one semantic conflict resolved into the owner crates): #17893 runtime LoRA adapter identity (lib/lora/src/runtime.rs taken whole; it bounds worker configs by the router's WorkerConfigLike and reads RoutingConstraints, which upstream imports from dynamo_kv_router::protocols and the stack serves from dynamo_scheduler::host, so lib/lora gains the dynamo-scheduler host edge that lib/llm, ext-proc and mocker already have, with WorkerId from dynamo_router_protocol::worker; the file differs from upstream only in those four path lines; lib/llm/src/lora.rs RuntimeLoraWorkerFacts impl and the DYN_LORA_* environment names auto-merged; three lockfiles regenerated, one line each), validated with cargo test -p dynamo-lora (171 + 19 passed, all 14 new runtime::tests by name), cargo test -p dynamo-llm kv_router (505 passed), cargo test -p dynamo-runtime environment_names and lora clippy -D warnings. A thirty-sixth merge commit eb3e41ca84 absorbs upstream main as of 27f6052cd5 (1 commit, no conflicts): #18302 split tool-call openers preserved after reasoning (components/src/dynamo/frontend/prepost.py holds back the trailing text after a reasoning end that is a proper prefix of a tool-start marker in a new _pending_tool_opener and prepends it to the next delta, so an opener split across chunks starts tool buffering instead of leaking as content; tests/test_vllm_processor_unit.py gains the parametrized test_hermes_split_tool_opener_at_reasoning_end), Python only, outside the router. A thirty-seventh merge commit 8d6d7dffe6 absorbs upstream main as of b977beb708 (1 commit, no conflicts): the automated docs: refresh community events data refresh (no PR number; one line of docs/fern/components/events.generated.ts), generated docs data only, outside the router. A thirty-eighth merge commit 281cf248b5 absorbs upstream main as of 02a6ccbb0e (1 commit, four conflicts): #18281 worker-selection policies written inline per role (each worker_selection role selects a bare policy type name or a {type, parameters} mapping; the instances: list is deprecated for removal in v1.8 and still accepted with a warning), with the new WorkerSelectionPolicySelection enum and per-role parser kept beside WorkerSelectionConfig in lib/router/kv-router/src/plugins/worker_selection/config.rs, WorkerSelectionPolicySelections and the CLI/env/YAML precedence in kv-router scheduling/config.rs, and the registry's bare-type and inline resolution in kv-router plugins/registry.rs; the kernel, dynamo-worker-selection and dynamo-admission are untouched. The four conflicts are import and test-helper hunks (lib/router/worker-selection/src/lib.rs keeps the stack's trait module, upstream's one-line re-export landing in the kv-router plugins/worker_selection.rs shim instead; registry.rs, scheduling/config.rs and policy_config.rs keep the stack's owner-crate imports beside upstream's new ones) plus one semantic conflict (upstream's new registry test passes &KvRouterConfig to the factory, the stack's takes &SelectionConfig::from(&config) like its existing tests); plugins::prelude exports WorkerSelectionPolicySelection. Validated with the slim chain (28 min, every stage 0 except the expected kvbm-consolidator manifest failure), cargo test -p dynamo-kv-router (251 + 2 + 7 passed, the registry, policy_config, config-precedence and services::selection tests by name), cargo test -p dynamo-router-policies (110 passed, all nine fixture tests by name), the mocker, ext-proc and lib/llm watcher fixture tests by name, and test_kv_router_args.py (91 passed). A thirty-ninth merge commit 19d3753420 absorbs upstream main as of 3a8226a768 (19 commits, one conflict): #17923 no block-lookup growth for existing-key updates, landing in lib/router/kv-indexer/src/concurrent_radix_tree_compressed/block_lookup.rs (rename-detected; the file differs from upstream only by its crate::prelude import); #17741 dynamo-tokenizers 4.1.2, dynamo-renderer 9.0.0 and tokenizers 0.23.2 together with #17520's dynamo-protocols 8.5.0 pin, whose root Cargo.toml hunk was the one conflict (upstream's pins kept beside the stack's workspace members and path crates; the four lockfiles regenerated with no drift); #18380 and #18448 bindings py_convert.rs, the direct token_ids and extra_args reads in RoutedEngine.generate (llm/kv.rs extract_token_ids now delegates to it, the stack's owner-crate imports kept); the other fourteen commits are lib/llm frontend code (chat-completions passthrough, media decoders and EXIF, speculative prefill, images aggregator), lib/runtime TCP_NODELAY, Python frontend, vllm, trtllm and common, recipes and docs, all auto-merged. Validated with the slim chain (40 min, every stage 0 except the expected kvbm-consolidator manifest failure), the CRTC tests (70 passed including the new existing_keys_do_not_grow_a_full_lookup), upstream's lib/llm tests by name (514 passed across the media, speculative-prefill, chat-completions and images modules; tokenizer_cache 2 passed), the bindings clippy with --all-targets, and test_preprocessed_request_conversion.py (14 passed on a wheel built from the merged tree). A fortieth merge commit 7ba3ab340c absorbs upstream main as of 24c7f0c38e (2 commits, two conflicts): #18247 Kimi K3 prompt controls in the SGLang processor (Python frontend only) and #18345 the Anthropic streaming and Qwen grammar fixes (dynamo-renderer 10.1.0, dynamo-protocols 10.0.1, dynamo-parsers 11.0.11, dynamo-parsers-v2 0.10.1; lib/llm Anthropic stream converter and types, OpenAI validation with two new tests, the chat-completions parsers and aggregators, perf/logprobs.rs, preprocessor.rs, discovery/watcher.rs, and the generate-format tests), all outside the router. The conflicts are the root Cargo.toml pin hunk again (upstream's dynamo-protocols pin kept beside the stack's path crates) and two hunks in lib/bindings/kvbm/Cargo.lock whose upstream side was empty beside the stack's kv-crate entries (kept, then the four lockfiles regenerated with no drift). Validated with the slim chain (27 min, every stage 0 except the expected kvbm-consolidator manifest failure) and upstream's lib/llm tests by name (validate and the Anthropic and chat-completions modules 478 passed including both new validation tests, aggregators 36 including the new v2-parser test, logprob_analysis_integration 6, postprocessor_parsing_stream 117). A forty-first merge commit b4053467aa absorbs upstream main as of 72ca6717d9 (2 commits, one conflict): #17889 generic candidate policy hooks (lib/llm only: routing_policy.rs gains the crate-private HostedCandidatePolicy and KvAdmissionPolicy traits, the new lora_routing.rs holds LegacyLoraPolicy and LegacyLoraAdmission with the load lease, the routing host's lora field becomes candidate_policy and the request guard's load guard an _attempt_lease; both new files import SchedulingRequest, ScheduleMode and KvSchedulerError from dynamo_scheduler::host, HostedWorkerPicker from dynamo_worker_selection::host and WorkerWithDpRank from dynamo_router_protocol::worker instead of the pre-split dynamo_kv_router paths; both traits take lib/llm's PreprocessedRequest and RuntimeConfigWatch, so nothing moves into a router crate and the plugin prelude is unchanged) and #18490 one docs page. The conflict is routing_host/hosted.rs, where upstream deleted select_lora_target and the stack had added .map_err(KvSchedulerError::from) to its picker call (the owner crate's HostedWorkerPicker::select returns SelectionFailure); the deletion is taken and the mapping travels with the code into lora_routing.rs. Validated with the slim chain (22 min, every stage 0 except the expected kvbm-consolidator manifest failure, 2,670 tests), the routing-host test module by name (79 passed including the extended builtin_lora_keeps_separate_selection_and_cleanup), upstream's two new lora_routing tests, and cargo check -p dynamo-kv-router --all-features for the compatibility paths. A forty-second merge commit 984b3646b3 absorbs upstream main as of 02235ef5cb (1 commit, no conflicts): #17891 runtime LoRA placement policy (lib/lora only: the new policy.rs validates the router mode for LoRA serving and places a versioned request-time adapter over RuntimeLoraWorkerFacts + WorkerConfigLike under RoutingConstraints, filter.rs gains filter_worker_ids_for_runtime_lora; lib/lora is a kernel host, so the new policy.rs import of dynamo_kv_router::{WorkerConfigLike, protocols::RoutingConstraints} is re-pointed to dynamo_scheduler::host, which already re-exports both, and nothing moves into a router crate). Validated at Tier B: cargo check --workspace --all-targets, cargo test -p dynamo-lora --all-targets (178 + 19 passed, 1 ignored) with upstream's seven new tests by name, cargo clippy -p dynamo-lora --all-targets -- -D warnings, cargo check -p dynamo-kv-router --all-features for the compatibility paths, the layering gates (36 / 0, --host 55 / 0), pre-commit over the round's files and origin/main..HEAD, and the four lockfiles --locked. A forty-third merge commit a9b3636e5c absorbs upstream main as of 3a524d0d70 (1 commit, 9 conflicts): #17895 safe runtime LoRA routing (the kernel's new AccountingAttribution enum and its threading through ScheduleRequest, SchedulingRequest, SelectionOperation, PendingSelection and the reservation booking land whole in lib/router/scheduler, re-exported through dynamo_scheduler::host and the plugin prelude, with the 1.7 path dynamo_kv_router::scheduling::AccountingAttribution kept in compat.rs; lib/llm's routing_policy.rs, lora_routing.rs, kv_router.rs and routing_host/{hosted,kv_selection}.rs and lib/lora's new tests/accounting.rs keep upstream's content with their dynamo_kv_router imports re-pointed to dynamo_scheduler::host, dynamo_worker_selection::host, dynamo_slot_tracker::host and dynamo_router_protocol::worker, plus the stack's KvSchedulerError mapping on the hosted picker call in the new LoraHostedPolicy; the stack-only queue_yaml_tests.rs literal and upstream's new kernel test import were the two fixes upstream could not see; the example policy crates, builtin policies, mocker and the kv-router bench take the new field via Default::default()). Validated at Tier C: slim chain 24 min 43 s (09:28:48 to 09:53:31) with every stage 0 except the expected kvbm-consolidator manifest failure, 2,689 tests passed / 0 failed over 22 suites (router crates 1,351 incl. the scheduler's 356, kv-router 251 + 2 + 7 and the prelude test 2, llm kv_router 517, discovery 283, pool_relay 269, lora_routing 7), then by name: selection core 99 (incl. upstream's explicit_reservation_preserves_legacy_adapter_attribution and accounting_attribution_preserves_routing_identity_and_reservation_replay), kernel selector/types/pending 60, cargo test -p dynamo-lora --all-targets 181 + 1 + 19 (1 ignored) with upstream's three runtime::tests and tests/accounting.rs by name, llm kv_router::routing_host 123 and upstream's fourteen new lib/llm tests by name (routing host, hosted, kv_selection, builder, activation, entrypoint, migration) plus the six lora_routing tests, the two example policy crates 3 + 2, dynamo-router-policies 71 + 5 + 1 + 3 + 20 + 3, mocker replay 184, cargo check -p dynamo-kv-router --all-features 0 for the compatibility paths; the layering gates (36 / 0, --host 55 / 0), codeowners strict, fmt in all four workspaces, machete, pre-commit 10 / 0 / 30 on the round's 46 files and 22 / 0 / 18 over origin/main..HEAD, and the four lockfiles --locked with no drift. A forty-fourth merge commit c4c5e8cebc absorbs upstream main as of 1a08f91b25 (1 commit, no conflicts): the automated docs: refresh community events data refresh of one line in docs/fern/components/events.generated.ts, no PR number, no router-owned path; Tier A: cargo check --workspace --all-targets 0, the layering gates (36 / 0, --host 55 / 0), pre-commit 12 / 0 / 28 on the round's file and 22 / 0 / 18 over origin/main..HEAD, and the four lockfiles --locked with no drift. A forty-fifth merge commit 455fd49eca absorbs upstream main as of 6b2b25cddc (1 commit, no conflicts): #17190 expands the operator's kube-api-linter to the full API conventions, 34 files all under deploy/operator/ (.golangci-kal.yml, Dockerfile, Makefile, the api/v1alpha1 and api/v1beta1 Go types and conversions, the roundtrip fuzz tests, the CRD bases YAML, api/scripts/generate_pydantic_from_go.py) plus the two generated API reference docs pages, taken whole; no router-owned path, no Rust; Tier A: cargo check --workspace --all-targets 0, the layering gates (36 / 0, --host 55 / 0), pre-commit 17 / 0 / 23 on the round's 34 files and 22 / 0 / 18 over origin/main..HEAD, and the four lockfiles --locked with no drift. A forty-sixth merge commit f24e0fa464 absorbs upstream main as of 4b1a320423 (1 commit, no conflicts): #17943 drops the cache_transceiver_config key from the LLaVA TRT-LLM aggregated engine config, one file examples/backends/trtllm/engine_configs/llava-v1.6-mistral-7b-hf/agg.yaml (+1 / -2), taken whole; no router-owned path, no Rust, no Python; Tier A: cargo check --workspace --all-targets 0, the layering gates (36 / 0, --host 55 / 0), pre-commit 11 / 0 / 29 on the round's file and 22 / 0 / 18 over origin/main..HEAD, and the four lockfiles --locked with no drift. A forty-seventh merge commit eb8e9cde4c absorbs upstream main as of f42e5b1fdf (4 commits, one conflict): #18384 admits sequential disagg decode with transferred prompt KV in the lib/llm mocker frontend (lib/llm/src/mocker.rs, auto-merged; the LiveEngine::submit_decode handoff it calls stays in lib/mocker); #18199 keeps mm-routing opt-in for the default wheel (the sglang and trtllm sidecars gain a [features] table with default = ["mm-routing"], the bindings pass default-features = false to both and the wheel's mm-routing feature enables theirs; the one conflict, lib/sidecar/trtllm/Cargo.toml, resolved as upstream's features block and bare dynamo-llm line with the split's dynamo-kv-events dependency kept in place of the pre-split dynamo-kv-router edge); #18351 moves Planner tests to their own CI lane (two workflows, container/Dockerfile.test, pytest.mark.planner or core on seven test files); #18429 keeps declared Responses tools in the prompt under allowed_tools (three lib/llm files). No router-owned path, no lockfile change; Tier C: chain v2 full on the final tree (every stage 0, 2,682 tests passed, gate 36 / 0 and --host 55 / 0), the four upstream tests by name, both sidecars with and without default features, the default wheel resolving no dynamo-llm/mm-routing edge, pre-commit 17 / 0 / 23 on the round's 17 files and 22 / 0 / 18 over origin/main..HEAD, and the four lockfiles --locked with no drift. A forty-eighth merge commit 4df5cd20dc absorbs upstream main as of c2a6b68bc7 (2 commits, no conflicts): #18425 forwards logit_bias from the SGLang chat processor (components/src/dynamo/frontend/sglang_processor.py converts the JSON's str keys to the int-keyed PreprocessedRequest map; components/src/dynamo/sglang/request_handlers/llm/decode_handler.py stringifies them again for SGLang's msgpack IPC; one new frontend unit test, one decode-handler assertion update); #18316 makes the GMS V1 SGLang plugin honor disabled memory-saver requests (lib/gpu_memory_service/v1/integrations/sglang/plugin.py defers to the original TorchMemorySaverAdapter.create when enable is false, plus its test). Python only, taken whole; no router-owned path, no Rust, no lockfile change; Tier A plus the touched Python tests: cargo check --workspace --all-targets 0, the layering gates (36 / 0, --host 55 / 0), pre-commit 14 / 0 / 26 on the round's six files and 22 / 0 / 18 over origin/main..HEAD, the four lockfiles --locked with no drift, and the three touched pytest files (783 passed, including the round's six tests by name) in the worktree venv with sglang 0.5.23. A forty-ninth merge commit 37231331e6 absorbs upstream main as of 8510b90261 (1 commit, no conflicts): #18465 returns 400 for media modalities the frontend cannot decode (lib/llm/src/preprocessor/media/loader.rs: a missing image, video, or audio decoder, the media-ffmpeg feature being off, or an unknown content part type now maps to InvalidArgument with a fixed public message instead of an unclassified 500; new test test_unsupported_media_is_invalid_argument). Rust in dynamo-llm's media preprocessor only, taken whole; no router-owned path, no lockfile change; Tier A plus the round's test: cargo check --workspace --all-targets 0, cargo test -p dynamo-llm --lib preprocessor::media::loader 29 / 0 including the new test by name, the layering gates (36 / 0, --host 55 / 0), pre-commit 10 / 0 / 30 on the round's file and 22 / 0 / 18 over origin/main..HEAD, and the four lockfiles --locked with no drift. A fiftieth merge commit 046e51ad24 absorbs upstream main as of 227196a725 (2 commits, no conflicts): #18367 drops the trailing dot before the PodCliqueSet name hash (deploy/operator/internal/dynamo/graph.go, with the case added to graph_test.go), and #18188 makes component, disaggregated-set and Grove restart progress wait for the requested restart revision (deploy/operator/internal/controller: the dgd_*_restart_progress_resolver.go resolvers and dgd_grove_program.go, plus controller and envtest coverage; one line in the dynamo-graph-deployment.mdx Kubernetes API page). Go operator code only, taken whole; no router-owned path, no Rust, no Python, no lockfile change; Tier A: cargo check --workspace --all-targets 0, the layering gates (36 / 0, --host 55 / 0), pre-commit 11 / 0 / 29 on the round's 14 files and 22 / 0 / 18 over origin/main..HEAD, and the four lockfiles --locked with no drift. A fifty-first merge commit 21cab08d6c absorbs upstream main as of 104e488026 (6 commits, no conflicts): #18464 records public Responses tool names and namespaces in the request trace (lib/llm/src/request_trace, http/service/openai.rs, protocols/openai/responses/tool_names.rs), #18321 reports queued requests in vLLM FPM heartbeats during long passes (components/src/dynamo/common/forward_pass_metrics.py, vllm/instrumented_scheduler.py, the new test_vllm_instrumented_scheduler.py), #18404, #18405 and #18406 read, detokenize and edit the response items of Migration, Backend and the preprocessor in place (lib/llm/src/migration.rs, backend.rs, preprocessor.rs, the new preprocessor/edit.rs and tests/test_streaming_usage.rs), and #18416 rejects unsupported modalities before decoding in the vLLM multimodal request processor (vllm/multimodal_utils/request_processor.py and its test). All taken whole; the split's re-pointed imports in openai.rs, migration.rs, preprocessor.rs and request_trace/mod.rs survive the auto-merge; no router-owned path, no Cargo.toml or lockfile change. Tier C slim: chain v2 (router crates, kv-router, workspace clippy, hosts, bindings, cargo hack, gates 36 / 0 and --host 55 / 0, codeowners, fmt, machete, four lockfiles) 2,682 passed / 0 failed, cargo check --workspace --all-targets 0, by name test_streaming_usage 15, migration 67, backend 117, preprocessor 417, request_trace 88, http::service::openai 171, cargo check -p dynamo-llm --features mm-routing --all-targets 0, pre-commit 15 / 0 / 25 on the round's 18 files and 22 / 0 / 18 over origin/main..HEAD; in the worktree venv test_recv_forward_pass_metrics.py 14 / 0 (the two new test_vllm_* files need vLLM, which that venv does not carry). A fifty-second merge commit be6d22917e then absorbs upstream main as of 33cfa1271c (3 commits: #18520 the runtime Controller's pending stop or kill wait made cheap to re-poll (lib/runtime/src/pipeline/context.rs: an AtomicU8 state plus a Notify replace the watch pair, two new tests), #18426 logit_bias forwarded in SGLang multimodal E/PD and E/P/D requests (components/src/dynamo/sglang/protocol.py, request_handlers/multimodal/worker_handler.py, one new parametrized test in test_sglang_decode_handler.py), #17924 the TRT-LLM runtime image's system-site datasets, fsspec, filelock and pyasn1 refresh (container/templates/trtllm_runtime.Dockerfile)). All taken whole and byte-identical to main (the split touches none of the five files); no conflicts, no router-owned path, no Cargo.toml or lockfile change. Tier C slim: chain v2 2,682 passed / 0 failed with gates 36 / 0 and --host 55 / 0, cargo check --workspace --all-targets 0 before and after folding the two later commits, by name cargo test -p dynamo-runtime --lib pipeline::context 11 and pipeline 335 with both new tests ok under --exact, pre-commit 14 / 0 / 26 on the round's 5 files and 22 / 0 / 18 over origin/main..HEAD, four lockfiles --locked 0 with no drift; in the worktree venv (sglang 0.5.23) test_sglang_decode_handler.py 280 / 0 with the new logit_bias test 6 / 0 by name.

Unit Fork PR Scope Commits
01 ishandhanani#174 dynamo-router-protocol, the wire vocabulary crate, and the fail-closed layering gate 1
02 ishandhanani#175 the hash contract to dynamo-kv-hashing 1
03 ishandhanani#176 dynamo-kv-events: event types, the engine ZMQ codec, the cursor 1
04 ishandhanani#177 dynamo-zmq, the socket helper (folded back in unit 20) 1
05 ishandhanani#178 dynamo-kv-indexer in four slices 5
06 ishandhanani#179 WorkerCapacity to protocol and dynamo-slot-tracker 2
07 ishandhanani#180 dynamo-session-affinity 1
08 ishandhanani#181 open the worker-selection stage and move it to lib/router/worker-selection 6
09 ishandhanani#182 open the admission stage and move it to lib/router/admission 5
10 ishandhanani#183 the scheduling kernel and the selection core to lib/router/scheduler 10
11 ishandhanani#184 thin kv-router to the composition crate with adapter features and plugins::prelude 4
12 ishandhanani#185 every host imports the owning crates through stage host modules 7
13 ishandhanani#186 kv-router imports the owners inside and deprecates its shim paths 3
14 ishandhanani#187 layering gate v2 in CI 2
15 ishandhanani#188 the error contract to dynamo-error 2
16 ishandhanani#189 protocol cleanup: load, hints, residency, candidates, multimodal, and ActiveLoad to their owners (unit 16 introduced dynamo-multimodal; a later commit, 1e45e0395f, folded it into dynamo-kv-hashing as the multimodal module after the crates.io name collision was found) 7
17 ishandhanani#190 remove the deprecated shims and the kernel's foreign re-exports 2
18 ishandhanani#194 trim the migration guardrails: plugin crates depend on kv-router alone, snapshots dropped, gate slimmed to edges 4
19 ishandhanani#196 protocol narrows: RouterHintSourceMetadata to kv-indexer, WorkerAffinityTarget to session-affinity 2
20 ishandhanani#198 fold dynamo-zmq back into kv-router behind its ZMQ features 1

The merge commit absorbs these router PRs, each placed in the crate that owns the code at the stack head:

Validation

Chain v2 in full mode on the merged head (/tmp/rr-split/chain-v2.sh, MODE=full, LLM_EXTRA_TESTS="discovery pool_relay"): 40 min 27 s wall, every stage exit 0 except the pre-existing kvbm-consolidator manifest failure the chain expects.

  • Stage 1, the 12 router crates (1 min 53 s): clippy -D warnings under default and all features, cargo test --all-features 1,277 passed / 5 ignored (kv-indexer 609, scheduler 296, slot-tracker 131, kv-events 69, worker-selection 54, kv-hashing 19 + 17 + 3, error 29, admission 22, session-affinity 10, protocol 8, kv-hints 4, multimodal 4), rustdoc -D warnings with all features.
  • Stage 2, kv-router (1 min 21 s): clippy with http,ingest-zmq,replica-sync-zmq,peer-recovery,metrics,testing, the bench-feature bench check, 235 tests (226 + 2 + 7) plus the prelude integration test (2).
  • Stage 3, cargo clippy --workspace --all-targets -D warnings (4 min 41 s): clean.
  • Stage 4, hosts (30 min 17 s): dynamo-llm lib alone; its kv_router 493, discovery 281, pool_relay 264 tests; clippy --tests per host for the 13 host crates (mocker, bench, ext-proc, router-policies, policy catalog, SGLang and vLLM sidecars, backend-common, SGLang and vLLM mockers, the EPP and soft-pin-repin examples, libdynamo_llm); the Python bindings crate with kv-indexer,slot-tracker,select-service.
  • Stage 5, gates (2 min 15 s): cargo hack check --each-feature over kv-router's features (then wrapped in router_features_check.sh, now the inline line in rust-feature-builds), router_layering_check.sh 38 checks / 0 failures and --host 57 checks / 0 failures, codeowners 7711/7711 explicitly owned, cargo fmt --check, the three lockfiles --locked.

Upstream tests re-run by name in their new crates:

pre-commit run --from-ref origin/main --to-ref HEAD on the current head: 20 hooks passed, 20 skipped for having no applicable files, 0 failed, no hook-made edits (the router-layering hook is gone, see Review follow-ups).

Size against main at 182039dbb5: 563 files changed, 26,158 insertions, 20,211 deletions (git diff --shortstat origin/main...HEAD); 77 .rs files added, 12 deleted, 254 modified. Moved-versus-new text over every .rs file changed (scripts/router_move_diff.py origin/main HEAD, 266 source files deleted or modified against 331 added or modified): 199,955 normalized lines before, 205,423 after, 198,436 common (96.6% of the resulting text is moved verbatim), 6,987 new (3.4%: the hosted picker port, the bucketed transfer summary, the relocated and new tests, the prelude and host lists, the gate script), 1,519 dropped (0.76% of the base: the deleted shim modules and the dead SchedulingPolicy and dummy shared-cache server).

CI status

Pre-existing on upstream main, not caused by this PR: planner / CPU Test (amd64 and arm64) and dynamo-runtime / test / sequential (amd64 and arm64) fail on main's own commits 182039dbb5 and 73014901d4 with the same three tests, test_live_retune_frontend_config.py::test_effective_argv[command7-args7-power_of_two] and ::test_worker_router_mode_matches_the_real_worker_parser[args3-env3] (both because components/src/dynamo/common/configuration/groups/router_args.py no longer lists power-of-two while the planner test still passes it) and test_frontend_parser_builder.py::test_no_registered_env_var_is_read_from_os_environ. Under components/ this PR changes only path strings that follow the moves (one test fixture path, three docstring or comment references), none of the planner or parser code those tests exercise. Also pre-existing on main: #18405's new backend::tests::backend_output_is_the_same_when_the_engine_is_pending_between_chunks is nondeterministic (14 of 20 solo runs fail on this head; lib/llm/src/backend.rs is byte-identical to main): the bare-EOF flush iterates decoders: HashMap<u32, Decoder>, so the two synthetic per-choice STOP chunks come out in hash order and the test compares two independent runs.

Head be6d22917e is the fifty-second upstream merge (upstream main 33cfa1271c, 3 commits: #18520 the runtime Controller's pending stop or kill wait made cheap to re-poll (lib/runtime/src/pipeline/context.rs, AtomicU8 state plus Notify, two new tests), #18426 logit_bias forwarded in SGLang multimodal E/PD and E/P/D requests (sglang/protocol.py, multimodal/worker_handler.py, one new parametrized test), #17924 TRT-LLM runtime image system-site dependency refresh (trtllm_runtime.Dockerfile); no conflicts, no router-owned path, no Cargo.toml or lockfile change, all five files byte-identical to main; Tier C slim: chain v2 2,682 passed / 0 failed with gates 36 / 0 and --host 55 / 0, cargo check --workspace --all-targets 0 before and after folding the two later commits, by name cargo test -p dynamo-runtime --lib pipeline::context 11 and pipeline 335 with both new tests --exact, pre-commit 14 / 0 / 26 on the round's 5 files and 22 / 0 / 18 over origin/main..HEAD, four lockfiles --locked 0 with no drift, test_sglang_decode_handler.py 280 / 0 in the worktree venv with the new logit_bias test 6 / 0 by name), on top of 21cab08d6c, the fifty-first upstream merge (upstream main 104e488026, 6 commits: #18464 request-trace public Responses tool names and namespaces (lib/llm/src/request_trace, http/service/openai.rs, tool_names.rs), #18321 queued requests in vLLM FPM heartbeats (forward_pass_metrics.py, instrumented_scheduler.py, new test), #18404 / #18405 / #18406 in-place response items in Migration, Backend and the preprocessor (migration.rs, backend.rs, preprocessor.rs, new preprocessor/edit.rs and tests/test_streaming_usage.rs), #18416 unsupported modalities rejected before decoding (vllm/multimodal_utils/request_processor.py and its test); no conflicts, no router-owned path, no Cargo.toml or lockfile change, the split's re-pointed imports intact; Tier C slim: chain v2 2,682 passed / 0 failed with gates 36 / 0 and --host 55 / 0, cargo check --workspace --all-targets 0, by name test_streaming_usage 15, migration 67, backend 117, preprocessor 417, request_trace 88, http::service::openai 171, cargo check -p dynamo-llm --features mm-routing --all-targets 0, pre-commit 15 / 0 / 25 on the round's 18 files and 22 / 0 / 18 over origin/main..HEAD, four lockfiles --locked 0 with no drift, test_recv_forward_pass_metrics.py 14 / 0 in the worktree venv), on top of 046e51ad24, the fiftieth upstream merge (upstream main 227196a725, 2 commits: #18367 the PodCliqueSet name hash no longer follows a trailing dot (deploy/operator/internal/dynamo/graph.go, one test case), #18188 component, disaggregated-set and Grove restart progress waits for the requested restart revision (deploy/operator/internal/controller restart progress resolvers and Grove program, controller and envtest tests, one line in the dynamo-graph-deployment.mdx Kubernetes API page); no conflicts, no router-owned path, no Rust, no Python, no lockfile change; Tier A: cargo check --workspace --all-targets 0, gate 36 / 0 and --host 55 / 0, pre-commit 11 / 0 / 29 on the round's 14 files and 22 / 0 / 18 over origin/main..HEAD, four lockfiles --locked 0 with no drift), on top of 37231331e6, the forty-ninth upstream merge (upstream main 8510b90261, 1 commit: #18465 media modalities the frontend cannot decode answered with 400 (lib/llm/src/preprocessor/media/loader.rs, one new test); no conflicts, no router-owned path, no lockfile change; Tier A plus the round's test: cargo check --workspace --all-targets 0, cargo test -p dynamo-llm --lib preprocessor::media::loader 29 / 0 with test_unsupported_media_is_invalid_argument ok, gate 36 / 0 and --host 55 / 0, pre-commit 10 / 0 / 30 on the round's file and 22 / 0 / 18 over origin/main..HEAD, four lockfiles --locked 0 with no drift), on top of 4df5cd20dc, the forty-eighth upstream merge (upstream main c2a6b68bc7, 2 commits: #18425 logit_bias forwarded from the SGLang chat processor to the decode handler (frontend sglang_processor.py, SGLang decode_handler.py, two tests), #18316 the GMS V1 SGLang plugin honors disabled memory-saver requests (lib/gpu_memory_service plugin and test); no conflicts, no router-owned path, no Rust, no lockfile change; Tier A plus the touched Python tests: cargo check --workspace --all-targets 0, gate 36 / 0 and --host 55 / 0, pre-commit 14 / 0 / 26 on the round's 6 files and 22 / 0 / 18 over origin/main..HEAD, four lockfiles --locked 0 with no drift, the three touched pytest files 783 / 0 in the worktree venv with sglang 0.5.23 and the round's six tests by name), on top of eb8e9cde4c, the forty-seventh upstream merge (upstream main f42e5b1fdf, 4 commits: #18384 sequential disagg decode admission in the lib/llm mocker frontend, #18199 mm-routing opt-in for the default wheel (sidecar [features] tables, default-features = false in the bindings), #18351 Planner tests moved to their own CI lane, #18429 declared Responses tools kept in the prompt under allowed_tools; one conflict, lib/sidecar/trtllm/Cargo.toml, resolved keeping the split's dynamo-kv-events edge under upstream's new features block; no router-owned path, no lockfile change; Tier C: chain v2 full on the final tree every stage 0 with 2,682 tests passed, gate 36 / 0 and --host 55 / 0, the four upstream tests by name, sidecars with and without default features, cargo check --workspace --all-targets 0, pre-commit 17 / 0 / 23 on the round's 17 files and 22 / 0 / 18 over origin/main..HEAD, four lockfiles --locked 0 with no drift), on top of f24e0fa464, the forty-sixth upstream merge (upstream main 4b1a320423, 1 commit: #17943 drops the cache_transceiver_config key from the LLaVA TRT-LLM aggregated engine config, one file examples/backends/trtllm/engine_configs/llava-v1.6-mistral-7b-hf/agg.yaml (+1 / -2), taken whole; no conflicts, no router-owned path, no Rust, no Python; Tier A: cargo check --workspace --all-targets 0, gate 36 / 0 and --host 55 / 0, pre-commit 11 / 0 / 29 on the round's file and 22 / 0 / 18 over origin/main..HEAD, four lockfiles --locked 0 with no drift), on top of 455fd49eca, the forty-fifth upstream merge (upstream main 6b2b25cddc, 1 commit: #17190 kube-api-linter expanded to the full API conventions, 34 files all under deploy/operator/ (.golangci-kal.yml, Dockerfile, Makefile, the api/v1alpha1 and api/v1beta1 Go types and conversions, the roundtrip fuzz tests, the CRD bases YAML, api/scripts/generate_pydantic_from_go.py) plus the two generated API reference docs pages, taken whole; no conflicts, no router-owned path, no Rust; Tier A: cargo check --workspace --all-targets 0, gate 36 / 0 and --host 55 / 0, pre-commit 17 / 0 / 23 on the round's 34 files and 22 / 0 / 18 over origin/main..HEAD, four lockfiles --locked 0 with no drift), on top of c4c5e8cebc, the forty-fourth upstream merge (upstream main 1a08f91b25, 1 commit: the automated docs: refresh community events data refresh of one line in docs/fern/components/events.generated.ts, no PR number; no conflicts, no router-owned path; Tier A: cargo check --workspace --all-targets 0, gate 36 / 0 and --host 55 / 0, pre-commit 12 / 0 / 28 on the round's file and 22 / 0 / 18 over origin/main..HEAD, four lockfiles --locked 0 with no drift), on top of a9b3636e5c, the forty-third upstream merge (upstream main 3a524d0d70, 1 commit: #17895 safe runtime LoRA routing; 9 conflicts resolved into the kernel (AccountingAttribution and its threading, whole, in lib/router/scheduler, re-exported via dynamo_scheduler::host, the prelude and a compat.rs line for the 1.7 path) and into lib/llm and lib/lora with owner-crate re-points only, plus the stack's KvSchedulerError mapping on the new LoraHostedPolicy's picker call; Tier C: slim chain 24 min 43 s with every stage 0 except the expected kvbm-consolidator, 2,689 tests passed / 0 failed, selection core 99 and kernel selector/types/pending 60 by name, lora 181 + 1 + 19 with upstream's four new tests by name, llm routing host 123, upstream's fourteen lib/llm tests and six lora_routing tests by name, example crates 3 + 2, router-policies 103, mocker replay 184, cargo check -p dynamo-kv-router --all-features 0, gate 36 / 0 and --host 55 / 0, pre-commit 10 / 0 / 30 and 22 / 0 / 18, four lockfiles --locked 0 with no drift), on top of 984b3646b3, the forty-second upstream merge (upstream main 02235ef5cb, 1 commit: #17891 runtime LoRA placement policy, lib/lora only, no conflicts; the new policy.rs import of dynamo_kv_router::{WorkerConfigLike, protocols::RoutingConstraints} is re-pointed to dynamo_scheduler::host, which already re-exports both, because lib/lora has no dynamo-kv-router edge after the split; Tier B: cargo check --workspace --all-targets 0, cargo test -p dynamo-lora --all-targets 178 + 19 passed / 1 ignored / 0 failed with upstream's seven new policy::tests and filter::tests by name (17 passed), cargo clippy -p dynamo-lora --all-targets -- -D warnings 0, cargo check -p dynamo-kv-router --all-features 0, gate 36 / 0 and --host 55 / 0, pre-commit 10 / 0 / 30 on the round's files and 22 / 0 / 18 over origin/main..HEAD, four lockfiles --locked 0 with no drift), on top of b4053467aa, the forty-first upstream merge (upstream main 72ca6717d9, 2 commits: #17889 generic candidate policy hooks, lib/llm only, with the routing_host/hosted.rs conflict resolved by taking upstream's removal of select_lora_target and carrying the stack's KvSchedulerError mapping into the new lora_routing.rs, whose imports and routing_policy.rs's point at dynamo_scheduler::host, dynamo_worker_selection::host and dynamo_router_protocol::worker, and #18490 one docs page; Tier C: slim chain 22 min 14 s with every stage 0 except the expected kvbm-consolidator, 2,670 tests passed / 0 failed, routing-host tests 79 by name including the extended builtin_lora_keeps_separate_selection_and_cleanup, lora_routing 2, cargo check -p dynamo-kv-router --all-features 0, gate 36 / 0 and --host 55 / 0, pre-commit 11 / 0 / 29 on the round's files and 22 / 0 / 18 over origin/main..HEAD, four lockfiles --locked 0), on top of 5d7ada5f65, the compatibility-paths commit (one commit, 9 files; see Compatibility paths above; slim chain 30 min 19 s with every stage 0 except the expected kvbm-consolidator, 2,668 tests passed / 0 failed, gate 36 / 0 and --host 55 / 0, cargo hack --each-feature 0, out-of-workspace probes 797 / 858 / 903 paths, pre-commit on the 9 files) on top of 7ba3ab340c, the fortieth upstream merge (upstream main 24c7f0c38e, 2 commits: #18247 Kimi K3 prompt controls, Python only, and #18345 dependency bumps to dynamo-renderer 10.1.0 / dynamo-protocols 10.0.1 / dynamo-parsers 11.0.11 / dynamo-parsers-v2 0.10.1 with lib/llm protocol code; root Cargo.toml pin hunk and two empty-upstream hunks in the kvbm lockfile resolved, four lockfiles regenerated with no drift; Tier C: slim chain 27 min with every stage 0 except the expected kvbm-consolidator, kv-router 251 + 2 + 7, llm kv_router 505, lib/llm by name 478 + 36 + 6 + 117, gate 36 / 0 and --host 55 / 0, pre-commit 22 / 0 / 18, four lockfiles --locked 0), on top of the thirty-ninth merge 19d3753420 (upstream 3a8226a768, 19 commits: #17923 block-lookup growth fix in lib/router/kv-indexer, #17741 dynamo-tokenizers 4.1.2 / dynamo-renderer 9.0.0 / tokenizers 0.23.2, #17520 dynamo-protocols 8.5.0, #18380 and #18448 bindings py_convert.rs, and lib/llm, lib/runtime, Python, recipes and docs changes; one root Cargo.toml conflict resolved as upstream's pins beside the stack's path crates, four lockfiles regenerated with no drift; Tier C: slim chain 40 min with every stage 0 except the expected kvbm-consolidator, kv-indexer 613 and CRTC 70 by name, kv-router 251 + 2 + 7, llm kv_router 505 plus 514 named lib/llm tests and tokenizer_cache 2, gate 36 / 0 and --host 55 / 0, pre-commit 22 / 0 / 18, four lockfiles --locked 0, test_preprocessed_request_conversion.py 14 passed on a Modal-built wheel), on top of the thirty-eighth merge 281cf248b5 (upstream 02a6ccbb0e, #18281 worker-selection policies written inline per role: the new WorkerSelectionPolicySelection enum, per-role parser and bare-type or inline registry resolution all kept in lib/router/kv-router beside WorkerSelectionConfig; four import or test-helper conflicts resolved to the stack's owner-crate imports plus one semantic conflict (&SelectionConfig::from(&config) in upstream's new registry test); Tier C: slim chain 28 min with every stage 0 except the expected kvbm-consolidator, kv-router 251 + 2 + 7, router-policies 110, llm kv_router 505, gate 36 / 0 and --host 55 / 0, pre-commit 22 / 0 / 18, four lockfiles --locked 0), on top of the thirty-seventh merge 8d6d7dffe6 (upstream b977beb708, the automated docs: refresh community events data refresh of one line in docs/fern/components/events.generated.ts, no PR number; no conflicts), on top of the thirty-sixth merge eb3e41ca84 (upstream 27f6052cd5, #18302 split tool-call openers preserved after reasoning: components/src/dynamo/frontend/prepost.py holds a partial tool-start marker after a reasoning end in _pending_tool_opener and prepends it to the next delta, with the parametrized test_hermes_split_tool_opener_at_reasoning_end in tests/test_vllm_processor_unit.py; Python only, no router-owned path, no conflicts; Tier A: cargo check --workspace --all-targets 0, gate 36 / 0 and --host 55 / 0, pre-commit 22 / 0 / 18, four lockfiles --locked 0), on top of the thirty-fifth merge 84de392d42 (upstream 05ee045712, #17893 runtime LoRA adapter identity: lib/lora/src/runtime.rs taken whole with its WorkerConfigLike and RoutingConstraints imports re-pointed to dynamo_scheduler::host and WorkerId to dynamo_router_protocol::worker, lib/lora/Cargo.toml gaining the dynamo-scheduler host edge, three lockfiles regenerated; no textual conflicts, one semantic conflict; cargo test -p dynamo-lora 171 + 19 passed with all 14 new tests by name, cargo test -p dynamo-llm kv_router 505 passed), on top of the thirty-fourth merge bdc8812aab (upstream 1dd1eefd9d, #18053 SGLang reasoning gate advertised for multimodal LLM workers in components/src/dynamo/sglang, Python only; no conflicts), on top of the thirty-third merge eccdcd1ffb (upstream 335d2f6a18, #17730 Harbor and Substrate agents tutorial: new examples/agents/harbor-substrate/ tree, docs pages and assets, areas.yaml agents glob with the generated CODEOWNERS regenerated byte-identical; no conflicts), on top of the thirty-second merge 8536781e4a (upstream 4948d19a4a, the automated docs: refresh community events calendar refresh in docs/fern/components/events.generated.ts, one line; no conflicts), on top of the thirty-first merge 5e36cd5bd6 (upstream 51a6b3f2f3, #18353 deploy tests ride out 60 s vCluster tunnel restarts in tests/deploy/{dgdr_utils,test_dgdr_utils,vcluster_utils}.py with two new tests; no conflicts), on top of the thirtieth merge 079928f7ed (upstream 403ead2f65, #18311 TRT-LLM guided-decoding grammar rejections returned as 400 in components/src/dynamo/trtllm with two new tests, and #18324 Responses API echoing description: null on function tools in lib/llm/src/protocols/openai/responses with two new tests; no conflicts), on top of the twenty-ninth merge e69788f9d2 (upstream 58be22021c, #18295 Worker registration fetches HF models without weight files in lib/backend-common/src/worker.rs with one new test; no conflicts), on top of the twenty-eighth merge dfe78dab37 (upstream 16c7f5d176, #18198 router replicas registered through global discovery in the lib/global-* crates with two new integration test files, and #17227 TRT-LLM container images on the upstream NIXL stack; no conflicts), on top of the twenty-seventh merge 0a5745b148 (upstream baba4997ea, #17517 metrics review prompt, review config and a lib/runtime module doc comment, no Rust code, no conflicts), the twenty-sixth merge e361e9607b (upstream 1d6e9e0846, #18338 dynamo-runtime parallel CPU-only test timeout, one workflow file, no conflicts), the twenty-fifth merge 58d4775769 (upstream dec6c2200e, #17751 planner sensitivity docs, six docs pages, no conflicts), the twenty-fourth merge 6fd5106465 (upstream ef8d3b1a0b, #18072 LoRA S3 TLS knob, #17250 EPP tokenization off the async workers with stage metrics, #18003 container bumps; two import-only conflicts in ext-proc epp.rs and server.rs resolved by keeping the stack's owner-crate imports beside upstream's new ones; slim chain green, cargo test -p dynamo-ext-proc 192 + 1 passed including all 23 new and 2 renamed or rewritten tests, cargo test -p dynamo-lora 157 + 19 passed including the three new tests), the twenty-third merge 791179e68f (upstream 5c17fb8ab6, #15856 recipe lifecycle badges, docs/recipes catalog and CI filters only, no conflicts), the twenty-second merge 9bcad0cde3 (upstream ad8ecd2e5d, #18293 SGLang MultimodalProcessorOutput import fix, one Python file, no conflicts), the twenty-first merge 0ac05d1d90 (upstream 2680b5064d, #18252 muse parser alias deprecation, lib/llm parser code plus frontend Python and docs, no conflicts), the twentieth merge bb61c38da3 (upstream 7d8eab2b94, #17575 Kubernetes discovery persistence bounded, lib/runtime only, no conflicts), the nineteenth merge b14856a71e (upstream 197c7e56dd, #18255 Kimi-K3.1-Pro recipe alignment, recipes and docs only, no conflicts), the eighteenth merge 7ab262c4a0 (upstream 3d477a4129, #16735 planner startup capacity, Python only, no conflicts), the seventeenth merge 61da52673c (upstream aa620b7181, #18334 recipes and #17937 multimodal audio loader, no conflicts), the sixteenth merge 7e00036163 (upstream d2ceb6a9b8, #17978 release-note credits and #18236 DeepSeek V4 recipes, no conflicts) and the fifteenth merge dbbe5e42fe (upstream 57b5a57b24, #15773 session-affinity hits logged and measured like KV selections, placed into session-affinity, worker-selection and the kernel, plus #15671; chain v2 slim 26:18 green with 2,664 tests, the round's seven kernel tests, two lib/llm tests, dynamo-router-policies 110 and mocker 322 by name, gate 36 / 55); CI on it is running.

Fixed on the previous heads: the TRT-LLM test_deployment[aggregated_router-2] failure was the test's own DYN_LOG naming the pre-split dynamo_kv_router::scheduling::selector target, and Check for broken markdown links flagged two README links the lib/router/ move lengthened (both in the fourth follow-up commit); before that, the cargo machete findings and the lib/bindings/python cargo fmt --check failure, the SPDX headers, the CI-filter coverage of the feature script, the stale docs link, and the lib/runtime/examples lockfile.

Regression check against the base wheel

Same scenario on two wheels, the PR head 6c50b1d0f2 and its merge parent 182039dbb5 (both built the same way), compared output for output. Every item had identical outcomes on both wheels:

  • Router e2e with mockers (tests/router -m "router and gpu_0", 69 ids) and the bindings and config unit tests (178 ids, plus the planner file with the two pre-existing failures).
  • Every router mode (round-robin, random, kv, direct, least-loaded, device-aware-weighted) and every builtin policy (default cost function, two-tier, thunderagent, sticky min-load): 200 requests each, same per-request selection fields and logits; worker labels differ only where equal-cost ties break through an unseeded RNG.
  • Offline replay (dynamo-bench replay-bench): canonical reports byte-identical on four golden rows and four forced-pressure rows (vLLM and SGLang, aggregated and disaggregated, preemption and retraction).
  • Frontend and worker /metrics names, labels, and help strings; prometheus_names.py codegen; libdynamo_llm_capi exported symbols (31); cargo test -p dynamo-ext-proc (168).
  • Standalone indexer, slot tracker, and selection service with replica sync: 92 recorded HTTP responses identical.
  • Frontend throughput A/B (kv mode, four mockers, aiperf, six interleaved runs per side at concurrency 64 and 256): throughput and TTFT p50/p99 medians within 2 percent.
  • Real engines: SGLang 0.5.21 gpu_1 router cases on one L4, and on L40S the full tests/router/test_router_e2e_with_sglang.py (one and two GPUs, including disaggregated and EAGLE) plus the KV-router tests/serve/test_sglang.py configs: same pass set, and Selected worker ... logit= logged once per routed request on both wheels.

The one difference found is the log-target rename that the fourth follow-up commit addresses: DYN_LOG=...,dynamo_kv_router=debug on the old wheel also enabled the selection-core debug lines that now live under dynamo_scheduler; adding dynamo_scheduler=debug restores the identical line set.

Review follow-ups

Four commits on top of the merge commit, from the review and the first CI run on 2026-11-04:

  • dynamo-kv-router and dynamo-kv-hints moved under lib/router/ (lib/router/kv-router, lib/router/kv-hints): crates only the router produces live there; the shared contract crates (lib/kv-events, lib/kv-hashing, lib/multimodal at the time, lib/error) and lib/router-plugins/ stay. 65 files renamed, 59 byte-identical and 6 changed only in a path string (the AGENTS.md headers and one include_str!); the orphan src/sequences/ docs directory is deleted; every path reference in the manifests, codeowners, the gate script, docs, skills, and comments follows, and the lockfiles are untouched by the move.
  • The layering gate stays in CI (the rust-clippy job's Check router crate layering step, with --host) and runs locally on demand; its pre-commit hook is dropped. The per-feature build check is one cargo hack check --locked -p dynamo-kv-router --each-feature --all-targets line inside rust-feature-builds, and scripts/router_features_check.sh is gone. scripts/router_move_diff.py stays as a dev tool.
  • Dependencies the split left unused are dropped (dynamo-kv-router: arc-swap, crossbeam-queue, derive_builder, flume, futures-util, xxhash-rust; dynamo-scheduler: dashmap, ordered-float; dynamo-admission: serde_json; dynamo-kv-indexer: seqlock; libdynamo_llm: dynamo-router-protocol), the three affected lockfiles are regenerated, and cargo fmt --all is applied in every workspace (two import blocks in lib/bindings/python).
  • The two DYNO-195 commits were re-signed, with the history above them rebuilt with identical trees.
  • dynamo-multimodal is folded into dynamo-kv-hashing as its multimodal module (1e45e0395f): the name dynamo-multimodal is already published on crates.io by ai-dynamo/frontend-crates (0.1.0 to 0.2.4), so cargo package -p dynamo-kv-hashing could not resolve dynamo-multimodal = "^1.7.0". Every one of its ten dependents already depended on dynamo-kv-hashing, which already depended on it, so no dependency edge changes and the node disappears from the graph (12 crates). The three source files moved verbatim (git mv, byte-identical); lib.rs became multimodal/mod.rs with the same mod lines and pub use list; the 60 dynamo_multimodal:: paths in 30 files are dynamo_kv_hashing::multimodal::, the root re-exports none of it. Workspace member and dependency, nine manifest lines, the gate row and lists, and the lib/multimodal/ codeowners glob (now lib/kv-hashing/src/multimodal/, router, multimodal and kv-memory teams) follow; the four lockfiles are regenerated. cargo package --locked --no-verify -p dynamo-tokens -p dynamo-kv-hashing succeeds on the folded tree. Validation of the fold: chain v2 in full mode on its tree, 38 min 28 s, every stage exit 0 except the pre-existing kvbm-consolidator manifest failure (2,656 tests passed, 0 failed, 5 ignored; the 11 router crates' clippy, tests, and rustdoc under default and all features; kv-router 237 plus the prelude test; workspace clippy; dynamo-llm lib alone and its kv_router 503, discovery, pool_relay tests; per-host clippy --tests for the 13 hosts and the bindings; the gate 36 plain / 55 host checks, 0 failures; codeowners strict; fmt in all four workspaces; machete; four lockfiles --locked); by name, dynamo-kv-hashing 23 + 17 + 3 (the four multimodal:: tests among them), dynamo-kv-events 74 (extra_key 5, pad normalization 5), dynamo-kv-indexer mm_exclusion 7, dynamo-vllm-generate 37, dynamo-trtllm-sidecar 93 + 3, dynamo-llm preprocessor 397; scripts/router_move_diff.py over the four files: 147 normalized lines before, 146 after, 146 common, 0 new, 1 dropped (the #![deny(missing_docs)] the crate root already sets); pre-commit 22 passed.
  • Log directives follow the split: dynamo_kv_router::scheduling::selector= becomes dynamo_scheduler::selector=, dynamo_kv_router::scheduling=debug becomes dynamo_scheduler=debug,dynamo_admission=debug,dynamo_worker_selection=debug, and the nemotron recipes that set dynamo_kv_router=debug now also name the eight crates that code moved into (the TRT-LLM serve test's DYN_LOG was why test_deployment[aggregated_router-2] missed its Selected worker ... logit= line). The deprecations page carries the same mapping. Two README links the lib/router/ move lengthened by one directory are corrected.

Validation of the three commits: chain v2 in full mode (MODE=full, LLM_EXTRA_TESTS="discovery pool_relay") on the final tree, 34 min 18 s wall for the rebuild the manifest changes force, every stage exit 0 except the pre-existing kvbm-consolidator manifest failure and one hit of a pre-existing race-test flake, dynamo-kv-indexer concurrent_radix_tree_compressed::tests::race_tests::cleanup::race_cleanup_with_dead_child_reuse_keeps_restored_child (unexpected edge lengths: [2, 2, 3]; the module and the test are identical to main modulo import paths, and 1 of 30 isolated runs fails), after which the router test stage was re-run with --no-fail-fast (1,277 passed, 0 failed, 5 ignored) and the whole chain once more on the same tree (2 min 43 s warm; 2,552 tests passed, 0 failed, 5 ignored). Covered: the 12 router crates' clippy, tests, and rustdoc under default and all features; kv-router 237 tests plus the prelude test; cargo clippy --workspace --all-targets -D warnings; dynamo-llm lib alone and its kv_router 493, discovery 281, pool_relay 264 tests; per-host clippy --tests for the 13 hosts and the Python bindings crate; the layering gate 38 checks and --host 57 checks, 0 failures; codeowners 7707/7707 explicitly owned; cargo fmt --check in all four workspaces; cargo machete clean apart from the three pre-existing kvbm-* manifest load errors; the four lockfiles --locked; git diff -M over the move commit shows 65 renames (59 byte-identical, 6 changed only in a path string) and no lockfile change. The first chain pass, on the move and CI commits alone, took 39 min 0 s and was green throughout.

Release note

crates.io carries dynamo-runtime, dynamo-llm, dynamo-kv-router, and dynamo-kv-hashing at 1.6.2 and none of the ten crates this PR adds. This lands in 1.8.0, and that release must publish the new crates first: dynamo-error before dynamo-runtime, and the router crates before dynamo-kv-router and dynamo-llm. .github/scripts/stage_crates.py already discovers and orders them (45 crates at this head instead of 35 on main, with dynamo-error first), so the release script needs no change, only the awareness that the first publish of each new name happens in that release.

Tickets

Part of DYNO-170
Part of DYNO-171
Part of DYNO-172
Part of DYNO-173
Part of DYNO-174
Part of DYNO-175
Part of DYNO-176
Part of DYNO-177
Part of DYNO-178
Part of DYNO-179
Part of DYNO-180
Part of DYNO-181
Part of DYNO-182
Part of DYNO-183
Part of DYNO-184
Part of DYNO-185
Part of DYNO-186
Part of DYNO-190
Part of DYNO-191
Part of DYNO-192
Part of DYNO-193
Part of DYNO-194
Part of DYNO-195
Part of DYNO-196

DEP to follow; this draft is for early review of the shape.

Extract the identities and worker-facing vocabulary every router crate
names into lib/router/protocol: worker and rank ids, partition, pool,
owner, and domain identities, WorkerType, the residency vocabulary, the
block-hash newtypes, worker load and metadata wire types, KV hint wire
types, SessionContext, and AttemptId. lib/kv-router keeps pub use shims at
every old path, so no other crate's imports change.

AttemptId has no public constructor any more. AttemptIdMinter::mint is the
only way to obtain one and yields fresh ids only; the slot tracker's
RequestIndex holds the one minter. Tests use for_testing behind the
testing feature, benches for_benchmark behind bench. ResidencyOwner's
BLAKE3 compact_key is hashing, not wire, so it becomes the
ResidencyOwnerKeyExt extension trait in dynamo_kv_router::protocols.

The crate depends on exactly serde, serde_json, and thiserror.
scripts/router_layering_check.sh enforces that by set equality, forbids
behavior in the crate, and fails closed; it runs in pre-commit and, with
--host, in the Rust CI job. scripts/router_move_diff.py shows the move is
a move: 3,858 lines found, 45 new (minter, extension trait, one test),
9 dropped (the replaced constructor paths).

Step 1 of the router split v2 plan. DYNO-170.

Signed-off-by: Ishan Dhanani <ishandhanani@gmail.com>
Move the router's block and sequence hash functions out of
lib/kv-router/src/protocols.rs into dynamo_kv_hashing::routing: the XXH3
seed, compute_block_hash, BlockHashOptions, the multimodal pad constants and
pad_value_for_mm_hash, hash_mm_identifier, compute_block_hash_for_seq and
its seeded variants, compute_next_seq_hash, compute_seq_hash_for_block,
compute_seq_hash_for_tokens_with_seeds, and TokensWithHashes. The eighteen
tests that exercise them move with them. The three seeded helpers that were
pub(crate) become pub with their existing docs, because the tracking hash
in kv-router calls them across the crate boundary.

Move the multimodal block and request metadata (BlockMmObjectInfo,
BlockExtraInfo, RequestMmObjectInfo, RequestExtraInfo) into
dynamo_router_protocol::mm: KV events, the hash contract, and the frontend
preprocessor all name it, so it is shared vocabulary.

dynamo-kv-hashing gains dependencies on dynamo-router-protocol and
xxhash-rust; protocol does not depend on it. lib/kv-router keeps pub use
shims at the old paths, so no other crate changes. The layering gate gains
a dynamo-kv-hashing row and forbids kv-hashing in protocol's tree.

Step 3a of the router split v2 plan, taken before step 2 because the ZMQ
event codec calls these helpers. DYNO-171.

Signed-off-by: Ishan Dhanani <ishandhanani@gmail.com>
…ine codec

Move everything about KV cache events out of lib/kv-router into
lib/kv-events: the event types (KvCacheEvents, KvCacheEvent,
KvCacheEventData, store, remove, and stored-block data, KvCacheEventError),
the RouterEvent envelope and its builders, KV_EVENT_SUBJECT, the engine ZMQ
msgpack codec (zmq_wire), the replay cursor (recovery/cursor.rs), and the
publisher placement helpers (Placement, PlacementOwner, PlacementEvent).
The RouterEvent legacy-wire tests and the router event constructor test
move with the types; the codec's tests move with the codec.

Both ends of the KV-event wire depend on this crate: workers, sidecars,
and the C bindings publish with it, the router's indexer ingests with it.
It depends on dynamo-router-protocol and dynamo-kv-hashing plus serde,
rmp-serde, rustc-hash, thiserror, and tracing; no sockets, tasks, or
servers. lib/kv-router keeps pub use shims at every old path, including the
zmq_wire and recovery module paths, so no other crate's imports change.

KvCacheEventError is non-exhaustive, which binds only across crates, so
kv-router's two exhaustive matches on it (behind the metrics feature) gain a
wildcard arm and an "unknown" status counter.

The layering gate gains a dynamo-kv-events row; CODEOWNERS claims
lib/kv-events and follows the moved extra_keys override.

Step 2 of the router split v2 plan. DYNO-172.

Signed-off-by: Ishan Dhanani <ishandhanani@gmail.com>
Move lib/kv-router/src/services/common/zmq.rs to lib/zmq as the
dynamo-zmq crate: the async ZmqSocket over tokio's AsyncFd, multipart send
and receive, the process-wide context pool sized by DYN_ZMQ_IO_THREADS, the
SUB, DEALER, and bound PUB constructors, and validate_endpoint. The file
moves verbatim; the feature gates that selected functions per standalone
service are dropped because the crate is unconditional, and pub(crate)
becomes pub. The error type stays anyhow for now so callers do not change;
it becomes a thiserror enum when those callers move into the router adapter
layer.

lib/kv-router re-exports the crate at services::common::zmq behind the same
standalone features, so no caller changes. The layering gate gains a
dynamo-zmq row; CODEOWNERS claims lib/zmq.

Step 4 of the router split v2 plan. DYNO-173.

Signed-off-by: Ishan Dhanani <ishandhanani@gmail.com>
…gate, and counters

Create lib/router/kv-indexer, the KV indexer stage crate, and move its
first slice out of lib/kv-router: the cuckoo-filter pool index (cuckoo and
its eleven modules), the hash lifecycle delegate, and the indexer counters.
The files move verbatim with their tests; imports that pointed at
kv-router's protocols shims now name dynamo-router-protocol,
dynamo-kv-events, and dynamo-kv-hashing directly, and pub(crate) items that
sibling files still in kv-router use become documented pub.

Two things could not move. KvIndexerMetrics::from_component needs a runtime
Component, so it becomes the KvIndexerMetricsExt extension trait in
dynamo_kv_router::indexer behind runtime-protocols, built on the new
KvIndexerMetrics::from_counters; the one lib/llm caller gains the trait
import.
The counters are Prometheus IntCounterVec fields, so the stage carries a
transitional `export` feature that kv-router's `metrics` forwards to; the
metrics behavior change replaces it with atomics.

lib/kv-router re-exports cuckoo, the delegate, and the counters at their
old paths. The layering gate gains a dynamo-kv-indexer row that allows
prometheus only under export; CODEOWNERS claims the crate.

Step 5a, slice 1 of the router split v2 plan. DYNO-174.

Signed-off-by: Ishan Dhanani <ishandhanani@gmail.com>
…o dynamo-kv-indexer

Slice 2 of step 5a. Move out of lib/kv-router into lib/router/kv-indexer:
the radix trees (radix_tree, positional, compressed_radix), the concurrent
compressed tree and its sixteen modules, the ThreadPoolIndexer and
BranchShardedIndexer wrappers with the shard handle, KvIndexer, approximate
LRU retention and pruning, the bench-only observation plumbing, the indexer
traits (KvIndexerInterface, SyncIndexer), the task and request types, the
active-set helper, the residency projection and routing snapshot, overlap
scores, KV transfer candidates with the cache owner's hashed hint keys, the
shared test helpers (now dynamo_kv_indexer::testing), the indexer_delegate
bench, and the index README. The files move verbatim with their tests;
shared sibling-crate names come through a crate-private explicit prelude.

Three items stay in lib/kv-router by design and are cut, not moved:
SharedKvCache is a kernel hook and lands in scheduling/shared_cache.rs;
TieredMatchProvider and the Wire<->LowerTierMatchDetails conversions join
the lower-tier files they return; the request-plane DTOs (KvState*,
WorkerKvQuery*, IndexerQuery*, IndexerRecordRoutingDecision*) become
indexer/api.rs because lib/llm implements the runtime MaybeError on them.
The LocalKvIndexer tests and the delegate tests stay with local.rs and the
lower tier (indexer/local_tests.rs, indexer/delegate_tests.rs) until those
slices move; one radix-tree test compares the dump payloads directly
instead of wrapping them in WorkerKvQueryResponse.

Items that files still in kv-router call become documented pub:
mix_worker, ApproximateLruTask::complete and ApproximateLruReply,
CacheOwnerKvTransferKeys fields, label_key and topology_keys, and the
test-only KvIndexer::snapshot_event_sender (gated on `testing`).
lib/kv-router re-exports the crate at indexer::*, protocols::, kv_hints::,
and test_utils::. The gate row allows tokio for the indexer: the stage
rule forbids sockets and HTTP, not channels and actors.

Step 5a, slice 2 of the router split v2 plan. DYNO-174.

Signed-off-by: Ishan Dhanani <ishandhanani@gmail.com>
…mo-kv-indexer

Slice 3 of step 5a. Move lower_tier.rs (LowerTierIndexer, the continuation
index over (parent, local) -> child edges with residency-owner tables,
LowerTierContinuation, LowerTierMatchDetails, KvTransferExtensions,
WorkerBlockIndex) and lower_tier_indexers.rs (LowerTierIndexers,
TieredMatchDetails, LowerTierQueryOptions, TieredMatchProvider, the
tier-walk helpers) verbatim from lib/kv-router into lib/router/kv-indexer,
with their tests. Imports that named kv-router's shims now go through the
crate prelude, which gains ResidencyOwnerKey, ResidencyProjection, and
ResidencyRoutingSnapshot. lib/kv-router keeps every old path through its
indexer::* re-export; nothing outside kv-router changes.

Step 5a, slice 3 of the router split v2 plan. DYNO-174.

Signed-off-by: Ishan Dhanani <ishandhanani@gmail.com>
The indexer's request-plane responses (WorkerKvQueryResponse,
IndexerQueryResponse, IndexerRecordRoutingDecisionResponse) implemented
dynamo_runtime's MaybeError inside lib/kv-router behind the
runtime-protocols feature. The DTOs are about to move into
dynamo-kv-indexer, which must not depend on dynamo-runtime, and neither
kv-router nor lib/llm may implement a foreign trait on a foreign type.

Add lib/llm/src/kv_router/indexer/reply.rs with three #[serde(transparent)]
newtypes (WorkerKvQueryReply, IndexerQueryReply,
IndexerRecordRoutingDecisionReply) that implement the runtime MaybeError
exactly as the removed impls did: Error(message) is the error state, and
from_err produces it. Use the newtypes at the typed boundaries only: the
AddressedPushRouter stream in the worker query transport, the PushRouter
and Ingress generics and the two served engines in remote.rs, and the
WorkerKvQueryEngine stream builders. Every other caller keeps the bare
enums through From in both directions. The wire encoding is unchanged
(transparent newtype); tests pin that and the error mapping.

Remove the three runtime impls from lib/kv-router's indexer/api.rs; the
runtime-protocols feature keeps KvIndexerMetricsExt.

Prerequisite for step 5a slice 4 of the router split v2 plan. DYNO-174.

Signed-off-by: Ishan Dhanani <ishandhanani@gmail.com>
…ocal indexer, and DTOs

Slice 4 of step 5a, the last move into lib/router/kv-indexer:

- api: the request-plane DTOs (KvState*, WorkerKvQuery*, IndexerQuery*,
  IndexerRecordRoutingDecision*, the endpoint names) and the indexer's own
  MaybeError trait, verbatim.
- local: LocalKvIndexer, with its tests and the delegate tests that build
  lower-tier and local variants.
- backend, lookup, recording, session_updates: the ingest backend
  (Indexer, IndexerPolicy, RemotePrimary, SideIndexer, create_indexer*),
  the lookup pipeline, approximate request leases, and session updates.
- session_prefix_index: SessionPrefixIndexer; the cleanup cadence becomes a
  per-stage constant instead of reaching into kv-router's cleanup module.
- identity: IndexerIdentitySpec, ExplicitIdentityMap, IdentitySpecError,
  CanonicalIdentityMaterial, the explicit-identity limits and tag constants,
  and their tests. kv-router keeps the protocol re-export and
  default_routing_group.
- config: ApproximateCachePolicyKind moves here, and the new IndexerConfig
  slice (use_kv_events, approximate_cache_policy, predicted_ttl_secs,
  ttl_secs) replaces KvRouterConfig in IndexerPolicy::from_router_config,
  now from_config with the same checks and messages. kv-router converts
  once with From<&KvRouterConfig> for IndexerConfig at its three call
  sites; the policy tests build IndexerConfig literals.

backend::test_util is pub under `testing` for kv-router's service tests;
the test-only Indexer::flush drops its standalone-selection gate.
lib/kv-router re-exports everything at the old paths: indexer::*,
services::indexer::{backend, lookup, recording, session_updates},
session_prefix_index, identity::*, config::ApproximateCachePolicyKind.
Nothing outside kv-router changes.

Step 5a, slice 4 of the router split v2 plan. DYNO-174.

Signed-off-by: Ishan Dhanani <ishandhanani@gmail.com>
WorkerCapacity (a rank's advertised max_num_batched_tokens and
total_kv_blocks, zero meaning unreported) and DEFAULT_MAX_BATCHED_TOKENS
lived in worker selection's inputs module, but the slot tracker stores a
capacity per rank and BusyThresholds judges fullness against it. Stage
crates do not depend on each other, so the type moves to
dynamo_router_protocol::load beside SchedulerLoad and ActiveLoad: the
values cross the process boundary in the worker's runtime config, and
every stage names them. prefill_budget becomes pub for the same reason.

WorkerCapacity::of(&impl WorkerConfigLike) cannot follow (the config
trait is router-side), so it becomes the WorkerCapacityExt extension
trait next to the trait, re-exported from plugins::worker_selection; the
five kv-router call sites and the one mocker call site import it.
inputs.rs re-exports both names at their old paths.

Step 5b-0 of the router split v2 plan (prerequisite for
dynamo-slot-tracker). DYNO-175.

Signed-off-by: Ishan Dhanani <ishandhanani@gmail.com>
Step 5b of the router split v2 plan. Create lib/router/slot-tracker and
move the slot tracker out of lib/kv-router verbatim with its tests: the
active-sequence trackers (single, multi_worker), the prompt registry with
its membership trie and load projection, the block and prefill trackers,
the request index that holds the one AttemptIdMinter, worker topology, the
replication effects, the sharded lock, and cleanup.rs (used only by the
membership trie). Three items join it from other kv-router files because
the slot tracker owns them: the replication wire (ActiveSequenceEvent*,
ActiveBlockEvent*, PrefillEvent*, PrefillLoadHint, MAX_REPLICA_BATCH_*)
from protocols.rs as `wire`, BusyThresholds and FullRankCounts from
scheduling/policy_config.rs as `busy`, and SchedulerBookingDescriptor from
scheduling/queue.rs as `booking` (documented, no longer doc-hidden).

Two seams needed shaping so the stage never sees router-side types.
BusyThresholds::has_room is generic over a RankLoad reading trait with two
accessors; the slot tracker's WorkerLoadProjection implements it, and
kv-router implements it for worker selection's WorkerLoadInput, so every
existing call keeps its text. worker_is_full(&impl WorkerConfigLike)
becomes worker_is_full_in_ranks(worker_id, dp_ranks, capacity, loads) in
the stage plus kv-router's BusyThresholdsExt::worker_is_full with the old
signature; the mocker imports the trait. rank_is_full reads the projection
map directly instead of building a LOAD row (same result).

Stage operations the router drives (the booking lifecycle methods on
ActiveSequencesMultiWorker, SequenceTrackerOptions and its fields,
RateLimitedFailureLog::record, PrefillCompletion, WorkerLoadReader and its
get, the load projection pass) become documented pub; request_booking
drops its standalone-slot-tracker gate. The crate depends on
dynamo-router-protocol and dynamo-tokens only (kv-hashing is a
dev-dependency for one test); tokio stays for channels and timers.

lib/kv-router aliases the crate as `sequences` and re-exports the wire,
busy, and booking names at their old paths; it drops crossbeam-utils,
indexmap, seqlock, and slotmap, which only the tracker used. The layering
gate gains the slot-tracker row; CODEOWNERS claims the crate.

DYNO-175.

Signed-off-by: Ishan Dhanani <ishandhanani@gmail.com>
…tage crate

Step 5c of the router split v2 plan. Create lib/router/session-affinity
and move services/selection/affinity.rs out of lib/kv-router verbatim as
the crate root: the SessionAffinity table, WeakSessionAffinity, the acquire
protocol (Hold, AffinityInitialization, AffinityLease), AffinityVersion,
the AffinityReplicaSink port, AffinityError, SessionAffinityConfig and
SessionAffinityMode, the limits, subagent_group_affinity_id, and the
tests. AffinityBindingEvent moves from the replica mesh
(services/common/replica_sync.rs) into the crate's `wire` module with its
target() and version() accessors made pub and ungated: both ends of that
wire are affinity tables, and the mesh that carries it stays a kv-router
adapter. SessionAffinity::validate_ttl becomes documented pub for the
router's config validation.

The crate depends on dynamo-router-protocol only among router crates
(WorkerAffinityTarget is the protocol's); feature `testing` gates the
test hooks the file already had. lib/kv-router aliases the crate at
services::selection::affinity and re-exports the wire type at its old
path, so lib/llm's session_affinity module keeps every import. The
layering gate gains the session-affinity row; CODEOWNERS claims the crate.

DYNO-176.

Signed-off-by: Ishan Dhanani <ishandhanani@gmail.com>
…quest

WorkerSelectionContext, the view filters, scorers, and pickers read, held
a reference to the kernel's SchedulingRequest and answered its accessors
by reaching into it. The worker-selection stage must not depend on the
kernel's request type, so the context now carries the values a policy may
read: request id, ISL tokens, request blocks, block size, prefill-token
tracking, pinned worker, temperature override, session context, affinity
target, expected output tokens, the two priorities, and the shared-cache
hits. The kernel builds it once per selection with from_request. Two
accessors are new (request_id, shared_cache_hits); the existing accessors
return the same values from the copied fields.

The reference selector was the only reader of the request through the
context: it now uses the accessors, and the per-worker local transfer
prefix it needs for the fetchable-blocks term is passed in by the caller,
which already computes it from the request. Plugin crates read the context
through accessors only and do not change.

Step 5d-0 of the router split v2 plan (worker-selection boundary). DYNO-177.

Signed-off-by: Ishan Dhanani <ishandhanani@gmail.com>
Worker-selection code read the flat KvRouterConfig: the policy factory type
took it, WorkerSelectionPolicy::new carried it, the reference selector
scored with its weights, and ConditionalDisaggPolicy read six of its
fields. The stage must not depend on the router's flat configuration, so
two slices now own those fields.

SelectionConfig (plugins::worker_selection::selection_config) holds the
scoring weights, the sampling temperature, the shared-cache multiplier and
type, and conditional_disagg_enabled, with Default equal to the flat
defaults, From<&KvRouterConfig> next to the flat type, and validate() using
the validators that moved with it. SharedCacheType moves into the slice
module. WorkerSelectionPolicyFactory takes &SelectionConfig;
WorkerSelectionPolicy::new, new_with_filters, and reference, and the
reference DefaultWorkerSelector, take SelectionConfig. The registry, the
selection core, lib/llm's prepared-policy wrapper, the mocker, and the
in-tree plugins convert once at the boundary.

ConditionalDisaggConfig (conditional_disagg) holds the six
conditional_disagg_* fields with Default and From<&KvRouterConfig>;
ConditionalDisaggPolicyKind moves beside it; make_conditional_disagg_policy
and the policies' from_config take the slice, and lib/llm's prefill leg
converts once. config:: re-exports every moved name and the root exports
the two slices, so existing paths resolve.

The builtin plugin's default policy, its parameters, and its selector take
SelectionConfig; the four example policies and the builtin, kv-router, and
lib/llm tests convert where they built a flat config for selection.

Step 5d-1 of the router split v2 plan (worker-selection boundary). DYNO-177.

Signed-off-by: Ishan Dhanani <ishandhanani@gmail.com>
… ranks them

The transfer-source summary that credits a candidate worker for the KV
prefix it could fetch read two things the worker-selection stage must not
see: each worker's config (through WorkerConfigLike::kv_transfer_keys) and
the indexer's residency routing snapshot (through
KvTransferCandidates.routing_snapshot). Split the step in two.

The router resolves first: kv_hints::resolve_transfer_sources walks the
request's candidates with today's skips (sources no longer than every
target's local prefix or longer than the retained chain, worker ranks that
publish no source endpoint, cache owners missing from the snapshot, the
attached worker's topology for an owner that publishes none) and yields
ResolvedTransferSource records: who holds the prefix, how long it is, the
hashed hint worker type, the attached worker, and the topology labels.
KvTransferSourceSummary::new takes those records and the scope and ranks
them exactly as before; source_for is unchanged.

Worker keys move the same way. WorkerKvTransferKeys keeps its data,
target(), and a from_parts constructor over already-hashed labels, with
worker_type() and topology() accessors; deriving the keys from a config
becomes the WorkerKvTransferKeysExt extension trait beside
WorkerConfigLike, and WorkerKvTransferKeysCache moves next to it.
KvTransferCandidateSource moves to dynamo_router_protocol::kv_hints as
shared vocabulary; the indexer and kv-router re-export it at the old
paths. The label hashing (label_key, topology_keys) stays with the indexer,
which builds the snapshot keys; the router calls it for worker keys.

Step 5d-2 of the router split v2 plan (worker-selection boundary). DYNO-177.

Signed-off-by: Ishan Dhanani <ishandhanani@gmail.com>
Step 5d-3 of the router split (DYNO-177). The composed worker-selection
policy no longer reads SchedulingRequest, RoutingEligibility, or worker
configs. The kernel implements a CandidateRowSource (walk the eligible
ranks, each a CandidateRank that yields its row at the requested
columns and its capacity; a rank's capacity and booked load by worker)
and drives the policy through WorkerSelectionPolicy::select_rows(
context, source, prepared, trace), which returns a RowPick or a
SelectionFailure. The walk takes a RankVisitor whose visit method is
generic over the rank, so the walk, the rank, and the row build are
monomorphized together; a dyn visitor cost 11 ns per rank. The kernel maps the pick
to WorkerSelectionResult, logs the selection, and turns NoCandidates
into overloaded-vs-no-endpoints from eligibility; the stage reports
AllFiltered only when the source yielded a row.

Rows still materialize lazily: a policy with filters asks a rank for
the filters' columns and for the remaining columns only once it is
kept, and the no-filter path stays one pass. CandidateData gains a u32
raw_cached_tokens column (packed beside inputs; the row stays 128
bytes) that the kernel fills for tracked requests, so the stage folds
the best eligible cache reuse over kept rows without the request.

The fetch column is filled as each row is built. Its summary's scope is
every eligible rank's target and local prefix (KvTransferScope::empty +
add_target), computed once; this admits only sources no kept rank can
use, which source_for already rejects, so kept rows see the same
sources as the kept-rows scope did. The selected worker's fetch source
keeps its single-target scope when no scorer declared KV_TRANSFER.

CandidatePreparer::prepare_with owns the pooled buffer and the kernel's
CandidatePreparerExt::prepare builds rows into it; PreparedCandidates
is now the kernel wrapper carrying the fetch summary beside the stage's
PreparedRows. Selection-trace assembly and the per-candidate TRACE log
move into the stage with the scratch rows they read. The reference
selector keeps reading the request through the kernel's row source.

Validation: kv-router 594 + 2 + 7 (four new stage-only tests drive
select_rows over a table with no request in sight), builtin plugin
tests, lib/llm kv_router tests, clippy clean, layering gate 20 checks.
criterion worker_selection against the parent commit at 10k workers:
custom_no_filter +1.6%, custom_keep_all_filter +1.2%,
custom_capacity_filter +2.4%, unused_preferred_taint_metadata no
change, reference direct -1.6%; default_policy_wrapper -2.5% to -8% at
32 and 1024 workers, +9-11 ns per selection at 2 workers.

Signed-off-by: Ishan Dhanani <ishandhanani@gmail.com>
…her crate

Step 5d-3b of the router split (DYNO-177): the last boundary changes
before the stage becomes its own crate, each one something the kernel
could do only because it shared a crate with the stage.

- WorkerSelectionContext gains a public constructor: new(isl_tokens,
  block_size) with every other field at its default plus with_* setters;
  the kernel's from_request becomes selection_context in the selector.
- The row types the kernel fills have public fields: CandidateData,
  WorkerCacheData, WorkerLoadInput, WorkerTransferInput, CacheSnapshot,
  and WorkerInputRows with new/empty/read.
- What the stage derived from kernel-only types leaves it: the config-
  and projection-derived constructors are now the WorkerCapacityExt,
  WorkerInfoExt, and WorkerLoadInputExt traits in selector::derive, and
  the facade re-exports them at their old paths. WorkerInfo::of becomes
  WorkerInfoExt::of.
- RankLoad moves to the protocol crate's load module so both the slot
  tracker's projection and the stage's load column implement one trait
  without either stage naming the other; the slot tracker re-exports it.
- WorkerSelectionPolicyError, the selection trace types, and
  SharedCacheHits move into the stage's modules; scheduling::types and
  protocols re-export them, so every existing path still resolves.
- The reference picker moves into the stage behind the bench and test
  features as a push-based pick: begin(temperature), offer(worker,
  cost), finish(). The kernel's pick_default_worker offers scores in
  eligibility order, or sorted order when the picker is seeded, and
  never touches the RNG or the softmax scratch. Same draws, same ties.

Validation: protocol 10, slot-tracker 130, kv-router 595 + 2 + 7,
builtin plugin, lib/llm kv_router 485 tests; clippy clean on stage
crates and lib/llm, mocker, ext-proc, bench; layering gate 20 checks;
move-diff 16,744 common / 238 new / 125 dropped. Interleaved A/B bench
against the 5d-3 head at 10k workers: composed paths at parity, the
test-only reference path within about 1 ns per rank once the pick
state mirrors the old loop's locals (its first cut measured +3.8%).

Signed-off-by: Ishan Dhanani <ishandhanani@gmail.com>
…r-selection

Step 5d-4 of the router split (DYNO-177): dynamo-worker-selection, a
stage crate that depends on dynamo-router-protocol and nothing router-
side beyond it. A move: the files that made up the stage inside
lib/kv-router move as they are, and lib/kv-router re-exports every name
at the path plugins and hosts already use.

Moved: plugins/worker_selection/{context, inputs, runtime,
selection_config, error, trace, shared_cache, reference_picker}.rs, the
plugin traits from the plugins/worker_selection.rs facade,
scheduling/selector/policy.rs (the composed policy, CandidateRowSource,
RankVisitor, CandidateRank, CandidatePreparer, PreparedRows, the trace
builder), kv_hints/sources.rs as kv_transfer.rs, conditional_disagg.rs,
and scheduling/lora_filter.rs.

Stays in lib/kv-router: the WorkerSelector trait and its adapter, the
request-to-row source, the derived columns (selector/derive.rs), the
reference selector, the policy factory type, the registry, the host
instance config (plugins/worker_selection/config.rs), and the resolver
for kv-transfer sources. The policy tests that drive WorkerSelector with
a SchedulingRequest stay as selector/policy_tests.rs; the kv-transfer
summary tests that resolve against a routing snapshot stay in
kv_hints.rs; the stage keeps its own row-source, trace, picker, scope,
and conditional-disagg tests.

Visibility the crate boundary required: the selection config slice's
defaults and validators, which the flat router config shares, are pub,
as are the conditional-disagg defaults, WorkerInputs::ALL, and
WorkerSelectionPolicy::reference_parts; the reference selector reads the
context through its accessors. DEFAULT_MAX_BATCHED_TOKENS is taken from
the protocol crate directly.

Layering gate: a row for dynamo-worker-selection (22 checks). CODEOWNERS
gains lib/router/worker-selection/ under the router area.

Validation: stage 41 tests; kv-router 554 + 2 + 7 (554 + 41
moved = the 595 before); builtin plugin and lib/llm kv_router 485 tests;
clippy clean on the stage crates and lib/llm, mocker, ext-proc, bench;
layering gate 22 checks; move-diff 13748 common / 102 new / 35 dropped (the new lines are #[inline] hints and visibility).
Interleaved A/B bench against the 5d-3b head at 10k workers: parity or
better once the stage's small accessors carry #[inline] (non-generic
functions the kernel calls per rank across the crate boundary; without
the hints the reference path measured +2%).

Signed-off-by: Ishan Dhanani <ishandhanani@gmail.com>
Step 5e-0 of the router split (DYNO-178). The column-group bitset a
stage component declares (WorkerInputs) and the static facts about a
rank every column carries (WorkerInfo: identity plus advertised
capacity) move from dynamo-worker-selection to a new columns module in
dynamo-router-protocol. Worker selection builds its rows from them and
admission's classifier will read workers through them, so neither
stage should have to name the other. Worker selection re-exports both,
so every existing path in kv-router, lib/llm, and the plugins holds.
WorkerInputs::without, which only the composed policy used, is public
in its new home.

Validation: protocol 10 and worker-selection 41 tests; kv-router
554 + 2 + 7; builtin plugin and lib/llm kv_router 485 tests; clippy clean
on the stage crates and lib/llm, mocker, ext-proc, bench; layering gate
22 checks; move-diff 458 common / 1 new / 1 dropped.

Signed-off-by: Ishan Dhanani <ishandhanani@gmail.com>
… view

Step 5e-1 of the router split (DYNO-178). RequestClassifier::classify
takes `&dyn WorkerView` instead of worker selection's
`WorkerCandidates<'_>`. The view is the admission stage's own vocabulary:
one row per eligible rank, `info(row)` for identity and capacity, and
`load(row)` / `cache(row)` as plain column structs (WorkerLoadView,
WorkerCacheView) that read as None unless the classifier declared the
group. The kernel implements it over worker selection's rows with the
ClassifierRows newtype (scheduling/classifier_rows.rs), so each column
means what it means in a filter or scorer, and the admission crate will
depend on the protocol crate alone.

This is the one plugin-API signature change of the admission step. The
in-tree implementations (the ThunderAgent classifier, the lib/llm
watcher and routing-host test classifiers, the kv-router and builtin
test classifiers) change their parameter type and nothing else; none
of them read the table beyond `is_empty`. Groups other than CACHE and
LOAD are selection's and read as None through the view; no in-tree
classifier declared them. NoWorkers is the empty view.

Validation: kv-router 554 + 2 + 7; builtin plugin all targets; lib/llm
kv_router 485 and discovery watcher 36 tests; clippy clean on the stage
crates and lib/llm, mocker, ext-proc, bench; downstream check clean;
layering gate 22 checks; move-diff over the touched kv-router files
9,243 common / 124 new / 39 dropped (the view module, the newtype, and
the signature and test-site changes).

Signed-off-by: Ishan Dhanani <ishandhanani@gmail.com>
Step 5e-2 of the router split (DYNO-178). The resolved half of the
policy config leaves the YAML behind.

- BusyThresholds (the per-rank limits, is_configured, has_room) moves
  to dynamo-router-protocol's load module beside WorkerCapacity and
  RankLoad: admission configures it per class, the kernel's gate
  evaluates it, classifiers read it. The slot tracker re-exports it and
  keeps the rank checks over its own projection as BusyRanksExt
  (rank_is_full, worker_is_full_in_ranks).
- RouterQueuePolicy, PolicyClassConfig, PolicyProfile, and the family-
  bucket classifier move to scheduling/admission_config.rs, the stage's
  resolved config, with a public validated constructor:
  PolicyProfile::family_buckets(FamilyBucketTable) refuses an index out
  of range, a bucket list that does not start at zero or is not
  strictly increasing, or a table whose size is not families times
  buckets, returning AdmissionConfigError::Validation. The raw YAML
  forms, the file reader, RouterSettings, the identifier rules, and
  every message the resolver emits today stay in policy_config.rs; the
  resolver builds the table and calls the constructor, and
  RouterPolicyConfigError wraps the stage's error.
- config.rs and policy_config.rs re-export every moved name, so
  dynamo_kv_router::RouterQueuePolicy, scheduling::{BusyThresholds,
  PolicyClassConfig, PolicyProfile, RouterPolicyConfigError}, and the
  slot tracker's busy::BusyThresholds all still resolve. The mocker's
  offline replay, the one caller of rank_is_full outside the slot
  tracker, imports BusyRanksExt beside BusyThresholdsExt.

Validation: protocol 10, slot-tracker 130, worker-selection 41,
kv-router 554 + 2 + 7 (resolver messages unchanged), builtin plugin,
lib/llm kv_router 485, mocker 304; clippy clean on the stage crates and
kv-router, lib/llm, mocker, ext-proc, bench; layering gate 22 checks;
move-diff 3878 common / 109 new / 10 dropped.

Signed-off-by: Ishan Dhanani <ishandhanani@gmail.com>
…the view

Step 5e-3 of the router split (DYNO-178). RequestClassifierRuntime no
longer names KvSchedulerError: classify_with, begin_request, and the
terminal-delivery wait return ClassificationError, with one variant per
failure the runtime can detect (Panicked, Failed, DuplicateRequestId,
InvalidMetadata, LifecycleEnded, Shutdown) and the same messages the
kernel's variants carried. The kernel converts with
From<ClassificationError> for KvSchedulerError, so its error surface,
the selection service's status mapping, and every host path are
unchanged. classify_with takes `&dyn WorkerView`; the scheduler wraps
the rows it built in ClassifierRows at its two call sites, and the
runtime's tests classify over the empty view. WorkerView gains a Sync
supertrait: views are read-only snapshots, and a classification future
that borrows one must stay Send for spawned callers.

Validation: kv-router 554 + 2 + 7; builtin plugin all targets; lib/llm
kv_router 485 and discovery watcher 36; clippy clean on kv-router,
builtin, lib/llm, mocker, ext-proc, bench; downstream check clean;
layering gate 22 checks; move-diff 3,788 common / 91 new / 64 dropped.

Signed-off-by: Ishan Dhanani <ishandhanani@gmail.com>
Step 5e-4 of the router split (DYNO-178): dynamo-admission, a stage
crate that depends on dynamo-router-protocol and nothing router-side
beyond it. A move: the files that made up the stage inside
lib/kv-router move as they are, and lib/kv-router re-exports every name
at the path plugins and hosts already use.

Moved: scheduling/policy_queue.rs as queue, scheduling/queue_admission
as placement, scheduling/admission_config.rs as profile, and the
classifier side as classifier/{mod (the plugin API), inputs, progress,
view, runtime (scheduling/request_classifier.rs: the runtime and the
request lifecycle)}.

Stays in lib/kv-router: the YAML policy file and its resolver
(policy_config.rs), the classifier instance config and the registries,
the gate over the slot projection, ClassifierRows (the WorkerView over
worker selection's rows), and the ClassificationError conversion. The
24 policy-queue tests build their profiles through the YAML resolver,
so they stay as scheduling/policy_queue_tests.rs and read the queue's
internals through three testing-gated accessors (due_entry_count,
class_deficit, class_blocks_worker); the entry-size test moves with the
queue. The classifier runtime's tests move with it.

Visibility the crate boundary required: the kernel-facing constructors
and scheduling methods are pub (ClassifyRequest::with_*, into_queue_
inputs; ClassificationOverrides and QueueMetadata with their fields;
PolicyQueue::{has_ready, any_ready_head, next_due_at, recheck_worker,
enqueue_with_due_at, take_if_in_class, take_expired};
RequestClassifierRuntime and its methods), documented as kernel-facing
in the crate docs instead of #[doc(hidden)]. ClassifyRequest's fields
stay private: the runtime that reads them is a child module.

Layering gate: a row for dynamo-admission (24 checks). CODEOWNERS gains
lib/router/admission/ under the router area.

Validation: admission 22 tests; kv-router 532 + 2 + 7 (532 + 22 moved = the
554 before); builtin plugin, lib/llm kv_router 485 and watcher 36,
mocker 304; clippy clean on admission, kv-router, builtin, lib/llm,
mocker, ext-proc, bench; layering gate 24 checks; move-diff 3,941
common / 55 new / 49 dropped (visibility and the test hooks).

Signed-off-by: Ishan Dhanani <ishandhanani@gmail.com>
Step 6-0 of the router split (DYNO-179). The scheduler kernel stops
reading the flat KvRouterConfig. RouterConfig (scheduling/
router_config.rs) holds exactly what the kernel reads: the worker-
selection and conditional-disagg slices, use_kv_events,
track_prefill_tokens and assume_kv_reuse with their override-aware
helpers, the queue recheck interval, and a PolicyProfileSource that
resolves a partition's profile by model name. RouterConfig::validate
runs the selection slice's checks plus a non-zero recheck interval and
yields Validated<RouterConfig>, the only config SelectionCore::new_inner
accepts (invariant G4); the flat config converts with
From<&KvRouterConfig>, resolving profiles lazily through the policy
file it loaded so a bad file still fails where policy_profile failed
before.

OverlapAnalysis::new, cache_hit_estimates_from_tiered_matches, and
TieredOverlapRefresher::new take CacheHitWeights, a two-field view of
the selection slice's host and disk cache-hit weights, instead of the
whole flat config; the kernel derives it from config.selection, and
lib/llm and the mocker pass CacheHitWeights::from(&config) at their
five call sites. SelectionCore holds the validated slice; its
try_new_local constructors still take KvRouterConfig and convert, so
tests and benches are unchanged.

Validation: cargo test -p dynamo-kv-router --features
standalone-selection,standalone-slot-tracker,metrics,testing (532 lib +
2 plugin-API + 7 standalone HTTP); clippy -D warnings on kv-router
(all targets, same features), dynamo-llm, dynamo-mocker, dynamo-ext-proc,
dynamo-custom-policy-builtin, dynamo-bench (--tests), dynamo-sglang-sidecar;
builtin tests 56+5+1+3+20+3+4; cargo test -p dynamo-llm kv_router (485)
and --lib watcher (36); dynamo-mocker --lib 304; router layering gate
24/24; CODEOWNERS strict clean; cargo fmt --check clean; move-diff vs
37e939a lists only the RouterConfig/Validated/CacheHitWeights additions
and the removed KvRouterConfig reads (summary common=16814 new=198
dropped=58).

Signed-off-by: Ishan Dhanani <ishandhanani@gmail.com>
… selection core

Step 6-1a of the router split (DYNO-179). Three adapters the scheduler
kernel still reached are cut at the boundary the kernel crate will have,
without moving any file across crates yet.

SelectionError no longer implements axum's IntoResponse. It keeps
status_code() -> u16 and kind() as data (the numbers come from a private
status module instead of http::StatusCode); the response body, including
the classifier-failure sanitizer and the QueueRejected details, moves to
server.rs as selection_error_response(), which the eleven handler sites
call. A free function rather than an impl: once the error type lives in
the kernel crate, kv-router cannot implement a foreign trait for it.

The kernel's default KV index source is LocalIndexIngress, which opens a
partition's index and nothing else. SelectionServiceBuilder::new installs
ZmqDirectIngress explicitly (the standalone service's behavior is
unchanged; kv_index() still overrides it), and the ZMQ adapter moves to
zmq_ingress.rs with its endpoint validation and listener reconciliation.
try_new_local, whose callers are all tests and benches, now gets the
passive ingress, which with use_kv_events = false was never consulted.

services/common/replica_sync.rs splits in two. The channel plumbing the
selection core and the slot-tracker service drive (scoped sequence
events, ReplicaSyncConfig, the per-partition publisher and subscriber,
ReplicaInbox, the session-affinity sink, the host channels and
setup_scoped_replica_sync) is services/common/replication.rs, with no
ZMQ import; the ZMQ mesh (publisher task, PeerManager, peer commands,
topics, ReplicaSyncRuntime, setup_replica_sync) stays in replica_sync.rs
and re-exports the moved names at their previous visibility, so every
existing path still resolves and the tests stay where they were.

Validation: cargo test -p dynamo-kv-router --features
standalone-selection,standalone-slot-tracker,metrics,testing (532 lib +
2 plugin-API + 7 standalone HTTP), and --no-default-features
--features standalone-slot-tracker --lib (332); clippy -D
warnings on kv-router all targets under the full feature set, under
standalone-slot-tracker alone, and under standalone-selection alone
(the split replica-sync module compiles in all three), and on
dynamo-llm, dynamo-mocker, dynamo-ext-proc, dynamo-sglang-sidecar,
dynamo-custom-policy-builtin, dynamo-bench (--tests); builtin tests
56+5+1+3+20+3+4; cargo test -p dynamo-llm kv_router (485) and --lib watcher
(36); dynamo-mocker --lib 304; router layering gate 24 checks, 0 failures;
CODEOWNERS strict clean; cargo fmt --check clean; move-diff vs
e9256a3 common=7281 new=123 dropped=62, every line one of the
boundary edits named above.

Signed-off-by: Ishan Dhanani <ishandhanani@gmail.com>
…ners wrap it

Step 6-1b of the router split (DYNO-179). The selection core's last
transport-bound dependency was WorkerRegistry: one struct holding the
per-partition indexers, their policy and readiness signal, and the ZMQ
listener records, watermarks, and peers, with the ZMQ listener spawned
from its register path.

The index half becomes dynamo_kv_indexer::registry::IndexRegistry, a
stage type: partition -> indexer created on demand with the registry's
policy and metrics, the ready watch, and the claim-vs-remove lifecycle
lock. claim() runs under a LifecycleGuard the caller holds and rejects a
block-size mismatch with the same message register() produced;
release_if_unused() takes the lock, asks the caller whether anything
still names the partition, and removes otherwise (never when the
registry retains empty indexers); get_or_create_indexer() stays the
lock-free path recovery dumps and embedding hosts use. kv-router's
WorkerRegistry keeps the listeners, watermarks, and peers and holds an
Arc<IndexRegistry> it delegates to, so the standalone indexer service,
its HTTP tests, and the slot tracker keep their calls.

KvEventIngress speaks IndexRegistry: open() receives it, reconcile()
and detach() no longer take a registry, and a new attach(index,
cancel_token) hook lets the core hand its index registry and
cancellation scope to whichever ingress the builder installed.
ZmqDirectIngress builds its listener registry there, over the core's
index and inside the core's cancellation scope, so shutdown() cancels
listeners exactly as before. SelectionCore holds Arc<IndexRegistry> and
exposes it; peer recovery (reqwest) and the /dump snapshot move from
the core to the service over that accessor. lib/llm's RuntimeIngress
only changes its open() signature.

Validation: cargo test -p dynamo-kv-indexer --features testing --lib (583);
cargo test -p dynamo-kv-router under the full service feature set (532
lib + 2 plugin-API + 7 standalone HTTP), under standalone-slot-tracker
alone (332), and under standalone-indexer alone (335 + 1 + 6); clippy -D
warnings all targets on kv-indexer (testing) and on kv-router under the
full, slot-tracker-only, indexer-only, and selection-only feature sets,
and on dynamo-llm, dynamo-mocker, dynamo-ext-proc, dynamo-sglang-sidecar,
dynamo-custom-policy-builtin, dynamo-bench (--tests); builtin tests
56+5+1+3+20+3+4; cargo test -p dynamo-llm kv_router (485) and --lib
watcher (36); dynamo-mocker --lib 304; router layering gate 24 checks,
0 failures; CODEOWNERS strict clean; cargo fmt --check clean; move-diff
vs 768e5ea common=7551 new=272 dropped=129, every line one of the
boundary edits named above (the indexer creation path and the lock-free
accessors moved verbatim).

Signed-off-by: Ishan Dhanani <ishandhanani@gmail.com>
…l module

Step 6-2a of the router split (DYNO-179). lib/kv-router/src/protocols.rs
mixed two things: re-exports of the stage crates' wire types, and the
router-local request vocabulary the scheduler kernel owns per the split
design (section 4, rows 40-41): KvHintTransferWorkerMetadata,
WorkerConfigLike, KvTransferEnforcement, RoutingConstraints,
RouterEventSink, RouterRequest, PotentialLoad, RouterResponse,
WorkerSelectionResult, and the two private helpers they use.

Those items move verbatim to scheduling/protocols.rs, the kernel module
that leaves with the rest of scheduling/ in 6-3, together with the tests
that exercise them. protocols.rs keeps the stage re-exports and
re-exports the moved names at their current paths, so no importer
inside or outside the crate changes.

Validation: move-diff vs 70059d9 common=557 new=2 dropped=0 (the
second test module's braces); cargo test -p dynamo-kv-router under the full
service feature set (532 lib + 2 plugin-API + 7 standalone HTTP) and
with default features (304 lib); clippy -D warnings all targets on
kv-router under both, and on dynamo-llm, dynamo-mocker, dynamo-ext-proc,
dynamo-sglang-sidecar, dynamo-custom-policy-builtin, dynamo-bench
(--tests); builtin tests 56+5+1+3+20+3+4; cargo test -p dynamo-llm
kv_router (485) and --lib watcher (36); dynamo-mocker --lib 304; router
layering gate 24 checks, 0 failures; CODEOWNERS strict clean; cargo fmt
--check clean.

Signed-off-by: Ishan Dhanani <ishandhanani@gmail.com>
…fig slice

Step 6-2b of the router split (DYNO-179). TrackingHashContext::from_config
read the flat KvRouterConfig, validated the option combination, and
opened the key file. The kernel must not read files or the flat config,
so the context now takes a resolved slice.

scheduling/tracking_hash.rs (kernel module, moves with the kernel in
6-3) holds the algorithm, the scope, the context with its hashing and
seed derivation, and the new TrackingHashConfig { algorithm, key_id,
provider_key }; TrackingHashContext::new(config) is the only
constructor there. lib/kv-router/src/tracking_hash.rs keeps the
flat-config side: validate_tracking_hash_options (the flat config's
validator still calls it), tracking_hash_config(&KvRouterConfig) with
the former from_config body (validation, key-file read, size check,
same messages), and a from_config shim over the two, so the twelve
callers do not change. The vector, framing, and zeroize tests move
with the context and build their keyed configs from bytes; the
key-file and option-validation tests stay with the resolver.

Validation: move-diff vs 5b21071 common=503 new=52 dropped=11 (the
config struct, new(), the resolver wrapper, and the tests' constructor
lines; hashing and key loading moved verbatim); cargo test -p dynamo-kv-router under the full
service feature set (532 lib + 2 plugin-API + 7 standalone HTTP) and
with default features (304 lib); clippy -D warnings all targets on
kv-router under both, and on dynamo-llm, dynamo-mocker, dynamo-ext-proc,
dynamo-sglang-sidecar, dynamo-custom-policy-builtin, dynamo-bench
(--tests); builtin tests 56+5+1+3+20+3+4; cargo test -p dynamo-llm
kv_router (485) and --lib watcher (36); dynamo-mocker --lib 304; router
layering gate 24 checks, 0 failures; CODEOWNERS strict clean; cargo fmt
--check clean.

Signed-off-by: Ishan Dhanani <ishandhanani@gmail.com>
…are numbers

Follow-up to 6-1a (DYNO-179). Taking axum out of the kernel's error type
left six hand-written status numbers behind a private module: HTTP
vocabulary without the HTTP type. The owner's call is to keep the
types-only http crate for the codes, so SelectionError::status() now
returns http::StatusCode, status_code() is its u16 view, and the HTTP
adapter uses the StatusCode directly instead of re-parsing a number.
The classification itself is unchanged; moving it onto dynamo-error's
error class is the E2 follow-up once #110 merges.

http 1.4 joins the workspace dependencies (already in the lockfile
through axum).

Validation: cargo test -p dynamo-kv-router under the full
service feature set (532 lib + 2 plugin-API + 7 standalone HTTP) and
with default features (304 lib); clippy -D warnings all targets on
kv-router under both, and on dynamo-llm, dynamo-mocker, dynamo-ext-proc,
dynamo-sglang-sidecar, dynamo-custom-policy-builtin, dynamo-bench
(--tests); builtin tests 56+5+1+3+20+3+4; cargo test -p dynamo-llm
kv_router (485) and --lib watcher (36); dynamo-mocker --lib 304; router
layering gate 24 checks, 0 failures; CODEOWNERS strict clean; cargo fmt
--check clean; Cargo.lock only adds http 1.4.1 to kv-router's list.

Signed-off-by: Ishan Dhanani <ishandhanani@gmail.com>
…to the kernel config

Step 6-2c of the router split (DYNO-179). RouterConfigOverride is the
per-request slice of the router configuration the kernel honors (it
already shaped RouterConfig::assume_kv_reuse in 6-0), and
RouterPrefillLoadModel is a kernel setting the flat config carries. Both
lived in scheduling/config.rs beside the flat KvRouterConfig and its
parsers.

They move verbatim to scheduling/router_config.rs, the kernel config
module that leaves with the kernel in 6-3: RouterPrefillLoadModel with
its Display, FromStr, and is_enabled; RouterConfigOverride with its
builder derive, validate, the RouterConfigOverrideSerde compatibility
shape and TryFrom, and the deprecated overlap_score_weight helper only
that TryFrom uses. config.rs re-exports both names, so crate::config,
scheduling::config, and the lib/llm imports keep resolving.

Validation: move-diff vs 0d9fa86 common=2483 new=0 dropped=0 (an exact
move); cargo test -p dynamo-kv-router under the full
service feature set (532 lib + 2 plugin-API + 7 standalone HTTP) and
with default features (304 lib); clippy -D warnings all targets on
kv-router under both, and on dynamo-llm, dynamo-mocker, dynamo-ext-proc,
dynamo-sglang-sidecar, dynamo-custom-policy-builtin, dynamo-bench
(--tests); builtin tests 56+5+1+3+20+3+4; cargo test -p dynamo-llm
kv_router (485) and --lib watcher (36); dynamo-mocker --lib 304; router
layering gate 24 checks, 0 failures; CODEOWNERS strict clean; cargo fmt
--check clean.

Signed-off-by: Ishan Dhanani <ishandhanani@gmail.com>
This was referenced Nov 11, 2026
@PeaBrane

Copy link
Copy Markdown
Contributor

damn dude

This branch was previously deployed

1 inactive deployment
external_collaborator — be6d2291 Deployed Nov 10, 2026 by ishandhanani via ok-to-test #49615
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

actions backend::sglang Relates to the sglang backend backend::vllm Relates to the vllm backend container documentation Improvements or additions to documentation external-contribution Pull request is from an external contributor frontend `python -m dynamo.frontend` and `dynamo-run in=http|text|grpc` planner refactor router Relates to routing, KV-aware routing, etc. xpu

Projects

None yet

Development

Successfully merging this pull request may close these issues.

8 participants