Skip to content

refactor(router): move the scheduling kernel into lib/router/scheduler - #141

Closed
ishandhanani wants to merge 1 commit into
idhanani/dyn-179-kernel-config-typesfrom
idhanani/dyn-179-scheduler-crate
Closed

ishandhanani wants to merge 1 commit into
idhanani/dyn-179-kernel-config-typesfrom
idhanani/dyn-179-scheduler-crate

Conversation

@ishandhanani

Copy link
Copy Markdown
Owner

Step 6-3 of the router split v2 (DYNO-179). Stacked on the 6-2c PR. The move: lib/router/scheduler (dynamo-scheduler) is the kernel crate; the scheduling kernel files leave lib/kv-router/src/scheduling/, and kv-router keeps scheduling as a shim.

Summary

  • Crate. dynamo-scheduler owns types (+ RejectThresholdsProvider, formerly in policy_config.rs), protocols, filter, queue (+ capacity), local, overlap, overlap_refresh, prefill_load, shared_cache, selector/*, classifier_rows, config (formerly router_config, with RouterConfigOverride and RouterPrefillLoadModel), tracking_hash, kv_hints (the transfer-source resolution that lived in kv-router's kv_hints.rs), identity (re-exports), and test_utils behind testing/bench. Dependencies: the five stages, protocol, kv-hashing, kv-events, tokens, http; gate row forbids dynamo-runtime, dynamo-kv-router, dynamo-zmq, axum, hyper, reqwest, zmq, prometheus.
  • Import flip (inventory split-v2/inventory-scheduler.md §7.1): crate::plugins::worker_selection::* -> dynamo_worker_selection::* (the derive *Ext traits stay crate::selector), crate::plugins::request_classifier::* -> dynamo_admission::classifier::*, crate::sequences::* -> dynamo_slot_tracker::*, crate::indexer::* -> dynamo_kv_indexer::*, crate::conditional_disagg::* -> dynamo_worker_selection::conditional_disagg::*, super::policy_config::{..} -> dynamo_slot_tracker::busy / dynamo_admission / crate::types, super::{request_classifier,policy_queue,queue_admission} -> dynamo_admission::{classifier,queue,placement}, crate::WorkerType -> dynamo_router_protocol::worker_type::WorkerType. The kernel's protocols module re-exports the wire vocabulary from the owning stages so crate::protocols::X keeps resolving inside the kernel.
  • kv-router shim. scheduling/mod.rs keeps the staying modules (config, policy_config, admission_config, policy_queue, request_classifier, queue_admission, lora_filter, policy) and re-exports every moved name at its previous path, with shim submodules for queue, selector, overlap, overlap_refresh, prefill_load, shared_cache, protocols, router_config, tracking_hash, classifier_rows; kv_hints.rs, test_utils.rs, and protocols.rs point at the kernel; policy_config.rs re-exports RejectThresholdsProvider; config.rs reads RouterConfigOverride/RouterPrefillLoadModel from dynamo_scheduler::config. The fourteen external dynamo_kv_router::scheduling::* paths (lib/llm, builtin, mocker, bindings, bench, examples) compile unchanged; step 8 flips hosts to the owners.
  • Boundary notes. Fourteen queue tests build profiles from the YAML policy file (kv-router's parser), so they moved to kv-router as scheduling/queue_yaml_tests.rs with the seven helpers they use; the kernel keeps its other 51 queue tests. What they and the composed core reach became public: enqueue_admitted_with_block_hashes_and_lease, build_classify_request, validate_classification, capacity_changed, free_if_booking_with_cleanup, apply_published_worker_configs, RequestLifecycleLease, ClassifierRows, tracking_hash::KEY_SIZE; testing-gated accessors SchedulerQueue::profile(), SchedulerQueue::hold_admission_permits() (opaque, so AdmissionCommand stays private), SchedulerCleanupAck::response(). KvSchedulerError drops #[non_exhaustive] so kv-router's status mapping stays an exhaustive match. TrackingHashContext::from_config becomes the free tracking_hash_context(&KvRouterConfig) (an inherent impl on a foreign type is not allowed); its twelve callers in kv-router, lib/llm, and the mocker call it. The reference picker is gated any(test, testing, bench) in the kernel so kv-router's tests see it.
  • Docs. scheduling/AGENTS.md + CLAUDE.md (the queue-actor guide) move to the kernel crate root as lib/router/scheduler/{AGENTS,CLAUDE}.md, retitled.
  • Workspace. Member + path dependency, areas.yaml claims lib/router/scheduler/, layering gate row, kv-router depends on the kernel and forwards testing/bench.

Validation

  • cargo test -p dynamo-scheduler --features testing: 171 pass (--all-features --lib: 171); cargo clippy --all-targets -- -D warnings on the kernel under both feature sets: clean. cargo tree -p dynamo-scheduler -e normal names no dynamo-runtime, dynamo-kv-router, dynamo-zmq, axum, hyper, reqwest, zmq, or prometheus.
  • cargo test -p dynamo-kv-router: 361 lib + 2 plugin-API + 7 standalone-HTTP (the kernel's 171 moved out: 361 + 171 = 532, the pre-move count) under the full service feature set (the fourteen moved YAML tests included); 133 lib with default features; 161 lib with standalone-slot-tracker alone (the kernel's own 171 tests left this crate).
  • cargo clippy --all-targets -- -D warnings: kv-router under the full, default, slot-tracker-only, indexer-only, and selection-only feature sets; downstream dynamo-llm, dynamo-mocker, dynamo-ext-proc, dynamo-sglang-sidecar, dynamo-custom-policy-builtin, dynamo-bench --tests; the two example policy crates: clean.
  • Builtin policy tests 56+5+1+3+20+3+4; cargo test -p dynamo-llm kv_router 485 and --lib watcher 36; cargo test -p dynamo-mocker --lib 304.
  • scripts/router_layering_check.sh: 26 checks, 0 failures (the scheduler row included). CODEOWNERS --strict clean with lib/router/scheduler/ claimed; regenerated CODEOWNERS committed. cargo fmt --all -- --check clean.
  • scripts/router_move_diff.py 9d11daff6e WORKTREE from the kv-router paths (the moved scheduling/ files, selector/, queue/, test_utils.rs, kv_hints.rs, protocols.rs, tracking_hash.rs, identity.rs, lib.rs, the shim modules, the selection service/core/test files, the tracking-hash bench, lib/llm kv_router.rs, the two mocker files) to lib/router/scheduler/src plus the same kv-router files: common=32706 new=271 dropped=108. The listed lines are of four kinds: inline paths rewritten to the owning stage (crate::sequences::WorkerLoadProjection -> dynamo_slot_tracker::WorkerLoadProjection, crate::config::RouterQueuePolicy -> dynamo_admission::RouterQueuePolicy, crate::BusyThresholds -> dynamo_slot_tracker::busy::BusyThresholds, and the like; use lines are normalized away but paths inside bodies are not), the seven queue-test helpers that now exist in both the kernel and queue_yaml_tests.rs, the testing accessors and the public RequestLifecycleLease/ClassifierRows, and tracking_hash_context replacing the inherent from_config. No logic line changed (artifact s6-3-move-diff.txt).

Step 6-3 of the router split (DYNO-179). dynamo-scheduler
(lib/router/scheduler) is the kernel crate: the request and response
vocabulary (types, protocols), eligibility (filter), the queue actor and
LocalScheduler, overlap and prefill-load analysis, the shared-cache port,
the selection kernel over the worker-selection stage (selector,
classifier_rows), the validated kernel configuration (config, formerly
router_config, with RouterConfigOverride and RouterPrefillLoadModel), the
tracking hash, the KV-hint transfer-source resolution that was kv-router's
kv_hints.rs, and the test helpers. It depends on the five stage crates,
dynamo-router-protocol, dynamo-kv-hashing, dynamo-kv-events, and
dynamo-tokens, plus http for the status codes; the layering gate forbids
dynamo-runtime, dynamo-kv-router, dynamo-zmq, axum, hyper, reqwest, zmq,
and prometheus in its tree.

The files move with git mv and their crate-internal imports flip from
kv-router's shims to the owning stage paths: plugins::worker_selection to
dynamo_worker_selection (the derive Ext traits stay kernel), plugins::
request_classifier to dynamo_admission::classifier, sequences to
dynamo_slot_tracker, indexer to dynamo_kv_indexer, policy_config's names
to dynamo_slot_tracker::busy and dynamo_admission, and the
RejectThresholdsProvider hook type to the kernel's types module. The
kernel's protocols module re-exports the wire vocabulary the kernel reads
from the owning stages, so crate::protocols keeps resolving inside it.

dynamo-kv-router keeps `scheduling` as a shim: the staying modules (flat
config, policy-file parsing, admission and worker-selection shims, the
dead policy module) plus re-exports of every moved name at its previous
path, so the fourteen external import paths and every in-crate importer
compile unchanged until step 8 flips hosts to the owners.

The scheduling AGENTS.md/CLAUDE.md pair (the queue-actor guide) moves to
the kernel crate root.

Validation: move-diff vs 9d11daf common=32706 new=271 dropped=108
(inline paths rewritten to the owning stage, the seven queue-test
helpers present on both sides, the testing accessors, and the
tracking-hash free function; no logic line changed); cargo test -p dynamo-scheduler --features
testing (171; the same 171 under --all-features); clippy -D warnings all
targets on the kernel under both feature sets; cargo tree -p
dynamo-scheduler -e normal names no dynamo-runtime, dynamo-kv-router,
dynamo-zmq, axum, hyper, reqwest, zmq, or prometheus; cargo test -p
dynamo-kv-router (361 lib + 2 plugin-API + 7 standalone
HTTP; 361 + the kernel's 171 = the pre-move 532) under the full service feature set, 133 lib
with default features, 161 lib under standalone-slot-tracker alone;
clippy -D warnings all targets on kv-router under the full, default,
slot-tracker-only, indexer-only, and selection-only feature sets, on
dynamo-llm, dynamo-mocker, dynamo-ext-proc, dynamo-sglang-sidecar,
dynamo-custom-policy-builtin, dynamo-bench (--tests), and the five
example policy crates; builtin tests 56+5+1+3+20+3+4; cargo test -p
dynamo-llm kv_router (485) and --lib watcher (36); dynamo-mocker --lib
304; router layering gate 26 checks, 0 failures (the scheduler row
included); CODEOWNERS strict clean with lib/router/scheduler/ claimed;
cargo fmt --check clean.

Signed-off-by: Ishan Dhanani <ishandhanani@gmail.com>
@ishandhanani

Copy link
Copy Markdown
Owner Author

Landed upstream as ai-dynamo#18055 (squash commit 79692cf on main, 2026-11-11). Closing this fork review unit.

This branch had an error being deployed

1 failed deployment
external_collaborator — 63f960f6 Deployed Nov 2, 2026 by ishandhanani via ok-to-test #486
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

documentation Improvements or additions to documentation refactor

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant