Skip to content

refactor(router): move the admission stage to lib/router/admission - #133

Closed
ishandhanani wants to merge 1 commit into
idhanani/dyn-178-classification-errorfrom
idhanani/dyn-178-admission
Closed

ishandhanani wants to merge 1 commit into
idhanani/dyn-178-classification-errorfrom
idhanani/dyn-178-admission

Conversation

@ishandhanani

@ishandhanani ishandhanani commented Oct 31, 2026 •

Copy link
Copy Markdown
Owner

Step 5e-4 of the router split v2 (DYNO-178). Stacked on #132. The crate move: dynamo-admission at lib/router/admission, depending on dynamo-router-protocol and nothing router-side beyond it. This completes step 5e pending merges.

Summary

  • A move. scheduling/policy_queue.rs → queue.rs; scheduling/queue_admission/mod.rs → placement.rs; scheduling/admission_config.rs → profile.rs; plugins/request_classifier.rs (the plugin API) → classifier/mod.rs with inputs.rs, progress.rs, view.rs, and scheduling/request_classifier.rs → classifier/runtime.rs (the runtime and RequestLifecycle). lib/kv-router re-exports every name at the paths plugins and hosts already use: plugins::request_classifier::*, scheduling::{policy_queue, policy_config, request_classifier, WorkerPlacement, RequestLifecycle, ClassificationError, ...}, config::RouterQueuePolicy. No downstream crate changes.
  • Stays in lib/kv-router. The YAML policy file and its resolver (policy_config.rs, ending in PolicyProfile::family_buckets), RequestClassifierConfig and the registries, the gate over the slot projection, ClassifierRows (the WorkerView over worker selection's rows), and From<ClassificationError> for KvSchedulerError.
  • Tests split by what they build. The 24 policy-queue tests build their profiles through RouterPolicyConfig::from_yaml, so they stay as scheduling/policy_queue_tests.rs and read the queue's internals through three testing-gated accessors (due_entry_count, class_deficit, class_blocks_worker); the entry-size assertion moves with the queue. The classifier runtime's tests move with the runtime; progress, view, and profile carry their own.
  • Visibility the boundary required. Kernel-facing constructors and scheduling methods are pub: ClassifyRequest::{new, with_timing, with_request_id, with_initial_policy_class, with_session_context, with_pinned_worker, with_sequence_hashes, with_busy_thresholds, into_queue_inputs}, ClassificationOverrides and QueueMetadata with their fields, PolicyQueue::{has_ready, any_ready_head, next_due_at, recheck_worker, enqueue_with_due_at, take_if_in_class, take_expired}, RequestClassifierRuntime::{new, worker_inputs, classify_with, begin_request, cached_overrides}. They are documented as kernel-facing in the crate docs; the #[doc(hidden)] attributes on RequestLifecycle are gone (the plan forbids hidden public items). ClassifyRequest's fields stay private: the runtime that reads them is a child module (invariant A5).
  • Features. testing forwards to protocol's and gates the three queue accessors; kv-router's testing forwards and its dev-dependency enables it. Dependencies: protocol, dynamo-tokens (SequenceHash), async-trait, futures-util, ordered-float, parking_lot, rustc-hash, serde, serde_json, thiserror, tokio, tokio-util, tracing.
  • Gate and ownership. scripts/router_layering_check.sh gains the dynamo-admission row (24 checks, 0 failures: router deps equal {dynamo-router-protocol}; no sockets, HTTP, Prometheus, or other router crates under --all-features). .github/codeowners/areas.yaml claims lib/router/admission/ under the router area and CODEOWNERS is regenerated. lib/router/README.md status updated.

Validation

  • cargo test -p dynamo-admission --all-features: 22 passed (runtime, progress, view, profile, queue entry size).

  • cargo test -p dynamo-kv-router --features standalone-selection,standalone-slot-tracker,metrics,testing: 532 + 2 + 7 passed (532 + 22 moved = the 554 before the move).

  • cargo test -p dynamo-custom-policy-builtin --all-features: 56 + 5 + 1 + 3 + 20 + 3 + 4 passed.

  • cargo test -p dynamo-llm kv_router: 485 passed; cargo test -p dynamo-llm --lib watcher: 36 passed; cargo test -p dynamo-mocker --lib: 304 passed.

  • cargo clippy --all-features --all-targets -D warnings on admission, kv-router, builtin: clean; cargo clippy --tests -D warnings on lib/llm (default features), mocker, ext-proc, bench: clean; cargo check --tests on lib/llm, mocker, bench, sglang-sidecar, ext-proc: clean.

  • scripts/router_layering_check.sh: 24 checks, 0 failures.

  • scripts/router_move_diff.py 45a8f95e46 WORKTREE from the kv-router paths to the new crate plus the kv-router remainder: 3,941 common / 55 new / 49 dropped. New and dropped are the pub(crate) → pub promotions above, ClassifyRequest's three fields losing pub(crate), and the queue tests reading due_entry_count/class_deficit/class_blocks_worker instead of the private fields.

Bench: policy queue across the crate boundary

lib/kv-router/benches/policy_queue.rs, A = #132 head 45a8f95e46 (queue in kv-router) vs B = this head 37e939a361 (queue in dynamo-admission). Both binaries built in the same tree (git checkout <commit> -- lib/ Cargo.toml Cargo.lock, release profile: thin LTO, codegen-units 1), run interleaved A, B, A, B on one idle core (taskset -c 9), separate CRITERION_HOME; criterion median point estimates.

Full suite, two rounds (2 s measurement), B/A per round: pop_once_exact 1..512 lanes -0.2..-3.2%; pop_once_exact/1024 +7.0% / -0.7%; pop_once_exact/10000 +6.6% / +10.8%; drain_4096_exact 1..512 +0.3..+4.4%; drain_one_per_exact_lane 8..512 +0.1..+4.4%.

Focused re-run of the groups that stood out, three rounds (3 s measurement):

group A1 B1 A2 B2 A3 B3 B/A per round median
pop_once_exact/1024 (ns) 493 455 486 434 444 452 -7.7% / -10.5% / +1.8% -7.7%
pop_once_exact/10000 (ns) 1477 1294 1372 1360 1345 1355 -12.4% / -0.8% / +0.7% -0.8%
drain_4096_exact/8 (µs) 797.8 810.8 811.2 783.0 817.9 789.7 +1.6% / -3.5% / -3.4% -3.4%
drain_one_per_exact_lane/512 (µs) 102.3 101.0 101.9 101.7 101.1 102.0 -1.3% / -0.3% / +0.9% -0.3%

The deltas flip sign between sessions and rounds, within the 1-6% drift documented for this box (a leftover sglang::scheduler holds one core and migrates). Expected structurally: PolicyQueue<T> and its hot methods are generic, so they monomorphize in the caller's crate, and nothing in the queue's per-lane loop calls a non-generic function across the crate boundary. No measurable change.

@github-actions github-actions Bot added refactor documentation Improvements or additions to documentation labels Oct 31, 2026
@ishandhanani
ishandhanani added this pull request to stack #158 November 1, 2026 06:36
Step 5e-4 of the router split (DYNO-178): dynamo-admission, a stage
crate that depends on dynamo-router-protocol and nothing router-side
beyond it. A move: the files that made up the stage inside
lib/kv-router move as they are, and lib/kv-router re-exports every name
at the path plugins and hosts already use.

Moved: scheduling/policy_queue.rs as queue, scheduling/queue_admission
as placement, scheduling/admission_config.rs as profile, and the
classifier side as classifier/{mod (the plugin API), inputs, progress,
view, runtime (scheduling/request_classifier.rs: the runtime and the
request lifecycle)}.

Stays in lib/kv-router: the YAML policy file and its resolver
(policy_config.rs), the classifier instance config and the registries,
the gate over the slot projection, ClassifierRows (the WorkerView over
worker selection's rows), and the ClassificationError conversion. The
24 policy-queue tests build their profiles through the YAML resolver,
so they stay as scheduling/policy_queue_tests.rs and read the queue's
internals through three testing-gated accessors (due_entry_count,
class_deficit, class_blocks_worker); the entry-size test moves with the
queue. The classifier runtime's tests move with it.

Visibility the crate boundary required: the kernel-facing constructors
and scheduling methods are pub (ClassifyRequest::with_*, into_queue_
inputs; ClassificationOverrides and QueueMetadata with their fields;
PolicyQueue::{has_ready, any_ready_head, next_due_at, recheck_worker,
enqueue_with_due_at, take_if_in_class, take_expired};
RequestClassifierRuntime and its methods), documented as kernel-facing
in the crate docs instead of #[doc(hidden)]. ClassifyRequest's fields
stay private: the runtime that reads them is a child module.

Layering gate: a row for dynamo-admission (24 checks). CODEOWNERS gains
lib/router/admission/ under the router area.

Validation: admission 22 tests; kv-router 532 + 2 + 7 (532 + 22 moved = the
554 before); builtin plugin, lib/llm kv_router 485 and watcher 36,
mocker 304; clippy clean on admission, kv-router, builtin, lib/llm,
mocker, ext-proc, bench; layering gate 24 checks; move-diff 3,941
common / 55 new / 49 dropped (visibility and the test hooks).

Signed-off-by: Ishan Dhanani <ishandhanani@gmail.com>
@ishandhanani

Copy link
Copy Markdown
Owner Author

Landed upstream as ai-dynamo#18055 (squash commit 79692cf on main, 2026-11-11). Closing this fork review unit.

This branch had an error being deployed

1 failed deployment
external_collaborator — b964520a Deployed Nov 2, 2026 by ishandhanani via ok-to-test #521
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

documentation Improvements or additions to documentation refactor

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant