Repository navigation
refactor(router): move the admission stage to lib/router/admission - #133
Closed
ishandhanani wants to merge 1 commit into
Closed
ishandhanani wants to merge 1 commit into
ishandhanani wants to merge 1 commit into
Conversation
ishandhanani
had a problem deploying
to
external_collaborator
October 31, 2026 03:00 — with
GitHub Actions
Failure
Step 5e-4 of the router split (DYNO-178): dynamo-admission, a stage
crate that depends on dynamo-router-protocol and nothing router-side
beyond it. A move: the files that made up the stage inside
lib/kv-router move as they are, and lib/kv-router re-exports every name
at the path plugins and hosts already use.
Moved: scheduling/policy_queue.rs as queue, scheduling/queue_admission
as placement, scheduling/admission_config.rs as profile, and the
classifier side as classifier/{mod (the plugin API), inputs, progress,
view, runtime (scheduling/request_classifier.rs: the runtime and the
request lifecycle)}.
Stays in lib/kv-router: the YAML policy file and its resolver
(policy_config.rs), the classifier instance config and the registries,
the gate over the slot projection, ClassifierRows (the WorkerView over
worker selection's rows), and the ClassificationError conversion. The
24 policy-queue tests build their profiles through the YAML resolver,
so they stay as scheduling/policy_queue_tests.rs and read the queue's
internals through three testing-gated accessors (due_entry_count,
class_deficit, class_blocks_worker); the entry-size test moves with the
queue. The classifier runtime's tests move with it.
Visibility the crate boundary required: the kernel-facing constructors
and scheduling methods are pub (ClassifyRequest::with_*, into_queue_
inputs; ClassificationOverrides and QueueMetadata with their fields;
PolicyQueue::{has_ready, any_ready_head, next_due_at, recheck_worker,
enqueue_with_due_at, take_if_in_class, take_expired};
RequestClassifierRuntime and its methods), documented as kernel-facing
in the crate docs instead of #[doc(hidden)]. ClassifyRequest's fields
stay private: the runtime that reads them is a child module.
Layering gate: a row for dynamo-admission (24 checks). CODEOWNERS gains
lib/router/admission/ under the router area.
Validation: admission 22 tests; kv-router 532 + 2 + 7 (532 + 22 moved = the
554 before); builtin plugin, lib/llm kv_router 485 and watcher 36,
mocker 304; clippy clean on admission, kv-router, builtin, lib/llm,
mocker, ext-proc, bench; layering gate 24 checks; move-diff 3,941
common / 55 new / 49 dropped (visibility and the test hooks).
Signed-off-by: Ishan Dhanani <ishandhanani@gmail.com>
ishandhanani
force-pushed
the
idhanani/dyn-178-admission
branch
from
November 2, 2026 13:30
37e939a to
b964520
Compare
ishandhanani
had a problem deploying
to
external_collaborator
November 2, 2026 13:30 — with
GitHub Actions
Failure
ishandhanani
had a problem deploying
to
external_collaborator
November 2, 2026 13:53 — with
GitHub Actions
Failure
Owner
Author
|
Landed upstream as ai-dynamo#18055 (squash commit 79692cf on main, 2026-11-11). Closing this fork review unit. |
This branch had an error being deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Step 5e-4 of the router split v2 (DYNO-178). Stacked on #132. The crate move:
dynamo-admissionatlib/router/admission, depending ondynamo-router-protocoland nothing router-side beyond it. This completes step 5e pending merges.Summary
scheduling/policy_queue.rs→queue.rs;scheduling/queue_admission/mod.rs→placement.rs;scheduling/admission_config.rs→profile.rs;plugins/request_classifier.rs(the plugin API) →classifier/mod.rswithinputs.rs,progress.rs,view.rs, andscheduling/request_classifier.rs→classifier/runtime.rs(the runtime andRequestLifecycle).lib/kv-routerre-exports every name at the paths plugins and hosts already use:plugins::request_classifier::*,scheduling::{policy_queue, policy_config, request_classifier, WorkerPlacement, RequestLifecycle, ClassificationError, ...},config::RouterQueuePolicy. No downstream crate changes.lib/kv-router. The YAML policy file and its resolver (policy_config.rs, ending inPolicyProfile::family_buckets),RequestClassifierConfigand the registries, the gate over the slot projection,ClassifierRows(theWorkerViewover worker selection's rows), andFrom<ClassificationError> for KvSchedulerError.RouterPolicyConfig::from_yaml, so they stay asscheduling/policy_queue_tests.rsand read the queue's internals through threetesting-gated accessors (due_entry_count,class_deficit,class_blocks_worker); the entry-size assertion moves with the queue. The classifier runtime's tests move with the runtime;progress,view, andprofilecarry their own.pub:ClassifyRequest::{new, with_timing, with_request_id, with_initial_policy_class, with_session_context, with_pinned_worker, with_sequence_hashes, with_busy_thresholds, into_queue_inputs},ClassificationOverridesandQueueMetadatawith their fields,PolicyQueue::{has_ready, any_ready_head, next_due_at, recheck_worker, enqueue_with_due_at, take_if_in_class, take_expired},RequestClassifierRuntime::{new, worker_inputs, classify_with, begin_request, cached_overrides}. They are documented as kernel-facing in the crate docs; the#[doc(hidden)]attributes onRequestLifecycleare gone (the plan forbids hidden public items).ClassifyRequest's fields stay private: the runtime that reads them is a child module (invariant A5).testingforwards to protocol's and gates the three queue accessors; kv-router'stestingforwards and its dev-dependency enables it. Dependencies: protocol,dynamo-tokens(SequenceHash), async-trait, futures-util, ordered-float, parking_lot, rustc-hash, serde, serde_json, thiserror, tokio, tokio-util, tracing.scripts/router_layering_check.shgains thedynamo-admissionrow (24 checks, 0 failures: router deps equal{dynamo-router-protocol}; no sockets, HTTP, Prometheus, or other router crates under--all-features)..github/codeowners/areas.yamlclaimslib/router/admission/under the router area andCODEOWNERSis regenerated.lib/router/README.mdstatus updated.Validation
cargo test -p dynamo-admission --all-features: 22 passed (runtime, progress, view, profile, queue entry size).cargo test -p dynamo-kv-router --features standalone-selection,standalone-slot-tracker,metrics,testing: 532 + 2 + 7 passed (532 + 22 moved = the 554 before the move).cargo test -p dynamo-custom-policy-builtin --all-features: 56 + 5 + 1 + 3 + 20 + 3 + 4 passed.cargo test -p dynamo-llm kv_router: 485 passed;cargo test -p dynamo-llm --lib watcher: 36 passed;cargo test -p dynamo-mocker --lib: 304 passed.cargo clippy --all-features --all-targets -D warningson admission, kv-router, builtin: clean;cargo clippy --tests -D warningson lib/llm (default features), mocker, ext-proc, bench: clean;cargo check --testson lib/llm, mocker, bench, sglang-sidecar, ext-proc: clean.scripts/router_layering_check.sh: 24 checks, 0 failures.scripts/router_move_diff.py 45a8f95e46 WORKTREEfrom the kv-router paths to the new crate plus the kv-router remainder: 3,941 common / 55 new / 49 dropped. New and dropped are thepub(crate)→pubpromotions above,ClassifyRequest's three fields losingpub(crate), and the queue tests readingdue_entry_count/class_deficit/class_blocks_workerinstead of the private fields.Bench: policy queue across the crate boundary
lib/kv-router/benches/policy_queue.rs, A = #132 head45a8f95e46(queue in kv-router) vs B = this head37e939a361(queue indynamo-admission). Both binaries built in the same tree (git checkout <commit> -- lib/ Cargo.toml Cargo.lock, release profile: thin LTO, codegen-units 1), run interleaved A, B, A, B on one idle core (taskset -c 9), separateCRITERION_HOME; criterion median point estimates.Full suite, two rounds (2 s measurement), B/A per round:
pop_once_exact1..512 lanes -0.2..-3.2%;pop_once_exact/1024+7.0% / -0.7%;pop_once_exact/10000+6.6% / +10.8%;drain_4096_exact1..512 +0.3..+4.4%;drain_one_per_exact_lane8..512 +0.1..+4.4%.Focused re-run of the groups that stood out, three rounds (3 s measurement):
The deltas flip sign between sessions and rounds, within the 1-6% drift documented for this box (a leftover
sglang::schedulerholds one core and migrates). Expected structurally:PolicyQueue<T>and its hot methods are generic, so they monomorphize in the caller's crate, and nothing in the queue's per-lane loop calls a non-generic function across the crate boundary. No measurable change.