Repository navigation
refactor(router): open the admission stage and move it to lib/router/admission - #182
Closed
ishandhanani wants to merge 5 commits into
Closed
ishandhanani wants to merge 5 commits into
ishandhanani wants to merge 5 commits into
Conversation
Step 5e-0 of the router split (DYNO-178). The column-group bitset a stage component declares (WorkerInputs) and the static facts about a rank every column carries (WorkerInfo: identity plus advertised capacity) move from dynamo-worker-selection to a new columns module in dynamo-router-protocol. Worker selection builds its rows from them and admission's classifier will read workers through them, so neither stage should have to name the other. Worker selection re-exports both, so every existing path in kv-router, lib/llm, and the plugins holds. WorkerInputs::without, which only the composed policy used, is public in its new home. Validation: protocol 10 and worker-selection 41 tests; kv-router 554 + 2 + 7; builtin plugin and lib/llm kv_router 485 tests; clippy clean on the stage crates and lib/llm, mocker, ext-proc, bench; layering gate 22 checks; move-diff 458 common / 1 new / 1 dropped. Signed-off-by: Ishan Dhanani <ishandhanani@gmail.com>
… view Step 5e-1 of the router split (DYNO-178). RequestClassifier::classify takes `&dyn WorkerView` instead of worker selection's `WorkerCandidates<'_>`. The view is the admission stage's own vocabulary: one row per eligible rank, `info(row)` for identity and capacity, and `load(row)` / `cache(row)` as plain column structs (WorkerLoadView, WorkerCacheView) that read as None unless the classifier declared the group. The kernel implements it over worker selection's rows with the ClassifierRows newtype (scheduling/classifier_rows.rs), so each column means what it means in a filter or scorer, and the admission crate will depend on the protocol crate alone. This is the one plugin-API signature change of the admission step. The in-tree implementations (the ThunderAgent classifier, the lib/llm watcher and routing-host test classifiers, the kv-router and builtin test classifiers) change their parameter type and nothing else; none of them read the table beyond `is_empty`. Groups other than CACHE and LOAD are selection's and read as None through the view; no in-tree classifier declared them. NoWorkers is the empty view. Validation: kv-router 554 + 2 + 7; builtin plugin all targets; lib/llm kv_router 485 and discovery watcher 36 tests; clippy clean on the stage crates and lib/llm, mocker, ext-proc, bench; downstream check clean; layering gate 22 checks; move-diff over the touched kv-router files 9,243 common / 124 new / 39 dropped (the view module, the newtype, and the signature and test-site changes). Signed-off-by: Ishan Dhanani <ishandhanani@gmail.com>
Step 5e-2 of the router split (DYNO-178). The resolved half of the
policy config leaves the YAML behind.
- BusyThresholds (the per-rank limits, is_configured, has_room) moves
to dynamo-router-protocol's load module beside WorkerCapacity and
RankLoad: admission configures it per class, the kernel's gate
evaluates it, classifiers read it. The slot tracker re-exports it and
keeps the rank checks over its own projection as BusyRanksExt
(rank_is_full, worker_is_full_in_ranks).
- RouterQueuePolicy, PolicyClassConfig, PolicyProfile, and the family-
bucket classifier move to scheduling/admission_config.rs, the stage's
resolved config, with a public validated constructor:
PolicyProfile::family_buckets(FamilyBucketTable) refuses an index out
of range, a bucket list that does not start at zero or is not
strictly increasing, or a table whose size is not families times
buckets, returning AdmissionConfigError::Validation. The raw YAML
forms, the file reader, RouterSettings, the identifier rules, and
every message the resolver emits today stay in policy_config.rs; the
resolver builds the table and calls the constructor, and
RouterPolicyConfigError wraps the stage's error.
- config.rs and policy_config.rs re-export every moved name, so
dynamo_kv_router::RouterQueuePolicy, scheduling::{BusyThresholds,
PolicyClassConfig, PolicyProfile, RouterPolicyConfigError}, and the
slot tracker's busy::BusyThresholds all still resolve. The mocker's
offline replay, the one caller of rank_is_full outside the slot
tracker, imports BusyRanksExt beside BusyThresholdsExt.
Validation: protocol 10, slot-tracker 130, worker-selection 41,
kv-router 554 + 2 + 7 (resolver messages unchanged), builtin plugin,
lib/llm kv_router 485, mocker 304; clippy clean on the stage crates and
kv-router, lib/llm, mocker, ext-proc, bench; layering gate 22 checks;
move-diff 3878 common / 109 new / 10 dropped.
Signed-off-by: Ishan Dhanani <ishandhanani@gmail.com>
…the view Step 5e-3 of the router split (DYNO-178). RequestClassifierRuntime no longer names KvSchedulerError: classify_with, begin_request, and the terminal-delivery wait return ClassificationError, with one variant per failure the runtime can detect (Panicked, Failed, DuplicateRequestId, InvalidMetadata, LifecycleEnded, Shutdown) and the same messages the kernel's variants carried. The kernel converts with From<ClassificationError> for KvSchedulerError, so its error surface, the selection service's status mapping, and every host path are unchanged. classify_with takes `&dyn WorkerView`; the scheduler wraps the rows it built in ClassifierRows at its two call sites, and the runtime's tests classify over the empty view. WorkerView gains a Sync supertrait: views are read-only snapshots, and a classification future that borrows one must stay Send for spawned callers. Validation: kv-router 554 + 2 + 7; builtin plugin all targets; lib/llm kv_router 485 and discovery watcher 36; clippy clean on kv-router, builtin, lib/llm, mocker, ext-proc, bench; downstream check clean; layering gate 22 checks; move-diff 3,788 common / 91 new / 64 dropped. Signed-off-by: Ishan Dhanani <ishandhanani@gmail.com>
Step 5e-4 of the router split (DYNO-178): dynamo-admission, a stage
crate that depends on dynamo-router-protocol and nothing router-side
beyond it. A move: the files that made up the stage inside
lib/kv-router move as they are, and lib/kv-router re-exports every name
at the path plugins and hosts already use.
Moved: scheduling/policy_queue.rs as queue, scheduling/queue_admission
as placement, scheduling/admission_config.rs as profile, and the
classifier side as classifier/{mod (the plugin API), inputs, progress,
view, runtime (scheduling/request_classifier.rs: the runtime and the
request lifecycle)}.
Stays in lib/kv-router: the YAML policy file and its resolver
(policy_config.rs), the classifier instance config and the registries,
the gate over the slot projection, ClassifierRows (the WorkerView over
worker selection's rows), and the ClassificationError conversion. The
24 policy-queue tests build their profiles through the YAML resolver,
so they stay as scheduling/policy_queue_tests.rs and read the queue's
internals through three testing-gated accessors (due_entry_count,
class_deficit, class_blocks_worker); the entry-size test moves with the
queue. The classifier runtime's tests move with it.
Visibility the crate boundary required: the kernel-facing constructors
and scheduling methods are pub (ClassifyRequest::with_*, into_queue_
inputs; ClassificationOverrides and QueueMetadata with their fields;
PolicyQueue::{has_ready, any_ready_head, next_due_at, recheck_worker,
enqueue_with_due_at, take_if_in_class, take_expired};
RequestClassifierRuntime and its methods), documented as kernel-facing
in the crate docs instead of #[doc(hidden)]. ClassifyRequest's fields
stay private: the runtime that reads them is a child module.
Layering gate: a row for dynamo-admission (24 checks). CODEOWNERS gains
lib/router/admission/ under the router area.
Validation: admission 22 tests; kv-router 532 + 2 + 7 (532 + 22 moved = the
554 before); builtin plugin, lib/llm kv_router 485 and watcher 36,
mocker 304; clippy clean on admission, kv-router, builtin, lib/llm,
mocker, ext-proc, bench; layering gate 24 checks; move-diff 3,941
common / 55 new / 49 dropped (visibility and the test hooks).
Signed-off-by: Ishan Dhanani <ishandhanani@gmail.com>
ishandhanani
had a problem deploying
to
external_collaborator
November 2, 2026 13:53 — with
GitHub Actions
Failure
Owner
Author
|
Landed upstream as ai-dynamo#18055 (squash commit 79692cf on main, 2026-11-11). Closing this fork review unit. |
This branch had an error being deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Router split v2, unit 09 of 17. Stacked on #181 (unit 08).
Summary
dynamo-admission(lib/router/admission) admits, holds, or rejects now: the per-class policy queue, resolved profiles, the classifier runtime and request lifecycle, the classifier plugin API with its ownWorkerViewtrait,AdmissionConfig,ClassifierConfig,AdmissionError; it depends on protocol only, with atestingfeature, and its classifier API is tier 1 through kv-router's prelude. Four boundary changes first (#129WorkerInputsandWorkerInfoto protocol'scolumns; #130 classifiers read workers through an admission-owned view; #131 the stage's own resolved config, withBusyThresholdsto protocol; #132ClassificationErrorreported by the runtime), then #133 moves the files. Rule applied: the producer owns its output; the one plugin-API signature change (#130,classify(request, &dyn WorkerView)) is what keeps the stage from reaching the kernel's candidate table. kv-router re-exports every name at the paths plugins and hosts already use. Replaces fork PRs #129, #130, #131, #132, #133.What moves
12dd91e993refactor(router): move WorkerInputs and WorkerInfo to the protocol crate: from worker-selection'sinputs.rstolib/router/protocol/src/columns.rs; worker-selection re-exports them. Move-diff (its body): 458 common / 1 new / 1 dropped, the one pair beingpub(crate) fn withoutbecomingpub fn without.a90f585625refactor(router): classifiers read workers through an admission-owned view:RequestClassifier::classify(request, &dyn WorkerView)replacesWorkerCandidates<'_>;view.rs,classifier_rows.rs. Move-diff (its body): 9,243 common / 124 new / 39 dropped (new:view.rs,classifier_rows.rs, the&dyn WorkerViewparameters, the ported capacity-check test; dropped: theWorkerCandidates<'_>parameters, theread(|workers| classify(..))closures, theiter()form of that test).897923e4ffrefactor(router): give the admission stage its own resolved config:BusyThresholdsto protocolload(the slot tracker re-exports it and keeps the rank checks asBusyRanksExt);scheduling/admission_config.rswithRouterQueuePolicy,PolicyClassConfig,PolicyProfile,PolicyProfile::family_buckets,SYNTHETIC_POLICY_CLASS. Move-diff (its body): 3,878 common / 109 new / 10 dropped (new: thefamily_bucketsconstructor and its checks,FamilyBucketTable,AdmissionConfigErrorand itsFrom, theBusyRanksExttrait, two#[inline]hints; dropped: the struct-literal profile construction, theUncachedIslBucketvector in the resolver,pub(crate)onresolve_class_index_strict).5b8c1f3ebbrefactor(router): classifier runtime reports its own error and takes the view:ClassificationErrorwithFromintoKvSchedulerError;classify_with(request, &dyn WorkerView). Move-diff (its body): 3,788 common / 91 new / 64 dropped (new: the error enum, theFromimpl, the view parameter and the two call-site wraps,NoWorkersin the runtime tests, theSyncbound; dropped: theKvSchedulerErrorvariant constructions and signatures in the runtime, theWorkerInputRows::empty()arguments, the in-runtimeClassifierRowswrap).b964520ae4refactor(router): move the admission stage to lib/router/admission:policy_queue.rstoqueue.rs,queue_admission/mod.rstoplacement.rs,admission_config.rstoprofile.rs, the classifier plugin API toclassifier/mod.rswithinputs.rs,progress.rs,view.rs, and the runtime toclassifier/runtime.rs; the 24 policy-queue tests stay in kv-router aspolicy_queue_tests.rsover threetesting-gated accessors. Move-diff (its body): 3,941 common / 55 new / 49 dropped (thepub(crate)topubpromotions,ClassifyRequest's three fields losingpub(crate), the queue tests readingdue_entry_count/class_deficit/class_blocks_workerinstead of the private fields).Dependency edges
dynamo-admission-> dynamo-router-protocol (router deps; dynamo-runtime, dynamo-kv-router, dynamo-kv-indexer, dynamo-slot-tracker, dynamo-session-affinity, dynamo-worker-selection, dynamo-kv-events, dynamo-kv-hashing, dynamo-zmq, axum, hyper, reqwest, zmq, prometheus forbidden in its tree; direct column pinned in unit 14)Validation
Unit-level move-diff, run from the worktree at the unit head
b964520ae4:python3 -I scripts/router_move_diff.py 7598a008de b964520ae4 --from <17 paths> --to <28 paths>, the union of the replaced fork PRs'--from/--tolists filtered to the paths present at each end (full command in the artifact/tmp/rr-split/unit-prs/09-move-diff.txt).371 new / 154 dropped (the per-PR runs sum to 380 / 163). By category:
ClassificationError(the enum, itsFromintoKvSchedulerError, themap_errsites, theErr(ClassificationError::LifecycleEnded(_))assertions) replacing theKvSchedulerErrorvariant constructions (#132); the&dyn WorkerViewparameter (&NoWorkers15x,ClassifierRows(workers)) replacingWorkerInputRows::empty()(15x) and_workers: WorkerCandidates<'_>(13x) (#130);PolicyProfile::family_buckets,FamilyBucketTable,AdmissionConfigErrorand its validation messages replacing the struct-literal profile construction and theUncachedIslBucketvector (#131);BusyRanksExtand the two#[inline]hints (#131); visibility promotions for the move (64 / 39) and the queue tests readingdue_entry_count/class_deficit/class_blocks_workerinstead of the private fields (#133);pub(crate) fn withouttopub fn without(#129). Each fork PR body explains its own lines.New and dropped normalized lines (371 new, 154 dropped).
Chain v2, full mode, on the rebased stack head
ebd31cdf4d(every unit's commits are ancestors of it; the chain ran once on the head, not per unit), log/tmp/rr-split/srebase-validation.log: stage 1, the 13 router crates (clippy under default and all features,cargo test1,247 passed / 5 ignored, rustdoc with all features), 1 min 50 s; stage 2, kv-router (clippy, bench check, 229 tests including the prelude test), 1 min 20 s; stage 3, workspace clippy, 4 min 58 s; stage 4, hosts (dynamo-llmlib alone, its kv_router 492 / discovery 275 / pool_relay 261 tests, 13 host crates clippy, bindings clippy; the pre-existing kvbm-consolidator manifest failure is the one non-zero exit), 29 min 21 s; stage 5, gates (every kv-router feature alone and together, 6 public-API snapshots unchanged with kv-router at 918 items, layering gate 47 checks and--host66 checks with 0 failures, codeowners 7590/7590, fmt, 3 lockfiles--locked), 3 min 06 s; 40 min 35 s total, exit 0.Each replaced fork PR carried its own validation chain; see its body.
Tickets
Part of DYNO-178