Repository navigation
refactor(router): move the worker-selection stage to lib/router/worker-selection - #128
Closed
ishandhanani wants to merge 1 commit into
Closed
ishandhanani wants to merge 1 commit into
ishandhanani wants to merge 1 commit into
Conversation
ishandhanani
had a problem deploying
to
external_collaborator
October 31, 2026 00:51 — with
GitHub Actions
Failure
…r-selection
Step 5d-4 of the router split (DYNO-177): dynamo-worker-selection, a
stage crate that depends on dynamo-router-protocol and nothing router-
side beyond it. A move: the files that made up the stage inside
lib/kv-router move as they are, and lib/kv-router re-exports every name
at the path plugins and hosts already use.
Moved: plugins/worker_selection/{context, inputs, runtime,
selection_config, error, trace, shared_cache, reference_picker}.rs, the
plugin traits from the plugins/worker_selection.rs facade,
scheduling/selector/policy.rs (the composed policy, CandidateRowSource,
RankVisitor, CandidateRank, CandidatePreparer, PreparedRows, the trace
builder), kv_hints/sources.rs as kv_transfer.rs, conditional_disagg.rs,
and scheduling/lora_filter.rs.
Stays in lib/kv-router: the WorkerSelector trait and its adapter, the
request-to-row source, the derived columns (selector/derive.rs), the
reference selector, the policy factory type, the registry, the host
instance config (plugins/worker_selection/config.rs), and the resolver
for kv-transfer sources. The policy tests that drive WorkerSelector with
a SchedulingRequest stay as selector/policy_tests.rs; the kv-transfer
summary tests that resolve against a routing snapshot stay in
kv_hints.rs; the stage keeps its own row-source, trace, picker, scope,
and conditional-disagg tests.
Visibility the crate boundary required: the selection config slice's
defaults and validators, which the flat router config shares, are pub,
as are the conditional-disagg defaults, WorkerInputs::ALL, and
WorkerSelectionPolicy::reference_parts; the reference selector reads the
context through its accessors. DEFAULT_MAX_BATCHED_TOKENS is taken from
the protocol crate directly.
Layering gate: a row for dynamo-worker-selection (22 checks). CODEOWNERS
gains lib/router/worker-selection/ under the router area.
Validation: stage 41 tests; kv-router 554 + 2 + 7 (554 + 41
moved = the 595 before); builtin plugin and lib/llm kv_router 485 tests;
clippy clean on the stage crates and lib/llm, mocker, ext-proc, bench;
layering gate 22 checks; move-diff 13748 common / 102 new / 35 dropped (the new lines are #[inline] hints and visibility).
Interleaved A/B bench against the 5d-3b head at 10k workers: parity or
better once the stage's small accessors carry #[inline] (non-generic
functions the kernel calls per rank across the crate boundary; without
the hints the reference path measured +2%).
Signed-off-by: Ishan Dhanani <ishandhanani@gmail.com>
ishandhanani
force-pushed
the
idhanani/dyn-177-worker-selection
branch
from
November 2, 2026 13:30
280a785 to
7598a00
Compare
ishandhanani
had a problem deploying
to
external_collaborator
November 2, 2026 13:30 — with
GitHub Actions
Failure
ishandhanani
had a problem deploying
to
external_collaborator
November 2, 2026 13:53 — with
GitHub Actions
Failure
Owner
Author
|
Landed upstream as ai-dynamo#18055 (squash commit 79692cf on main, 2026-11-11). Closing this fork review unit. |
This branch had an error being deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Step 5d-4 of the router split v2 (DYNO-177). Stacked on #127. The crate move:
dynamo-worker-selectionatlib/router/worker-selection, depending ondynamo-router-protocoland nothing router-side beyond it.Summary
plugins/worker_selection/{context, inputs, runtime, selection_config, error, trace, shared_cache, reference_picker}.rs, theWorkerFilter/WorkerScorer/WorkerPickertraits from the facade,scheduling/selector/policy.rs(the composed policy,CandidateRowSource,RankVisitor,CandidateRank,CandidatePreparer,PreparedRows, the trace builder),kv_hints/sources.rs(askv_transfer.rs),conditional_disagg.rs, andscheduling/lora_filter.rsmove as they are.lib/kv-routerre-exports every name at the paths plugins and hosts already use:plugins::worker_selection::*,scheduling::selector::*,kv_hints::*,conditional_disagg::*,scheduling::lora_filter,protocols::{SelectionTrace*, SharedCacheHits},scheduling::types::WorkerSelectionPolicyError. No downstream crate changes.lib/kv-router. TheWorkerSelectortrait and its adapter overselect_rows, the request-to-rowSelectionRowSource, the derived columns (selector/derive.rs), the referenceDefaultWorkerSelector,WorkerSelectionPolicyFactory(namesRoutingPartitionRef), the registry, the host instance configplugins/worker_selection/config.rs(registry input; a deviation from the inventory's §2, recorded in §4.2), andkv_hints::resolve_transfer_sourceswithWorkerKvTransferKeysExt(they read worker configs and the indexer's routing snapshot).SchedulingRequestand callWorkerSelectorstay asselector/policy_tests.rs; the kv-transfer summary tests that resolve candidates against a residency snapshot stay inkv_hints.rs(summary_tests); the stage keepsrow_source_tests, the trace tests, the picker tests, theKvTransferScopecoverage test (it reads a private method), and the conditional-disagg and LoRA tests.KvRouterConfigshares, arepuband theselection_configmodule is public; the conditional-disagg defaults likewise;WorkerInputs::ALL(the reference row);WorkerSelectionPolicy::reference_parts(gatedtesting/bench). The reference selector reads the context through its accessors instead of fields.DEFAULT_MAX_BATCHED_TOKENSis taken fromdynamo_router_protocol::loaddirectly (itsscheduling::queuere-export for lib/llm is unchanged).testingandbencheach add the reference picker (parking_lot); kv-router'stesting/benchforward to them and its dev-dependency enablestesting.fastrandis a plain dependency (the random-bypass conditional-disagg policy draws from it).scripts/router_layering_check.shgains thedynamo-worker-selectionrow (22 checks, 0 failures: router deps equal{dynamo-router-protocol}, no sockets, HTTP, Prometheus, or other router crates under--all-features)..github/codeowners/areas.yamlclaimslib/router/worker-selection/under the router area andCODEOWNERSis regenerated.lib/router/README.mdstatus updated.Validation
cargo test -p dynamo-worker-selection --all-features: 41 passed (row-source, trace, picker, scope, conditional-disagg, LoRA, runtime, config).cargo test -p dynamo-kv-router --features standalone-selection,standalone-slot-tracker,metrics,testing: 554 + 2 + 7 passed (554 + 41 moved = the 595 before the move).cargo test -p dynamo-custom-policy-builtin --all-features: 56 + 5 + 1 + 3 + 20 + 3 + 4 passed.cargo test -p dynamo-llm kv_router: 485 passed.cargo clippy --all-features --all-targets -D warningson protocol, worker-selection, kv-router, builtin: clean;cargo clippy --tests -D warningson lib/llm (default features), mocker, ext-proc, bench: clean;cargo check --testson lib/llm, mocker, bench, sglang-sidecar, ext-proc: clean.scripts/router_layering_check.sh: 22 checks, 0 failures.scripts/router_move_diff.py 5deeea6e10 WORKTREEfrom the kv-router paths to the new crate plus the kv-router remainder: 13748 common / 102 new / 35 dropped. The new lines are the#[inline]hints (75) and the visibility promotions above; the dropped lines are the six accessor calls inreference.rs, the protocol constant path,mod testsrenamedmod summary_tests, and rustfmt re-wrapping four lines of the dedented test module.Bench: interleaved same-session A/B on an idle pinned core, the 5d-3b head's bench binary against this branch's, two rounds (the leftover SGLang scheduler was on cores 3 and 11; an earlier round pinned to core 3 ran both binaries 1.7× slower and was discarded):
default_worker_selection/direct/temperature_0/10000(reference)custom_worker_selection/10_000/custom_no_filtercustom_worker_selection/10_000/custom_keep_all_filterParity or better. Before the
#[inline]hints on the stage's small accessors (WorkerInputs::contains, the context getters, the row and view getters,WorkerCandidate::new,with_inputs_from) the reference path measured +2% in two interleaved rounds: those are non-generic functions the kernel now calls across a crate boundary per rank, and thin LTO alone did not inline all of them.