Repository navigation
refactor(router): narrow worker policy API - #13008
Merged
ishandhanani merged 16 commits intoAug 11, 2026
Merged
Conversation
Contributor
ishandhanani
commented
Aug 11, 2026
This comment has been minimized.
This comment has been minimized.
ishandhanani
marked this pull request as ready for review
August 11, 2026 07:09
Contributor
WalkthroughThe PR narrows public selector internals and expands documentation for custom scorer and picker implementation, registration, configuration, validation, Python frontend integration, and EPP execution. ChangesCustom worker-selection policies
Estimated code review effort: 3 (Moderate) | ~25 minutes 🚥 Pre-merge checks | ✅ 3 | ❌ 2❌ Failed checks (2 warnings)
✅ Passed checks (3 passed)
Comment |
This was referenced Aug 11, 2026
ishandhanani
force-pushed
the
idhanani/dyn-3712-worker-policy-api-hardening
branch
from
August 11, 2026 18:24
d8c7d06 to
76b0bd1
Compare
Signed-off-by: Ishan Dhanani <ishandhanani@gmail.com>
Signed-off-by: Ishan Dhanani <ishandhanani@gmail.com>
Signed-off-by: Ishan Dhanani <ishandhanani@gmail.com>
Signed-off-by: Ishan Dhanani <ishandhanani@gmail.com>
Signed-off-by: Ishan Dhanani <ishandhanani@gmail.com>
Signed-off-by: Ishan Dhanani <ishandhanani@gmail.com>
Signed-off-by: Ishan Dhanani <ishandhanani@gmail.com>
Signed-off-by: Ishan Dhanani <ishandhanani@gmail.com>
Signed-off-by: Ishan Dhanani <ishandhanani@gmail.com>
Signed-off-by: Ishan Dhanani <ishandhanani@gmail.com>
Signed-off-by: Ishan Dhanani <ishandhanani@gmail.com>
Signed-off-by: Ishan Dhanani <ishandhanani@gmail.com>
Signed-off-by: Ishan Dhanani <ishandhanani@gmail.com>
Signed-off-by: Ishan Dhanani <ishandhanani@gmail.com>
Signed-off-by: Ishan Dhanani <ishandhanani@gmail.com>
Signed-off-by: Ishan Dhanani <ishandhanani@gmail.com>
ishandhanani
force-pushed
the
idhanani/dyn-3712-worker-policy-api-hardening
branch
from
August 11, 2026 20:47
889e730 to
da253b8
Compare
nv-tusharma
approved these changes
Aug 11, 2026
8 tasks
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
This change narrows the Rust contract for custom worker scorers and pickers and adds strict rules for future API additions. The smaller contract lets Dynamo change its built-in routing formula without preserving intermediate values or default-policy components.
CLOSES: DYN-3712
How This Was Implemented
DefaultWorkerScorer,DefaultWorkerPicker,WorkerInputs::ALL, and bitset inspection internal to the selector.SessionContextwith the complete current session metadata. The internalAgentContextadapter is exhaustive, so a new protocol field requires an explicit API decision.Walkthrough
Mental model
A custom policy owns scoring and picking. Dynamo continues to own worker eligibility, score validation, accounting, and reservation.
flowchart LR A["Request"] --> B["Host eligibility"] B --> C["Custom scorer(s)"] C --> D["Host sums finite costs"] D --> E["Custom picker"] E --> F["Host validates the row"] F --> G["Accounting and reservation"]State and ordering
Each scorer requests only the
CACHE,LOAD, orROUTINGgroups that it reads. Dynamo calls every scorer for each eligible worker and adds their finite, lower-is-better costs. The picker receives the scored rows and the columns that it explicitly requested, then returns one row index.The built-in scorer and picker remain private implementation details. An external policy supplies both components through
WorkerSelectionPolicy::new.Request lifecycle
The host constructs request context and candidate rows after eligibility filtering. The policy cannot restore an ineligible worker. The host rejects non-finite scores and invalid picker rows before accounting or reservation.
Boundaries and limitations
This change does not alter the default routing calculation or the low-level
WorkerSelectorpath used across Dynamo crates.Validation
cargo fmt --all -- --checkgit diff --checkCARGO_TARGET_DIR=/ephemeral/dynamo/target cargo test -p dynamo-kv-router --features standalone-selection scheduling::selector(27 passed)CARGO_TARGET_DIR=/ephemeral/dynamo/target cargo test -p dynamo-mocker session_identity_reaches_scheduling_request --libCARGO_TARGET_DIR=/ephemeral/dynamo/target cargo check -p dynamo-llm --libCARGO_TARGET_DIR=/ephemeral/dynamo/target cargo check -p dynamo-llm --testsCARGO_TARGET_DIR=/ephemeral/dynamo/target cargo test -p dynamo-custom-policy-example-basic -p dynamo-custom-policy-example-disaggregated -p dynamo-custom-policy-example-catalogCARGO_TARGET_DIR=/ephemeral/dynamo/target cargo build -p dynamo-custom-policy-example-epppre-commit run --files <changed files>cd docs && fern check --warningscd docs && fern docs broken-linksSummary by CodeRabbit
Documentation
Refactor