Skip to content

refactor(router): give worker selection its own config slices - #124

Closed
ishandhanani wants to merge 1 commit into
idhanani/dyn-177-selection-context-fieldsfrom
idhanani/dyn-177-selection-config-slice
Closed

ishandhanani wants to merge 1 commit into
idhanani/dyn-177-selection-context-fieldsfrom
idhanani/dyn-177-selection-config-slice

Conversation

@ishandhanani

Copy link
Copy Markdown
Owner

Step 5d-1 (DYNO-177), stacked on #123. Second boundary change for dynamo-worker-selection; behavior-preserving, not a move. Plan: split-v2/inventory-worker-selection.md.

Summary

Worker selection read the flat KvRouterConfig in four places: the policy factory type, WorkerSelectionPolicy::new*, the reference selector's weights, and ConditionalDisaggPolicy. Two slices now own those fields, and the router converts once at the boundary.

  • SelectionConfig (plugins::worker_selection::selection_config, re-exported at config:: and the root): overlap_score_credit, overlap_score_credit_decay, prefill_load_scale, decode_active_request_weight, host_cache_hit_weight, disk_cache_hit_weight, router_temperature, shared_cache_multiplier, shared_cache_type, conditional_disagg_enabled. Default equals the flat defaults (the default_* helpers moved with it); From<&KvRouterConfig> lives beside the flat type; validate() uses the validators that moved with it (validate_overlap_score_credit, validate_finite_non_negative, validate_range, the range-error message). SharedCacheType moves into the slice module. This is the slice #106 built and the builtin plugin already reads (PolicyParameters::from).
  • WorkerSelectionPolicyFactory is Fn(&SelectionConfig, WorkerType, RoutingPartitionRef) -> WorkerSelectionPolicy. WorkerSelectionPolicy::{new, new_with_filters, reference} and the reference DefaultWorkerSelector take SelectionConfig. Call syntax is unchanged where the config came from the factory; the registry tests, the selection core (core/workers.rs), lib/llm's PreparedSelectionPolicy, the mocker, and the test helpers convert with SelectionConfig::from(&config).
  • ConditionalDisaggConfig (conditional_disagg, re-exported at config:: and the root): enabled, policy, eff_isl_threshold, eff_isl_ratio_threshold, prefill_busy_threshold, decode_busy_threshold, with Default and From<&KvRouterConfig>. ConditionalDisaggPolicyKind moves beside it. make_conditional_disagg_policy, policy_needs_prefill_worker_busy, and the policies' from_config take the slice; lib/llm's prefill leg converts once.
  • Builtin plugin: default_policy, startup_policy_parameters, PolicyParameters, DefaultWorkerSelector, the two-tier and ThunderAgent factories take SelectionConfig. The four example policies switch to SelectionConfig (they only cloned the config through).

Every old path still resolves (config::{SharedCacheType, ConditionalDisaggPolicyKind}, the root re-exports). Behavior is unchanged: the same values flow to the same reads.

Validation

  • cargo check --all-targets on kv-router, builtin, mocker, ext-proc; cargo check -p dynamo-llm --tests: clean.
  • cargo test -p dynamo-kv-router --features standalone-selection,standalone-slot-tracker,metrics,testing: 590 lib + 2 + 7 passed, 2 ignored.
  • cargo test -p dynamo-custom-policy-builtin: 56 + 5 + 1 + 3 + 20 + 3 + 4 passed, 2 ignored.
  • cargo test on the six example crates: all pass (2 + 1 + 1 + 3 tests; two crates have none).
  • cargo test -p dynamo-llm --lib kv_router::: 485 passed; the three entrypoint::input::http tests pass.
  • clippy -D warnings: kv-router + builtin (--all-features --all-targets), mocker + ext-proc (--tests), lib/llm (--tests, default features): clean.
  • scripts/router_layering_check.sh: 20 checks, 0 failures (no dependency change).

Worker-selection code read the flat KvRouterConfig: the policy factory type
took it, WorkerSelectionPolicy::new carried it, the reference selector
scored with its weights, and ConditionalDisaggPolicy read six of its
fields. The stage must not depend on the router's flat configuration, so
two slices now own those fields.

SelectionConfig (plugins::worker_selection::selection_config) holds the
scoring weights, the sampling temperature, the shared-cache multiplier and
type, and conditional_disagg_enabled, with Default equal to the flat
defaults, From<&KvRouterConfig> next to the flat type, and validate() using
the validators that moved with it. SharedCacheType moves into the slice
module. WorkerSelectionPolicyFactory takes &SelectionConfig;
WorkerSelectionPolicy::new, new_with_filters, and reference, and the
reference DefaultWorkerSelector, take SelectionConfig. The registry, the
selection core, lib/llm's prepared-policy wrapper, the mocker, and the
in-tree plugins convert once at the boundary.

ConditionalDisaggConfig (conditional_disagg) holds the six
conditional_disagg_* fields with Default and From<&KvRouterConfig>;
ConditionalDisaggPolicyKind moves beside it; make_conditional_disagg_policy
and the policies' from_config take the slice, and lib/llm's prefill leg
converts once. config:: re-exports every moved name and the root exports
the two slices, so existing paths resolve.

The builtin plugin's default policy, its parameters, and its selector take
SelectionConfig; the four example policies and the builtin, kv-router, and
lib/llm tests convert where they built a flat config for selection.

Step 5d-1 of the router split v2 plan (worker-selection boundary). DYNO-177.

Signed-off-by: Ishan Dhanani <ishandhanani@gmail.com>
@ishandhanani

Copy link
Copy Markdown
Owner Author

Landed upstream as ai-dynamo#18055 (squash commit 79692cf on main, 2026-11-11). Closing this fork review unit.

This branch had an error being deployed

1 failed deployment
external_collaborator — b9054333 Deployed Nov 2, 2026 by ishandhanani via ok-to-test #469
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant