Skip to content

refactor(router): point ext-proc and the Python bindings at the owning crates - #150

Closed
ishandhanani wants to merge 1 commit into
idhanani/dyn-181-stage-host-modulesfrom
idhanani/dyn-181-ext-proc-bindings
Closed

ishandhanani wants to merge 1 commit into
idhanani/dyn-181-stage-host-modulesfrom
idhanani/dyn-181-ext-proc-bindings

Conversation

@ishandhanani

Copy link
Copy Markdown
Owner

Step 8c of the router split v2 (DYNO-181). Stacked on #149. Two hosts move off kv-router's shims: the EPP ext-proc and the Python bindings.

Summary

  • ext-proc (deploy/inference-gateway/ext-proc): WorkerType, DEFAULT_ROUTING_GROUP, WorkerWithDpRank from dynamo_router_protocol; SelectionError, RoutingConstraints, RouterConfigOverride, PromptRequest, SelectAndReserveRequest, CatalogReconciler, WorkerCatalogSource, WorkerRequest from dynamo_scheduler::host; MAX_SESSION_AFFINITY_ID_BYTES from dynamo_session_affinity::host; DYNAMO_CACHE_SALT_PREFIX from dynamo_kv_events::zmq_wire; the registry types from dynamo_kv_router::plugins (not services::selection); the test policy through plugins::prelude. SelectionService, SelectionServiceBuilder, warn_for_unserved_worker_selection_policies, and the config readers stay on kv-router. Manifest: metrics, ingest-zmq, replica-sync-zmq, peer-recovery (the builder composes those adapters; runtime-protocols was unused) plus dynamo-kv-events, dynamo-router-protocol, dynamo-scheduler, dynamo-session-affinity.
  • Python bindings (lib/bindings/python): the protocols::* glob becomes explicit imports from the owners (events from dynamo_kv_events::event, hashes from dynamo_kv_hashing::routing, OverlapScores/RadixTree/ApproximateCachePolicyKind from dynamo_kv_indexer::host, worker ids and BlockExtraInfo/RouterHintSourceMetadata from the protocol crate, PotentialLoad/RoutingConstraints/RouterConfigOverride/PrefillLoadEstimator/TrackingHashAlgorithm/RouterPrefillLoadModel/KvTransferEnforcement and the selection DTOs from dynamo_scheduler::host, QueueRejection from dynamo_admission::host, SelectionConfig/ConditionalDisaggPolicyKind from dynamo_worker_selection::host); KvRouterConfig, the env readers, apply_deprecated_overlap_score_weight_override, RouterPlugins/RouterPluginRegistry, SelectionService/Builder/Config, and the services::{indexer, slot_tracker} entries stay on kv-router. Manifest: kv-indexer = http + ingest-zmq + peer-recovery, slot-tracker = http + replica-sync-zmq, select-service = all four adapters, custom-policy without a kv-router feature; path dependencies on the seven owner crates.
  • Host lists grow: kernel KvTransferEnforcement; kv-indexer ApproximateCachePolicyKind; worker-selection ConditionalDisaggPolicyKind.
  • Glob fallout in the bindings: the protocols::* glob had hidden a never-used name (WorkerWithDpRank), one used only in a comment (PotentialLoad), and two names kv.rs shadows with its own pyclasses (OverlapScores, RoutingConstraints); the explicit imports leave those out and the two Rust-side OverlapScores uses are fully qualified.
  • Gate --host mode (the mode CI runs at pre-merge.yml:605) failed on lib/llm and ext-proc before this change, for reasons unrelated to any host: dynamo-zmq is the ZMQ adapter and reaches zmq by definition; kv-indexer's export feature reaches prometheus by design until 9c. scripts/router_layering_check.sh gains HOST_ADAPTER_EXEMPT="dynamo-zmq:zmq" and drops prometheus from HOST_FORBIDDEN with a 9c note; --host now passes (45 checks).

Validation

  • ext-proc: cargo clippy -D warnings --all-targets clean; cargo test -p dynamo-ext-proc 167 + 1; cargo check -p dynamo-custom-policy-example-epp (depends on ext-proc) clean. cargo tree -p dynamo-ext-proc -e normal names dynamo-kv-router, dynamo-router-protocol, dynamo-scheduler, dynamo-session-affinity, dynamo-kv-events, dynamo-runtime; zmq, dynamo-zmq, reqwest stay through the adapters it enables for the selection service builder.
  • Python bindings (lib/bindings/python): cargo check and cargo clippy -- -D warnings under default features, --no-default-features, and kv-indexer,slot-tracker,select-service: clean; cargo fmt -- --check clean.
  • cargo check -p dynamo-scheduler -p dynamo-kv-indexer -p dynamo-worker-selection (host lists grew) and cargo check -p dynamo-kv-router under http,ingest-zmq,replica-sync-zmq,peer-recovery,metrics: clean. scripts/router_public_api_check.sh: snapshot unchanged (1392 items).
  • scripts/router_layering_check.sh --host: 42 checks, 0 failures (45 with 4 failures on the base, for the two pre-existing reasons above). build_codeowners.py --strict 100%; cargo fmt --all -- --check clean.

…g crates

Step 8c of the router split (DYNO-181). The EPP ext-proc and the Python
bindings reached stage and kernel types through dynamo-kv-router's shim
modules (protocols, scheduling, zmq_wire, the crate root, and the
selection service's re-exports). They now import the kernel through
dynamo_scheduler::host, each stage through its host module
(kv-indexer, worker-selection, admission, session-affinity), and the
vocabulary crates (dynamo-router-protocol, dynamo-kv-events,
dynamo-kv-hashing) directly; the composition items stay on kv-router:
the flat config and its env readers, the plugin registry and
RouterPlugins, SelectionService and its builder, and the three service
entries the binaries run.

Features follow: ext-proc enables the three adapters the selection
service builder composes (ingest-zmq, replica-sync-zmq, peer-recovery)
plus metrics, and drops runtime-protocols, which it never used; the
bindings' kv-indexer, slot-tracker, and select-service features name
the adapters each binary runs, and custom-policy enables no kv-router
feature (the catalog needs none since 8a). The bindings manifest, which
lives outside the workspace, gains path dependencies on the owner
crates. The host lists grow by what these two hosts needed:
KvTransferEnforcement on the kernel, ApproximateCachePolicyKind on
kv-indexer, ConditionalDisaggPolicyKind on worker-selection.

The bindings' `protocols::*` glob had hidden three things: a name it
never used (WorkerWithDpRank), one used only in a comment
(PotentialLoad), and two the file shadows with its own pyclasses
(OverlapScores, RoutingConstraints), which the explicit imports now
leave out; the two Rust-side OverlapScores uses are fully qualified.

The layering gate's --host mode, which CI runs, failed on every host
before this change for two reasons that have nothing to do with hosts:
dynamo-zmq is the ZMQ socket helper and reaches zmq by definition, and
the kv-indexer `export` feature (kv-router `metrics`) reaches prometheus
by design until step 9c moves exposition to dynamo-metrics. The gate now
exempts the declared adapter crate for its own dependency
(HOST_ADAPTER_EXEMPT) and lists prometheus as a 9c return; with that,
--host passes on lib/llm and ext-proc.

Validation: ext-proc clippy -D warnings --all-targets and tests (167 +
1); the EPP example checks; the Python bindings check and clippy -D
warnings under default features, no default features, and
kv-indexer,slot-tracker,select-service; kv-router check under the full
adapter set; the three host crates check; kv-router public-api snapshot
unchanged; router layering gate --host 42 checks, 0 failures (it failed
on every host before: see above); codeowners strict; fmt on the
workspace and the bindings. cargo tree -p dynamo-ext-proc -e normal
names kv-router, protocol, scheduler, session-affinity, kv-events, and
the runtime; zmq and reqwest remain through the adapters it enables.

Signed-off-by: Ishan Dhanani <ishandhanani@gmail.com>
@ishandhanani

Copy link
Copy Markdown
Owner Author

Landed upstream as ai-dynamo#18055 (squash commit 79692cf on main, 2026-11-11). Closing this fork review unit.

This branch had an error being deployed

1 failed deployment
external_collaborator — 413162b3 Deployed Nov 2, 2026 by ishandhanani via ok-to-test #495
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant