Repository navigation
refactor(router): point ext-proc and the Python bindings at the owning crates - #150
Closed
ishandhanani wants to merge 1 commit into
Closed
ishandhanani wants to merge 1 commit into
ishandhanani wants to merge 1 commit into
Conversation
ishandhanani
had a problem deploying
to
external_collaborator
October 31, 2026 11:36 — with
GitHub Actions
Failure
…g crates Step 8c of the router split (DYNO-181). The EPP ext-proc and the Python bindings reached stage and kernel types through dynamo-kv-router's shim modules (protocols, scheduling, zmq_wire, the crate root, and the selection service's re-exports). They now import the kernel through dynamo_scheduler::host, each stage through its host module (kv-indexer, worker-selection, admission, session-affinity), and the vocabulary crates (dynamo-router-protocol, dynamo-kv-events, dynamo-kv-hashing) directly; the composition items stay on kv-router: the flat config and its env readers, the plugin registry and RouterPlugins, SelectionService and its builder, and the three service entries the binaries run. Features follow: ext-proc enables the three adapters the selection service builder composes (ingest-zmq, replica-sync-zmq, peer-recovery) plus metrics, and drops runtime-protocols, which it never used; the bindings' kv-indexer, slot-tracker, and select-service features name the adapters each binary runs, and custom-policy enables no kv-router feature (the catalog needs none since 8a). The bindings manifest, which lives outside the workspace, gains path dependencies on the owner crates. The host lists grow by what these two hosts needed: KvTransferEnforcement on the kernel, ApproximateCachePolicyKind on kv-indexer, ConditionalDisaggPolicyKind on worker-selection. The bindings' `protocols::*` glob had hidden three things: a name it never used (WorkerWithDpRank), one used only in a comment (PotentialLoad), and two the file shadows with its own pyclasses (OverlapScores, RoutingConstraints), which the explicit imports now leave out; the two Rust-side OverlapScores uses are fully qualified. The layering gate's --host mode, which CI runs, failed on every host before this change for two reasons that have nothing to do with hosts: dynamo-zmq is the ZMQ socket helper and reaches zmq by definition, and the kv-indexer `export` feature (kv-router `metrics`) reaches prometheus by design until step 9c moves exposition to dynamo-metrics. The gate now exempts the declared adapter crate for its own dependency (HOST_ADAPTER_EXEMPT) and lists prometheus as a 9c return; with that, --host passes on lib/llm and ext-proc. Validation: ext-proc clippy -D warnings --all-targets and tests (167 + 1); the EPP example checks; the Python bindings check and clippy -D warnings under default features, no default features, and kv-indexer,slot-tracker,select-service; kv-router check under the full adapter set; the three host crates check; kv-router public-api snapshot unchanged; router layering gate --host 42 checks, 0 failures (it failed on every host before: see above); codeowners strict; fmt on the workspace and the bindings. cargo tree -p dynamo-ext-proc -e normal names kv-router, protocol, scheduler, session-affinity, kv-events, and the runtime; zmq and reqwest remain through the adapters it enables. Signed-off-by: Ishan Dhanani <ishandhanani@gmail.com>
ishandhanani
force-pushed
the
idhanani/dyn-181-ext-proc-bindings
branch
from
November 2, 2026 13:31
676aab8 to
413162b
Compare
ishandhanani
had a problem deploying
to
external_collaborator
November 2, 2026 13:31 — with
GitHub Actions
Failure
Owner
Author
|
Landed upstream as ai-dynamo#18055 (squash commit 79692cf on main, 2026-11-11). Closing this fork review unit. |
This branch had an error being deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Step 8c of the router split v2 (DYNO-181). Stacked on #149. Two hosts move off kv-router's shims: the EPP ext-proc and the Python bindings.
Summary
deploy/inference-gateway/ext-proc):WorkerType,DEFAULT_ROUTING_GROUP,WorkerWithDpRankfromdynamo_router_protocol;SelectionError,RoutingConstraints,RouterConfigOverride,PromptRequest,SelectAndReserveRequest,CatalogReconciler,WorkerCatalogSource,WorkerRequestfromdynamo_scheduler::host;MAX_SESSION_AFFINITY_ID_BYTESfromdynamo_session_affinity::host;DYNAMO_CACHE_SALT_PREFIXfromdynamo_kv_events::zmq_wire; the registry types fromdynamo_kv_router::plugins(notservices::selection); the test policy throughplugins::prelude.SelectionService,SelectionServiceBuilder,warn_for_unserved_worker_selection_policies, and the config readers stay on kv-router. Manifest:metrics, ingest-zmq, replica-sync-zmq, peer-recovery(the builder composes those adapters;runtime-protocolswas unused) plusdynamo-kv-events,dynamo-router-protocol,dynamo-scheduler,dynamo-session-affinity.lib/bindings/python): theprotocols::*glob becomes explicit imports from the owners (events fromdynamo_kv_events::event, hashes fromdynamo_kv_hashing::routing,OverlapScores/RadixTree/ApproximateCachePolicyKindfromdynamo_kv_indexer::host, worker ids andBlockExtraInfo/RouterHintSourceMetadatafrom the protocol crate,PotentialLoad/RoutingConstraints/RouterConfigOverride/PrefillLoadEstimator/TrackingHashAlgorithm/RouterPrefillLoadModel/KvTransferEnforcementand the selection DTOs fromdynamo_scheduler::host,QueueRejectionfromdynamo_admission::host,SelectionConfig/ConditionalDisaggPolicyKindfromdynamo_worker_selection::host);KvRouterConfig, the env readers,apply_deprecated_overlap_score_weight_override,RouterPlugins/RouterPluginRegistry,SelectionService/Builder/Config, and theservices::{indexer, slot_tracker}entries stay on kv-router. Manifest:kv-indexer = http + ingest-zmq + peer-recovery,slot-tracker = http + replica-sync-zmq,select-service = all four adapters,custom-policywithout a kv-router feature; path dependencies on the seven owner crates.KvTransferEnforcement; kv-indexerApproximateCachePolicyKind; worker-selectionConditionalDisaggPolicyKind.protocols::*glob had hidden a never-used name (WorkerWithDpRank), one used only in a comment (PotentialLoad), and two nameskv.rsshadows with its own pyclasses (OverlapScores,RoutingConstraints); the explicit imports leave those out and the two Rust-sideOverlapScoresuses are fully qualified.--hostmode (the mode CI runs atpre-merge.yml:605) failed on lib/llm and ext-proc before this change, for reasons unrelated to any host:dynamo-zmqis the ZMQ adapter and reacheszmqby definition; kv-indexer'sexportfeature reachesprometheusby design until 9c.scripts/router_layering_check.shgainsHOST_ADAPTER_EXEMPT="dynamo-zmq:zmq"and dropsprometheusfromHOST_FORBIDDENwith a 9c note;--hostnow passes (45 checks).Validation
cargo clippy -D warnings --all-targetsclean;cargo test -p dynamo-ext-proc167 + 1;cargo check -p dynamo-custom-policy-example-epp(depends on ext-proc) clean.cargo tree -p dynamo-ext-proc -e normalnamesdynamo-kv-router,dynamo-router-protocol,dynamo-scheduler,dynamo-session-affinity,dynamo-kv-events,dynamo-runtime;zmq,dynamo-zmq,reqweststay through the adapters it enables for the selection service builder.lib/bindings/python):cargo checkandcargo clippy -- -D warningsunder default features,--no-default-features, andkv-indexer,slot-tracker,select-service: clean;cargo fmt -- --checkclean.cargo check -p dynamo-scheduler -p dynamo-kv-indexer -p dynamo-worker-selection(host lists grew) andcargo check -p dynamo-kv-routerunderhttp,ingest-zmq,replica-sync-zmq,peer-recovery,metrics: clean.scripts/router_public_api_check.sh: snapshot unchanged (1392 items).scripts/router_layering_check.sh --host: 42 checks, 0 failures (45 with 4 failures on the base, for the two pre-existing reasons above).build_codeowners.py --strict100%;cargo fmt --all -- --checkclean.