You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
{{ message }}
Repository navigation
ci: split feature-gated Rust builds into a parallel pre-merge job - #17958
Move the isolated kv-router checks, standalone slot-tracker tests, canonical replay tests, and feature-gated benchmark compiles out of rust-tests into a new rust-feature-builds job. Each feature set builds a separate artifact tree, so running them after the full workspace test build exhausted runner disk; running them on their own runner removes that pressure and takes ~10 min off the rust-tests critical path.
Add a setup-rust-ci composite action that installs protoc, restores the cargo cache, and adds rustup components, replacing the inline copies in pre-merge rust-tests, rust-feature-builds, rust-clippy, and nightly rust-tests. Add the action to the rust change filter so edits to it run the Rust jobs.
Summary by CodeRabbit
Chores
Updated automated Rust checks to use a consistent setup across nightly and pre-merge workflows.
Added a separate set of checks for Rust feature builds, including isolated builds, tests, and benchmark compilation.
Pre-merge status now accounts for the feature-build checks.
Move the isolated kv-router checks, standalone slot-tracker tests,
canonical replay tests, and feature-gated benchmark compiles out of
rust-tests into a new rust-feature-builds job. Each feature set builds a
separate artifact tree, so running them after the full workspace test
build exhausted runner disk; running them on their own runner removes
that pressure and takes ~10 min off the rust-tests critical path.
Add a setup-rust-ci composite action that installs protoc, restores the
cargo cache, and adds rustup components, replacing the inline copies in
pre-merge rust-tests, rust-feature-builds, rust-clippy, and nightly
rust-tests. Add the action to the rust change filter so edits to it run
the Rust jobs.
Signed-off-by: Anant Sharma <anants@nvidia.com>
The pull request adds a shared Rust CI setup action and uses it in nightly and pre-merge workflows. Pre-merge CI also adds a dedicated job for Rust feature checks and a Python AIS feature test.
The composite action installs and verifies protoc, caches Cargo data, and adds optional rustup components. Nightly tests, pre-merge tests, and Clippy use the action. The Rust file filter includes the action.
The pre-merge status job requires rust-feature-builds. That job runs the kv-router, slot-tracker, canonical replay, and benchmark checks. The rust-tests job adds the Python AIS feature test.
The Rust feature checks now run separately; no material merge-blocking risk is established.
Pre-merge checks: 5
Check name
Status
Explanation
Title check
The title names the main change: feature-gated Rust builds are split into a separate job that runs in parallel in pre-merge. The raw summary confirms the new rust-feature-builds job and the removal of…
Description check
The description covers the main parts of the template. It explains the key changes: the new rust-feature-builds job, the reason for the split (runner disk exhaustion and the roughly 10 minute reductio…
Docstring Coverage
No eligible functions found in the changed files. Skipping docstring coverage check. Docstring coverage is scoped to functions touched by this diff with cyclomatic complexity of at least 2. Documentat…
Linked Issues check
Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check
Check skipped because no linked issues were found for this pull request.
Full details: Title check
Explanation
The title names the main change: feature-gated Rust builds are split into a separate job that runs in parallel in pre-merge. The raw summary confirms the new rust-feature-builds job and the removal of those checks from rust-tests. The title does not mention the setup-rust-ci composite action, but that is a secondary change.
Full details: Description check
Explanation
The description covers the main parts of the template. It explains the key changes: the new rust-feature-builds job, the reason for the split (runner disk exhaustion and the roughly 10 minute reduction on the rust-tests critical path), and the new setup-rust-ci composite action with its filter change. It does not include the template's Overview, Validation, or Issues sections as headings, and it has no validation steps, no linked issue (the Closes #XXXX line is missing), and no Pull Request Checklist entry. These gaps are non-critical. The description is mostly complete.
Full details: Docstring Coverage
Explanation
No eligible functions found in the changed files. Skipping docstring coverage check. Docstring coverage is scoped to functions touched by this diff with cyclomatic complexity of at least 2. Documentation-only edits and deletions within surviving functions are included; unchanged diff context and fully deleted functions are excluded. Analyzed 0 functions across 0 files. (4 skipped: 4 unsupported.)
Autofix · Keep fixing CodeRabbit findings and required CI, and resolving merge conflicts
Tip
🎯 Customize your risk assessments
Guide how CodeRabbit assesses merge risk and security architecture risk. Highlight sensitive changes and explain your security boundaries using Merge Risk Instructions and Security Risk Instructions in your review settings. Security architecture risk assessments require the Advanced plan or above.
✅ Automatic approvals, with your rules
Let CodeRabbit approve pull requests once required threads are resolved, pre-merge checks pass, and your risk limits are met. Enable Automatic approvals in review settings, with overrides for teams or file paths.
CodeRabbit Verify
Run CodeRabbit Verify to test the changes in this pull request.
The reason will be displayed to describe this comment to others. Learn more.
Additional comments: 1
.github/workflows/pre-merge.yml-499-501: 1
499-501: Minor · Security & Privacy · Quick win
Unverified finding
Verification ran but could not confirm this finding. It is shown for review, not as a verified issue.
Disable persisted checkout credentials in the new build job.
When rust-feature-builds runs Cargo, dependency build scripts can use the checkout credential because persist-credentials remains enabled. The job needs only contents: read, but it does not need authenticated Git commands after checkout. Set persist-credentials: false. (github.com)
Based on learnings, jobs that run dependency build code should disable persisted checkout credentials.
Reviewing files that changed from the base of the PR and between 3aafbef and 96cb8d2.
Files selected for processing: 4
.github/actions/setup-rust-ci/action.yml
.github/filters.yaml
.github/workflows/nightly-ci.yml
.github/workflows/pre-merge.yml
Included review availability: This review used your included allowance. Your plan provides up to 12 included reviews per hour; 11 remain after this review.
Datadog CI (rust-tests (.), last 7 days: 728 success / 149 error / 296 canceled):
24 of 149 errors mention "space" (e.g. "No space left on device. Free space left: 62 MB").
Median success is ~45 min. This PR removes ~650s (benchmark compile 501s + feature tests ~145s) from it.
Questions:
Do the seven benchmark builds in rust-feature-builds share one target/ dir? If so, could it hit the same disk limit on Fastchecker (which doesn't seem to be in Velonix, so can't check disk limit)?
No jobs failed. 2 vLLM deploy tests were cancelled because they hit the job's 25-minute limit, not by fail-fast or a manual cancel. This made deploy-status-check fail.
⏹️ vllm Deploy Test / agg_router: cancelled in Run Dynamo Deploy Test after "The job has exceeded the maximum execution time of 25m0s". Just before that, all 3 connection tests to frontend pod vllm-agg-router-0-frontend-* failed with Read timed out. (read timeout=5).
⏹️ vllm Deploy Test / agg: same pattern. The job hit the 25-minute limit in Run Dynamo Deploy Test after all 3 connection tests to vllm-agg-0-frontend-* timed out.
For agents
{"pr": 17958, "run_id": 41688205406, "run_attempt": 1, "head_sha": "96cb8d2e72822da2272e48d80f96b9185aee907f",
"failures": [],
"cancelled": [{"job": "vllm Deploy Test / agg_router", "job_id": 126125835071, "step": "Run Dynamo Deploy Test", "signature": "job exceeded 25m timeout; frontend connection test read timed out", "log_cmd": "gh api repos/ai-dynamo/dynamo/actions/jobs/126125835071/logs"},
{"job": "vllm Deploy Test / agg", "job_id": 126125835170, "step": "Run Dynamo Deploy Test", "signature": "job exceeded 25m timeout; frontend connection test read timed out", "log_cmd": "gh api repos/ai-dynamo/dynamo/actions/jobs/126125835170/logs"}]}
Posted automatically by Devin for run 41688205406. Updated on every full-CI run of this PR.
Do the seven benchmark builds in rust-feature-builds share one target/ dir? If so, could it hit the same disk limit on Fastchecker (which doesn't seem to be in Velonix, so can't check disk limit)?
Yes. All the cargo invocations in rust-feature-builds run in one job with the default target/, so they share one target dir. But that dir is now much smaller than it used to be. Before, these builds stacked on top of the full workspace test build in rust-tests (.) (cargo test --no-run + docs + --all-targets), and that full build is what used most of the disk. In the new job, target/ holds only the dynamo-kv-router and dynamo-bench feature builds plus kv_router_bench.
Absorbs the 95 upstream commits since 464ff29. Each upstream change lands in the crate that owns that code at the stack head; the stack's moves stay intact. Router-touching PRs and where they landed:
- ai-dynamo#17712 host-path routing modes for worker-selection plugins: &self + Send + Sync plugin traits and WorkerInputs::ACTIVE_REQUESTS (deprecated OCCUPANCY alias) in dynamo-worker-selection and dynamo-router-protocol; HostedWorkerPicker and HostedSelectionRequest in dynamo-worker-selection (hosted.rs), reporting SelectionFailure, which the kernel converts into KvSchedulerError; the InstanceSelector adapter in lib/llm (HostedInstanceSelector); the compat test's hosted coverage in kv-router's prelude test.
- ai-dynamo#17713 random/round-robin/least-loaded as builtin plugins: router_modes in dynamo-router-policies (renamed from dynamo-custom-policy-builtin) importing the prelude; RouterPluginRegistry::resolve_type in kv-router; the frontend's hosted path through dynamo_worker_selection::host.
- ai-dynamo#17899 1.7.0 removals: taken whole; the alias and the setter were gone at the head; power-of-two never arrives.
- ai-dynamo#17657 single-source fetch continuations: KvTransferSourceInterval and source_intervals in dynamo-kv-indexer, the bucketed summary in dynamo-kv-hints, the interval filter in the kernel's resolver; tests in the kernel.
- ai-dynamo#17856 KVCR hints in offline replay: mocker and lib/llm bodies; supports_kv_transfer_chain_retention on the flat config; its indexer-vs-config test in kv-router's core host tests.
- ai-dynamo#17794 vLLM partial-tail filter and ai-dynamo#17126 SGLang engine-owned multimodal pads: dynamo-kv-events zmq_wire (mm_exclusion.rs); the codec-plus-indexer tests in dynamo-kv-indexer; the engine_owned_mm_identity worker flag through the kernel's WorkerRequest, kv-router's indexer registry and run config, and the bindings.
- ai-dynamo#17653 SGLang 0.5.23: pad constant docs in dynamo-multimodal.
- ai-dynamo#15333, ai-dynamo#17874, ai-dynamo#17484, ai-dynamo#17688, ai-dynamo#17237, ai-dynamo#17894, ai-dynamo#17958: host-side changes merged in place (bindings, lib/llm, CI).
Lockfiles regenerated; no shim reintroduced.
Signed-off-by: Ishan Dhanani <ishandhanani@gmail.com>
One conflict, in the `needs` list of `pre-merge-status-check` in
`.github/workflows/pre-merge.yml`: main added `rust-feature-builds`
(#17958) and this branch had added `rust-affinity-store-tests`, both
before `rust-clippy`. Both are kept here, in job order; the job blocks
themselves merged cleanly.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Signed-off-by: Sachal Malick <s@chal.ai>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Move the isolated kv-router checks, standalone slot-tracker tests, canonical replay tests, and feature-gated benchmark compiles out of rust-tests into a new rust-feature-builds job. Each feature set builds a separate artifact tree, so running them after the full workspace test build exhausted runner disk; running them on their own runner removes that pressure and takes ~10 min off the rust-tests critical path.
Add a setup-rust-ci composite action that installs protoc, restores the cargo cache, and adds rustup components, replacing the inline copies in pre-merge rust-tests, rust-feature-builds, rust-clippy, and nightly rust-tests. Add the action to the rust change filter so edits to it run the Rust jobs.
Summary by CodeRabbit