[Ronald1995]

Scope

This PR aligns the Python and Rust sleep API contract and adds the operation observability needed for long sleep/wake operations.

• POST /sleep: {"status":"sleeping","level":0|1|2}.
• POST /release_kv_cache_memory: {"status":"kv_cache_released"}.
• POST /wake_up: {"status":"awake"|"sleeping","tags":null|[...]}. tags echoes the requested tags; null means wake all. Status uses the engine's fully_awake result, without a second state query.
• Both frontends validate sleep level 0..2 and mode before dispatch. Invalid query values return HTTP 400 without entering the operation recorder.
• Python DP wake now requires every relevant engine to report fully awake, matching Rust. Other utility RPCs retain their existing behavior.
• EngineClient.wake_up() now returns bool (fully awake); LLM.wake_up() propagates it. Interface change: out-of-tree EngineClient implementations must return bool, since a returned None would read as "sleeping".
• With DP load balancing, wake_up now reports fully awake only if every engine is. It reuses call_utility_all_async from [Core] Combine per-engine utility results like the Rust client #59240.

Metrics and review response

Request counts and HTTP status use the existing http_requests_total{handler,method,status}. The redundant sleep operation counter is removed. Rust generic HTTP middleware now includes /sleep, /release_kv_cache_memory, /wake_up, and /is_sleeping, matching Python.

Two sleep-specific metrics remain, with no rl_ prefix:

• vllm:sleep_mode_operation_duration_seconds{operation}: buckets suitable for long operations; operations are sleep, release_kv_cache_memory, and wake.
• vllm:sleep_mode_operations_in_flight{operation}: detects outstanding sleep/wake work; cleaned up on success, error, and cancellation.

No global HTTP in-progress middleware redesign or released/restored memory byte metrics are included. The existing engine sleep-state model is outside this PR; resource dimensions for level-0, partial wake, and KV release are implemented separately in draft follow-up #59130.

Testing

Current revision (47e7f39408):

• Passed Ruff check/format, Python syntax checks (10 changed files), test tethering for the Python tests, and git diff --check.
• Focused tests cover response schemas, validation before dispatch, DP all-engine wake including mixed results, generic HTTP 2xx/4xx/5xx counters, duration/in-flight cleanup, cancellation, and production /metrics scraping.
• Cargo/rustfmt are unavailable locally; revised Rust tests have not been run. No new H20 run was requested.

Previously verified Linux/H20 results at 403f4b348f, before these review changes (test report):

• Python CPU sleep route/metrics tests: 19 passed, 1 deselected.
• Rust metrics: 6 passed; sleep, release, and wake route tests: 2 passed each.
• Production Python serve: existing sleep integration test passed with Qwen/Qwen3-0.6B (model substitution); separate lifecycle driver passed 40/40 checks, including partial wake and metrics scrape.
• These results validate the earlier implementation and do not claim execution of the newly added tests. No Rust live-server E2E, level-2/mode-keep E2E, or model evaluation was run.

Rebased onto main aedaba8664. All commits include Signed-off-by. AI assistance was used; review and Linux CI of the current revision are needed before merge.

[claude]

Claude Code Review

This pull request is from a fork — automated review is disabled. A repository maintainer can comment @claude review to run a one-time review.

[Ronald1995]

@aoshen02 could you please review this pr，thanks!

[mergify]

This pull request has merge conflicts that must be resolved before it can be
merged. Please rebase the PR, @Ronald1995.

https://docs.github.com/en/pull-requests/collaborating-with-pull-requests/working-with-forks/syncing-a-fork

[coderabbitai]

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info

⚙️ Run configuration

Configuration used: Repository UI

Review profile: CHILL

Plan: Team

Run ID: 1e4a68b8-25ba-4870-a00c-051d774eb214

📥 Commits

Reviewing files that changed from the base of the PR and between 6fbb00b and 8ebc3072e3b4f3b0f1795f1c8e7d1fe4d14ebffd.

📒 Files selected for processing (13)

• tests/entrypoints/serve/dev/rlhf/__init__.py
• tests/entrypoints/serve/dev/rlhf/conftest.py
• tests/entrypoints/serve/dev/rlhf/info_retrieval/__init__.py
• tests/entrypoints/serve/dev/rlhf/info_retrieval/test_response_schema.py
• tests/entrypoints/serve/dev/rlhf/info_retrieval/test_weight_info.py
• tests/entrypoints/serve/dev/rlhf/state_transitions/__init__.py
• tests/entrypoints/serve/dev/rlhf/state_transitions/test_pause_resume.py
• tests/kernels/moe/test_b12x.py
• tests/models/test_initialization.py
• tests/v1/kv_connector/unit/test_mooncake_store_scheduler.py
• vllm/distributed/kv_transfer/kv_connector/v1/mooncake/store/scheduler.py
• vllm/entrypoints/serve/dev/sleep/api_router.py
• vllm/model_executor/layers/fused_moe/oracle/mxfp4.py

Included review availability: Your plan provides up to 10 included reviews per hour; 7 remain after this review.

---

Walkthrough

The changes add structured sleep, wake, and weight-version responses with RLHF lifecycle tests. They also improve RLHF server helpers and fix independent Mooncake scheduling, MXFP4 backend selection, and ROCm initialization test coverage.

Changes

RLHF HTTP lifecycle contracts

Layer / File(s)	Summary
RLHF server and request helpers tests/entrypoints/serve/dev/rlhf/__init__.py, tests/entrypoints/serve/dev/rlhf/conftest.py	The test server now uses RemoteOpenAIServer. Request failures propagate. Lifecycle and weight-version response helpers return raw responses.
Sleep, wake, and weight response contracts vllm/entrypoints/serve/dev/sleep/api_router.py, tests/entrypoints/serve/dev/rlhf/info_retrieval/*	Sleep and wake endpoints return validated state, tags, levels, and elapsed times. Tests cover state transitions, schemas, weight-version updates, and invalid payloads.
Pause and resume request lifecycle tests/entrypoints/serve/dev/rlhf/state_transitions/*	The pause/resume test waits for in-flight completion and records thread errors before asserting successful completion.

Mooncake load scheduling

Layer / File(s)	Summary
Reject non-selected connector loads vllm/distributed/kv_transfer/kv_connector/v1/mooncake/store/scheduler.py, tests/v1/kv_connector/unit/test_mooncake_store_scheduler.py	The scheduler skips load specifications that cannot load. The regression test verifies that rejected connector state is not retained.

MXFP4 backend selection

Layer / File(s)	Summary
Select supported activation variants vllm/model_executor/layers/fused_moe/oracle/mxfp4.py, tests/kernels/moe/test_b12x.py	Explicit non-B12X backends now try activation variants in priority order. The test covers FlashInfer Cutlass fallback to the W4A8-compatible variant.

ROCm initialization coverage

Layer / File(s)	Summary
Skip unsupported HYV4 initialization tests/models/test_initialization.py	The large-subset initialization test skips HYV4 architectures on ROCm platforms.

Estimated code review effort: 4 (Complex) | ~45 minutes

Sequence Diagram(s)

sequenceDiagram
  participant RLHFTestClient
  participant SleepAPI
  participant EngineClient
  RLHFTestClient->>SleepAPI: POST sleep or wake request
  SleepAPI->>EngineClient: Apply requested state transition
  EngineClient-->>SleepAPI: Return engine state
  SleepAPI-->>RLHFTestClient: Return status, tags, and elapsed time

Loading

Merge Risk: ⚪ Minimal · up to 8ebc3

The updated lifecycle responses, validation, scheduler handling, backend fallback, and test coverage do not leave an identified merge-blocking risk.

🚥 Pre-merge checks | ✅ 3 | ❌ 2

❌ Failed checks (2 warnings)

Check name	Status	Explanation	Resolution
Docstring Coverage	⚠️ Warning	Docstring coverage is 45.65% which is insufficient. The required threshold is 80.00%. Docstring coverage is scoped to functions touched by this diff. Analyzed 46 functions across 13 files.	Write docstrings for the functions missing them to satisfy the coverage threshold.
Out of Scope Changes check	⚠️ Warning	Several changes do not relate to the stated sleep API, metrics, or RLHF information-retrieval objectives. These include Mooncake scheduler changes, DeepSeek MoE backend changes, and the ROCm initializ…	Move the unrelated Mooncake, MoE backend, and ROCm initialization changes into separate pull requests, or document their direct relationship to this pull request's objectives.

✅ Passed checks (3 passed)

Check name	Status	Explanation
Linked Issues check	✅ Passed	The description includes references to follow-up issue #59130 and prior test evidence in pull request #52864.
Title check	✅ Passed	The title clearly describes the main sleep-mode API response and metrics changes covered by the pull request.
Description check	✅ Passed	The description directly relates to the changeset and explains the API contract, metrics, testing scope, and known limitations.

Full details: Out of Scope Changes check

Explanation

Several changes do not relate to the stated sleep API, metrics, or RLHF information-retrieval objectives. These include Mooncake scheduler changes, DeepSeek MoE backend changes, and the ROCm initialization skip.

---

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

• X
• Mastodon
• Reddit
• LinkedIn

_{Comment @coderabbitai help to get the list of available commands.}

[Ronald1995]

H20 verification — revision 2 (47e7f394)

Re-verified after the update. Verified commit: 47e7f394085ecdfaa68df604a182ee3c33d8a483 (current PR head; no rebase, no local cherry-picks), diff base aedaba8664f67d8ae1538e5e5cec2b1ce3f258dd (HEAD~3), 15 files.

Environment

Same as the previous run: 1 × NVIDIA H20-3e, driver 580.105.08, Slurm h20 partition, Pyxis container nvcr.io/nvidia/cuda:12.4.1-devel-ubuntu22.04. Compiled extensions were reused from a prebuilt vLLM build (no C++/CUDA surface is touched); the installed package carries the PR-head Python tree — all 2569 .py/.pyi files were verified byte-identical to the worktree at 47e7f394, plus the 7 changed files by sha256.

1. Pre-commit and DCO — now fully green

ruff check / ruff format / typos / check-test-tethering / mypy-3.10 /
check-spdx-header / check-root-lazy-imports / check-forbidden-imports /
prevent-new-torch.cuda-APIs / validate-config / docker-dep-graph ....... Passed
Rust - Format code ..................................................... Passed

rust-cargo-fmt — which rewrote 3 files in the previous revision — now passes, and cargo fmt --all -- --check exits 0 with the repo-pinned toolchain (rust-toolchain.toml → 1.95, rustfmt 1.9.0-stable). No hook left the worktree dirty.

DCO: all three commits (fa4a7048be, 5de03a85a0, 47e7f39408) carry a matching Signed-off-by; the PR's DCO app check is pass.

2. Python CPU unit tests — PASS

$ pytest -q --import-mode=importlib -m cpu_test \
    tests/entrypoints/serve/dev/test_sleep.py \
    tests/v1/engine/test_wake_up_contract.py \
    tests/entrypoints/llm/test_wake_up_contract.py
62 tests collected
61 passed, 1 deselected, 14 warnings

(1 deselected = the GPU-only test_sleep_mode.)

3. Rust tests — PASS

cargo fmt --all -- --check                                                  → clean
cargo test -p vllm-metrics --lib                                            → 6 passed
cargo test -p vllm-server --lib routes::tests::sleep                        → 2 passed
cargo test -p vllm-server --lib routes::tests::release_kv_cache_memory      → 2 passed
cargo test -p vllm-server --lib routes::tests::wake_up                      → 2 passed
cargo test -p vllm-engine-core-client --lib wake_up                         → 1 passed

The last one is the new tests::client::wake_up_requires_every_engine_to_be_fully_awake.

4. GPU end-to-end, single engine (DP=1) — PASS

test_sleep_mode still hardcodes the gated meta-llama/Llama-3.2-1B (not in this cluster's cache, no token), so its body was re-run verbatim with only MODEL_NAME replaced by Qwen/Qwen3-0.6B: 1 passed. A verifier-written driver against a real vllm serve --enable-sleep-mode then passed 43/43 checks:

sleep level=0                        → 200 {"status":"sleeping","level":0}
release_kv_cache_memory              → 200 {"status":"kv_cache_released"}
sleep level=1                        → 200 {"status":"sleeping","level":1}
sleep level=1&mode=wait              → 200 {"status":"sleeping","level":1}
sleep level=2&mode=keep              → 200 {"status":"sleeping","level":2}
wake_up                              → 200 {"status":"awake","tags":null}
wake_up?tags=weights                 → 200 {"status":"sleeping","tags":["weights"]}
wake_up?tags=kv_cache                → 200 {"status":"awake","tags":["kv_cache"]}
sleep?level=3 / level=-1 / level=invalid → 400, error.param == "query.level"
sleep?mode=invalid                   → 400, error.param == "query.mode"

Things specific to this revision that the live server confirmed:

• The metric surface really is narrowed. After the whole sequence, /metrics exposes only vllm:sleep_mode_operation_duration_seconds and vllm:sleep_mode_operations_in_flight. No sample from the removed counter, and no vllm:rl_sleep_mode_* / ..._operations_total name appears anywhere in the scrape — so the rename and the counter removal are complete, not just in the diff.
• tags is the field name on /wake_up (not tags_woken), and null correctly means "wake all".
• Rejected requests stay out of the sleep-mode metrics: the sum of ..._duration_seconds_count was unchanged across the four 400s (13 → 13), and in_flight is back to 0.
• Both release outcomes are recorded: the level-0 pause path produced duration_seconds_count{release_kv_cache_memory} = 2 after one success and one engine-precondition rejection, with in_flight back to 0 each time.
• The Rust middleware change has a Python-side counterpart in effect: http_requests_total{handler="/sleep",method="POST",status="2xx"} = 5 and {status="4xx"} = 4 now appear for these routes.
• vllm:engine_sleep_state is unchanged and still populated.

5. GPU end-to-end, multi-engine (DP=2) — PASS

vllm serve --data-parallel-size 2 --enable-sleep-mode on 2 H20s, then the same sleep/partial-wake/full-wake sequence. All HTTP checks passed, and the per-rank engine logs prove the fan-out:

(EngineCore_DP0 pid=3275455)  ... to fall asleep
(EngineCore_DP1 pid=3275456)  ... to fall asleep
(EngineCore_DP0 pid=3275455)  ... to wake up tags ['weights']
(EngineCore_DP1 pid=3275456)  ... to wake up tags ['weights']
(EngineCore_DP0 pid=3275455)  ... to wake up tags ['kv_cache']
(EngineCore_DP1 pid=3275456)  ... to wake up tags ['kv_cache']

• 2 EngineCore processes, ranks DP0 + DP1, launched with data_parallel_size: 2 and api_server_count: 2.
• 10 "wake up tags" rank-lines for 5 wake calls and 8 "fall asleep" rank-lines for 4 sleep calls — i.e. every wake/sleep reached both DP cores, which is what the new DPLBAsyncMPClient.wake_up_async is for.
• wake_up?tags=weights → {"status":"sleeping"} while is_sleeping stayed true; the follow-up wake_up?tags=kv_cache → {"status":"awake"}; three further sleep/wake round trips all returned awake; in_flight returned to 0 for both operations.

One limitation to be explicit about: sleep and wake apply to all ranks at once, so the two ranks cannot be driven into different states through the HTTP API. This run therefore proves the DP path executes on real engines and drives every rank, but it does not by itself distinguish all() from "first rank's result" — that discrimination rests on the new test_dp_wake_up_requires_all_engines_awake (mocked [True, False] and [False, True] → False) and the Rust wake_up_requires_every_engine_to_be_fully_awake test.

Not verified

• Rust frontend over real HTTP — only cargo test route/metric unit tests; no live vllm-server process was driven end to end.
• Non-bool wake_up from a real engine — unreachable with the current engine (it returns a bool), so the 500 ... must return a bool paths are covered only by the new unit tests.
• No model evaluation (accuracy/output) — the generation path is unchanged.
• Local note, unrelated to this PR: the container's CUDA 12.4 nvcc rejects FlashInfer 0.7.0's --compress-mode=size, which kills EngineCore at kernel warmup, so the GPU runs used VLLM_USE_FLASHINFER_SAMPLER=0.

_{Verification run with AI assistance; results are from Slurm job logs on the cluster. An assertion bug in the first revision of my DP verifier (wrong data_parallel_size grep, wrong EngineCore_DP* regex, and an off-by-one in the expected wake count) was corrected and re-evaluated against the same captured server log.}

[mergify]

This pull request has merge conflicts that must be resolved before it can be
merged. Please rebase the PR, @Ronald1995.

https://docs.github.com/en/pull-requests/collaborating-with-pull-requests/working-with-forks/syncing-a-fork

[aoshen02]

/ci run

[github-actions]

✅ @Ronald1995, CI is now available for this PR.

• /ci run starts upstream CI; /amd-ci run starts AMD CI only.
• Your branch must contain every commit currently on its upstream target branch. Merge or rebase onto the latest target branch, then rerun the command. Append --allow-stale to a run command to test an outdated branch at your own risk.
• /ci retry retries failed jobs in the CI build for the current PR head. If the current head has no CI build, it starts a new CI build for the current head containing only jobs that failed in the latest earlier CI build for this PR.
• /amd-ci retry retries failed jobs in AMD CI for the current PR head. Use /amd-ci run when the current head has no AMD CI build.
• /ci cancel cancels scheduled or running CI builds for this PR branch; /amd-ci cancel does the same for AMD CI only.

[github-actions]

✅ Triggered Buildkite CI #91995 for commit 6b1e75027b1f.

[aoshen02]

/ci run

[github-actions]

✅ CI is already running for this commit: https://buildkite.com/vllm/ci/builds/91995

[aoshen02]

/ci retry

[github-actions]

✅ Queued 1 failed job(s) for retry in Buildkite CI #91995.

[vllm-agent]

CI selector (shadow): 178 test steps (253 jobs) instead of 76 (92 jobs)

Shadow mode: this changes nothing about what CI runs. It shows what the evidence-based selector would pick for this PR, next to today's rules. How it works.

Feedback welcome: reply here if it would skip a step this change needs, or runs something unrelated.

steps (jobs)	Today's rules	Selector	Would skip	Would add
NVIDIA, CPU and others	76 (92)	178 (253)	2 (2)	104 (163)
AMD mirrors	67 (81)	152 (219)	1 (1)	86 (139)

Selector would run (178)

• amd-fp8-moe-kernels-mi355
• amd-kernels-mi355
• amd-native-quantization-kernels-mi355
• arm-cpu-test ×3
• async-engine-inputs-utils-worker
• basic-correctness ×2
• basic-correctness-cpu-offload
• basic-correctness-cumem
• basic-correctness-prefetch-offload
• basic-correctness-sleep-mode
• basic-models-test-other-cpu
• basic-models-tests-extra-initialization ×14
• basic-models-tests-initialization
• basic-models-tests-other
• batch-invariance-b200
• batch-invariance-h100
• benchmarks-cli-test
• cpu-distributed-tests-dp-tp
• cpu-distributed-tests-pp-tp
• cpu-kernel-tests ×2
• cpu-language-generation-and-pooling-model-tests ×3
• cpu-multi-modal-model-tests-n ×4
• cpu-multimodal-config
• cpu-params-env-tokenizers-parser
• cpu-quantization-model-tests
• cpu-qwen2-5-vl-multimodal-tests
• cpu-reasoning-renderers
• cpu-spec-decode-tests
• cpu-tool-parsers
• crosslayer-kv-layout-distributed-nixlconnector-pd-accuracy-tests-4-gpus
• cudagraph
• deepseek-v4-kernel-test-b200
• deepseek-v4-kernel-test-h100
• distributed-comm-ops
• distributed-compile-comm-4-gpus
• distributed-compile-rpc-tests-2-gpus
• distributed-compile-unit-tests-2xh100
• distributed-dp-tests-2-gpus
• distributed-dp-tests-4-gpus
• distributed-flashinfer-nixlconnector-pd-accuracy-4-gpus
• distributed-model-tests-2-gpus ×3
• distributed-mooncakeconnector-pd-accuracy-4-gpus
• distributed-nixlconnector-pd-accuracy-4-gpus
• distributed-tests-8xh100
• distributed-torchrun-examples-4-gpus
• distributed-torchrun-shutdown-tests-2-gpus
• docker-build-metadata
• dp-ep-distributed-nixlconnector-pd-accuracy-tests-4-gpus
• e2e-core-1-gpu
• e2e-core-large-memory
• e2e-scheduling-1-gpu
• elastic-ep-scaling-test
• engine
• engine-1-gpu
• entrypoints-integration-api-server ×4
• entrypoints-integration-api-server-generate
• entrypoints-integration-api-server-openai-chat_completion
• entrypoints-integration-api-server-openai-completion
• entrypoints-integration-llm
• entrypoints-integration-multimodal
• entrypoints-integration-pooling
• entrypoints-integration-responses-api
• entrypoints-integration-speech_to_text
• entrypoints-unit-tests
• eplb-algorithm
• eplb-execution
• examples
• extract-hidden-states-integration
• extract-hidden-states-integration-2-gpus
• fault-tolerance-e2e-2xh100
• fusion-and-compile-unit-tests-2xb200
• fusion-e2e-config-sweep-h100
• fusion-e2e-quick-h100
• fusion-e2e-tp2-ar-rms-config-sweep-h100
• fusion-e2e-tp2-asynctp-config-sweep-h100
• fusion-e2e-tp2-b200
• fusion-e2e-tp2-quick-h100
• gemm-rs-ar-2xb200
• glm5next-unit-tests
• hybrid-ssm-nixlconnector-pd-accuracy-tests-4-gpus
• hybrid-ssm-nixlconnector-pd-prefix-cache-2-gpus
• inkling-unit-tests-b200
• jit-monitor-no-runtime-jit
• kernels-attention-diffkv-test-h100
• kernels-attention-test ×7
• kernels-b200 ×3
• kernels-core-operation-test ×3
• kernels-deepgemm-test-h100
• kernels-fla-ops-test-b200
• kernels-flashmla-test-h100
• kernels-fusedmoe-layer-test-2-b200s
• kernels-fusedmoe-layer-test-2-h100s
• kernels-helion-test ×5
• kernels-mamba-test
• kernels-mhc-test-b200
• kernels-minimax-reduce-rms-test-2-gpus
• kernels-moe-test ×5
• kernels-quantization-test ×6
• kernels-root-misc-test-b200
• kimi-k3-unit-tests-b200
• kv-offload-large
• kv-offload-medium
• kv-offload-small
• language-models-tests-extra-standard ×2
• language-models-tests-granite-l4-compatibility
• language-models-tests-hybrid ×2
• language-models-tests-standard
• lm-eval-dspark-watermark-2xh100
• lm-eval-small-models
• lm-eval-turboquant-k3v4nc
• lm-eval-turboquant-k8v4
• lm-eval-turboquant-t3nc
• lm-eval-turboquant-t4nc
• lm-eval-watermarking
• lora ×4
• lora-tp-distributed ×4
• metrics-tracing-2-gpus
• model-executor
• mrcr-eval-small-models
• multi-modal-models-standard-1-qwen2
• multi-modal-models-standard-2-qwen3-gemma
• multi-modal-models-standard-3-llava-qwen2-vl
• multi-modal-models-standard-4-other-whisper
• multi-modal-processor ×4
• multi-modal-processor-cpu ×4
• multiconnector-nixl-offloading-pd-accuracy-2-gpus
• multiconnector-nixl-offloading-pd-edge-cases-2-gpus
• nixlconnector-pd-edge-cases-2-gpus
• openai-api-correctness
• pipeline-context-parallelism-4-gpus
• platform-tests
• plugin-tests-2-gpus
• push-nixlconnector-pp-prefill-pd-accuracy-4-gpus
• pytorch-compilation-dynamic-shapes
• pytorch-compilation-passes-unit-tests
• pytorch-compilation-unit-tests
• pytorch-compilation-unit-tests-h100
• pytorch-fullgraph-cudagraph-l4-compatibility
• pytorch-fullgraph-test
• quantization ×4
• quantized-fusions
• quantized-models-test
• quantized-moe-test-b200
• qwen4-exp-unit-tests
• qwen4-exp-unit-tests-cpu
• rayexecutorv2-4-gpus
• regression
• rust-frontend-cargo-style-clippy
• rust-frontend-cargo-tests
• rust-frontend-core-correctness
• rust-frontend-distributed
• rust-frontend-openai-coverage
• rust-frontend-serve-admin-coverage
• rust-frontend-tool-use
• samplers-multimodal-beam-search
• samplers-test
• scale-out-ec-e2e-2-gpus
• sharded-rdt-weight-transfer
• spec-decode-draft-model ×4
• spec-decode-eagle-1-deepseek-qwen
• spec-decode-eagle-2-llama3-qwen-vl-other
• spec-decode-mtp-deepseek-mimo
• spec-decode-mtp-gemma4
• spec-decode-mtp-qwen3-5
• spec-decode-ngram-suffix
• spec-decode-speculators
• v1-attention-b200 ×2
• v1-attention-h100-mi300 ×2
• v1-core
• v1-executor-worker
• v1-kv-connectors ×4
• v1-kv-offload
• v1-logits-oracle
• v1-metrics-lmeval
• v1-others-cpu
• v1-sample
• v1-spec-decode
• vllm-ir-tests

Would skip (today's rules run them) (2)

• ascend-npu-test
• mooncake-ec-tcp-e2e-2-gpus

Would add (today's rules do not run them) (104)

• amd-fp8-moe-kernels-mi355 (code map)
• amd-kernels-mi355 (code map)
• amd-native-quantization-kernels-mi355 (code map)
• arm-cpu-test ×3 (code map)
• basic-models-tests-extra-initialization ×14 (code map)
• batch-invariance-b200 (code map)
• batch-invariance-h100 (code map)
• cpu-distributed-tests-dp-tp (code map)
• cpu-distributed-tests-pp-tp (code map)
• cpu-kernel-tests ×2 (code map)
• cpu-multi-modal-model-tests-n ×4 (code map)
• cpu-quantization-model-tests (code map)
• cpu-qwen2-5-vl-multimodal-tests (code map)
• cpu-spec-decode-tests (code map)
• crosslayer-kv-layout-distributed-nixlconnector-pd-accuracy-tests-4-gpus (code map)
• cudagraph (code map)
• deepseek-v4-kernel-test-b200 (code map)
• deepseek-v4-kernel-test-h100 (code map)
• distributed-comm-ops (code map)
• distributed-compile-comm-4-gpus (code map)
• distributed-compile-unit-tests-2xh100 (code map)
• distributed-dp-tests-4-gpus (code map)
• distributed-flashinfer-nixlconnector-pd-accuracy-4-gpus (code map)
• distributed-model-tests-2-gpus ×3 (code map)
• distributed-mooncakeconnector-pd-accuracy-4-gpus (code map)
• distributed-nixlconnector-pd-accuracy-4-gpus (code map)
• distributed-torchrun-examples-4-gpus (code map)
• docker-build-metadata (code map)
• dp-ep-distributed-nixlconnector-pd-accuracy-tests-4-gpus (code map)
• eplb-algorithm (code map)
• eplb-execution (code map)
• extract-hidden-states-integration (code map)
• extract-hidden-states-integration-2-gpus (code map)
• fusion-and-compile-unit-tests-2xb200 (code map)
• fusion-e2e-config-sweep-h100 (code map)
• fusion-e2e-quick-h100 (code map)
• fusion-e2e-tp2-ar-rms-config-sweep-h100 (code map)
• fusion-e2e-tp2-asynctp-config-sweep-h100 (code map)
• fusion-e2e-tp2-b200 (code map)
• fusion-e2e-tp2-quick-h100 (code map)
• gemm-rs-ar-2xb200 (code map)
• glm5next-unit-tests (code map)
• hybrid-ssm-nixlconnector-pd-accuracy-tests-4-gpus (code map)
• hybrid-ssm-nixlconnector-pd-prefix-cache-2-gpus (code map)
• inkling-unit-tests-b200 (code map)
• jit-monitor-no-runtime-jit (code map)
• kernels-attention-diffkv-test-h100 (code map)
• kernels-attention-test ×7 (code map)
• kernels-b200 ×3 (code map)
• kernels-core-operation-test ×3 (code map)
• kernels-deepgemm-test-h100 (code map)
• kernels-flashmla-test-h100 (code map)
• kernels-fusedmoe-layer-test-2-b200s (code map)
• kernels-fusedmoe-layer-test-2-h100s (code map)
• kernels-helion-test ×5 (code map)
• kernels-mamba-test (code map)
• kernels-minimax-reduce-rms-test-2-gpus (code map)
• kernels-moe-test ×5 (code map)
• kernels-quantization-test ×6 (code map)
• kimi-k3-unit-tests-b200 (code map)
• kv-offload-large (code map)
• kv-offload-medium (code map)
• kv-offload-small (code map)
• language-models-tests-extra-standard ×2 (code map)
• lm-eval-dspark-watermark-2xh100 (code map)
• lm-eval-small-models (code map)
• lm-eval-turboquant-k3v4nc (code map)
• lm-eval-turboquant-k8v4 (code map)
• lm-eval-turboquant-t3nc (code map)
• lm-eval-turboquant-t4nc (code map)
• lm-eval-watermarking (code map)
• lora ×4 (code map)
• lora-tp-distributed ×4 (code map)
• model-executor (code map)
• mrcr-eval-small-models (code map)
• multiconnector-nixl-offloading-pd-accuracy-2-gpus (code map)
• multiconnector-nixl-offloading-pd-edge-cases-2-gpus (code map)
• nixlconnector-pd-edge-cases-2-gpus (code map)
• openai-api-correctness (code map)
• platform-tests (code map)
• plugin-tests-2-gpus (code map)
• push-nixlconnector-pp-prefill-pd-accuracy-4-gpus (code map)
• quantization ×4 (code map)
• quantized-fusions (code map)
• quantized-models-test (code map)
• quantized-moe-test-b200 (code map)
• qwen4-exp-unit-tests (code map)
• qwen4-exp-unit-tests-cpu (code map)
• rayexecutorv2-4-gpus (code map)
• samplers-multimodal-beam-search (code map)
• samplers-test (code map)
• scale-out-ec-e2e-2-gpus (code map)
• sharded-rdt-weight-transfer (code map)
• spec-decode-draft-model ×4 (code map)
• spec-decode-eagle-1-deepseek-qwen (code map)
• spec-decode-eagle-2-llama3-qwen-vl-other (code map)
• spec-decode-mtp-deepseek-mimo (code map)
• spec-decode-mtp-gemma4 (code map)
• spec-decode-mtp-qwen3-5 (code map)
• spec-decode-ngram-suffix (code map)
• spec-decode-speculators (code map)
• v1-attention-b200 ×2 (code map)
• v1-attention-h100-mi300 ×2 (code map)
• vllm-ir-tests (code map)

AMD mirrors: would skip (1)

• mooncake-ec-tcp-e2e-2-gpus

AMD mirrors: would add (86)

• basic-models-tests-extra-initialization ×14 (code map)
• batch-invariance-h100 (code map)
• crosslayer-kv-layout-distributed-nixlconnector-pd-accuracy-tests-4-gpus (code map)
• cudagraph (code map)
• deepseek-v4-kernel-test-b200 (code map)
• deepseek-v4-kernel-test-h100 (code map)
• distributed-flashinfer-nixlconnector-pd-accuracy-4-gpus (code map)
• distributed-model-tests-2-gpus ×3 (code map)
• distributed-mooncakeconnector-pd-accuracy-4-gpus (code map)
• distributed-nixlconnector-pd-accuracy-4-gpus (code map)
• distributed-torchrun-examples-4-gpus (code map)
• docker-build-metadata (code map)
• dp-ep-distributed-nixlconnector-pd-accuracy-tests-4-gpus (code map)
• eplb-algorithm (code map)
• eplb-execution (code map)
• extract-hidden-states-integration (code map)
• extract-hidden-states-integration-2-gpus (code map)
• fusion-and-compile-unit-tests-2xb200 (code map)
• fusion-e2e-config-sweep-h100 (code map)
• fusion-e2e-quick-h100 (code map)
• fusion-e2e-tp2-ar-rms-config-sweep-h100 (code map)
• fusion-e2e-tp2-asynctp-config-sweep-h100 (code map)
• fusion-e2e-tp2-b200 (code map)
• fusion-e2e-tp2-quick-h100 (code map)
• gemm-rs-ar-2xb200 (code map)
• glm5next-unit-tests (code map)
• hybrid-ssm-nixlconnector-pd-accuracy-tests-4-gpus (code map)
• hybrid-ssm-nixlconnector-pd-prefix-cache-2-gpus (code map)
• inkling-unit-tests-b200 (code map)
• jit-monitor-no-runtime-jit (code map)
• kernels-attention-diffkv-test-h100 (code map)
• kernels-attention-test ×7 (code map)
• kernels-b200 ×3 (code map)
• kernels-core-operation-test ×3 (code map)
• kernels-flashmla-test-h100 (code map)
• kernels-fp4-moe-test-b200 (code map)
• kernels-fusedmoe-layer-test-2-b200s (code map)
• kernels-fusedmoe-layer-test-2-h100s (code map)
• kernels-helion-test ×5 (code map)
• kernels-mamba-test (code map)
• kernels-minimax-reduce-rms-test-2-gpus (code map)
• kernels-moe-test ×5 (code map)
• kernels-quantization-test ×6 (code map)
• kimi-k3-unit-tests-b200 (code map)
• kv-offload-large (code map)
• kv-offload-medium (code map)
• kv-offload-small (code map)
• language-models-tests-extra-standard ×2 (code map)
• lm-eval-dspark-watermark-2xh100 (code map)
• lm-eval-small-models (code map)
• lm-eval-turboquant-k3v4nc (code map)
• lm-eval-turboquant-k8v4 (code map)
• lm-eval-turboquant-t3nc (code map)
• lm-eval-turboquant-t4nc (code map)
• lm-eval-watermarking (code map)
• lora ×4 (code map)
• lora-tp-distributed ×4 (code map)
• model-executor (code map)
• mrcr-eval-small-models (code map)
• multiconnector-nixl-offloading-pd-accuracy-2-gpus (code map)
• multiconnector-nixl-offloading-pd-edge-cases-2-gpus (code map)
• nixlconnector-pd-edge-cases-2-gpus (code map)
• nixlconnector-pd-spec-decode-acceptance-2-gpus (code map)
• openai-api-correctness (code map)
• plugin-tests-2-gpus (code map)
• push-nixlconnector-pp-prefill-pd-accuracy-4-gpus (code map)
• quantization ×4 (code map)
• quantized-fusions (code map)
• quantized-models-test (code map)
• quantized-moe-test-b200 (code map)
• qwen4-exp-unit-tests (code map)
• samplers-multimodal-beam-search (code map)
• samplers-test (code map)
• scale-out-ec-e2e-2-gpus (code map)
• sharded-rdt-weight-transfer (code map)
• spec-decode-draft-model ×4 (code map)
• spec-decode-eagle-1-deepseek-qwen (code map)
• spec-decode-eagle-2-llama3-qwen-vl-other (code map)
• spec-decode-mtp-deepseek-mimo (code map)
• spec-decode-mtp-gemma4 (code map)
• spec-decode-mtp-qwen3-5 (code map)
• spec-decode-ngram-suffix (code map)
• spec-decode-speculators (code map)
• v1-attention-b200 ×2 (code map)
• v1-attention-h100-mi300 ×2 (code map)
• vllm-ir-tests (code map)

CI results (2026-09-30 14:41 UTC)

165 passed, 0 failed, 1 pending.
CI is still running; the picture below is not final.

No failures to judge.

_{14 changed files · base 29773c722b · head e5f3d08d72 · Python record: not used (/tmp/ci-infra-selector/buildkite/ci_selector/coverage-data/table.json.gz is table version 5, expected 7; re-merge it from the raw recordings) · kernel record: table 866fa13 (build 92059), map 866fa13 · not counted: 10 build steps, 5 A100 steps the generator no longer emits, 124 optional steps the selector would also run}