[aoshen02]

Purpose

Implements the test matrix proposed in RFC #45585.

Adds tests/entrypoints/serve/dev/rlhf/ with two files that cover the full
RL lifecycle through the production HTTP endpoint surface (VLLM_SERVER_DEV_MODE=1).
All tests require --enable-sleep-mode — without it sleep/wake_up are no-ops
and the underlying bugs cannot be exercised.

Files

test_sleep_wake.py — 6 classes, 26 tests

Class	What it tests
TestSleepWakeFlags	is_sleeping flag + Prometheus vllm:engine_sleep_state metrics (supersedes test_sleep.py)
TestPhysicalMemory	GPU free-bytes actually change per sleep level/tag; Prometheus metrics cross-validated
TestSchedulerGate	Scheduler must not dispatch during partial or full wake; direct regression for #44483
TestOutputCorrectness	Golden-output roundtrip: sleep(1)→wake, staged wake, 3-cycle stability, prefix-cache flush
TestPauseResume	/pause /resume /is_paused — all three PauseMode values (abort/wait/keep)
TestErrorPaths	Idempotency (double-sleep, wake-while-awake), abort+sleep interaction

test_weight_update.py — 3 classes, 7 tests

Class	What it tests
TestWeightTransferProtocol	/start_weight_update → /finish_weight_update framing, prefix-cache flush after update
TestWeightUpdateWithTensors	Output changes after pushing real tensors (skipped unless VLLM_TEST_MODEL_ALT is set)
TestCompoundRLStep	Full RL training-step: sleep → update → wake(weights) → wake(kv_cache) → rollout

Bugs directly addressed

Bug	Test
#44483 partial-wake IMA	TestSchedulerGate::test_partial_wake_blocks_until_kv_resident
#45326 request hang during sleep	TestSchedulerGate::test_sleep_abort_mode_blocks_new_requests
#45518 double-sleep idempotency	TestErrorPaths::test_double_sleep_idempotent

Test plan

VLLM_TEST_MODEL=/path/to/Llama-3.2-1B-Instruct \
  pytest tests/entrypoints/serve/dev/rlhf/ -v

Verified on H200, Qwen3-0.6B, --enforce-eager.

Notes

• test_sleep.py (existing) is left untouched; TestSleepWakeFlags supersedes it
  and it can be removed in a follow-up once this suite is stable in CI.
• TestWeightUpdateWithTensors is skipped by default (requires VLLM_TEST_MODEL_ALT);
  intended to run nightly or manually.
• AI assistance was used to draft these tests; all assertions were verified by
  running against both patched and un-patched vLLM on H200.

[mergify]

This pull request has merge conflicts that must be resolved before it can be
merged. Please rebase the PR, @aoshen02.

https://docs.github.com/en/pull-requests/collaborating-with-pull-requests/working-with-forks/syncing-a-fork

[AlanFokCo]

@aoshen02 Great test matrix — this kind of systematic lifecycle coverage is exactly what sleep mode needs.

A few edge cases from our multi-model hot-swap work that aren't in the current matrix:

1. Discarded-then-slept double-unmap: if a tag's physical memory is released via unmap_and_release (e.g. stale KV cache freed to make room for a new model) and then a full sleep() is called, the current sleep() implementation iterates all pointer_to_data entries and calls unmap_and_release again on the already-released handles → CUDA_ERROR_INVALID_VALUE. We hit this in production when combining selective memory management with full sleep cycles.

2. CUDA graph pool VMM slowdown: VMM operations (sleep/wake_up) take 3-5x longer when a CUDA graph capture pool is present on the device. Our measurements on 32B TP=8:

   • Without graph pool: ~2.0s
   • With graph pool: ~6-9s

   This isn't a bug per se, but it's a significant behavioral cliff that users will hit in production. Might be worth a test that asserts sleep/wake timing doesn't regress by more than Nx when enforce_eager=False.

3. Cross-tag wake ordering: waking "weights" before "kv_cache" is safe, but waking "kv_cache" first can trigger flash_attn to access remapped-but-uninitialized weight memory if a forward pass is accidentally dispatched between the two wake calls. A test asserting that partial wake + forward → clean error (not segfault) would be valuable.

Happy to contribute these as additional test cases if they fit the scope.

[aoshen02]

@aoshen02 Great test matrix — this kind of systematic lifecycle coverage is exactly what sleep mode needs.

A few edge cases from our multi-model hot-swap work that aren't in the current matrix:

1. Discarded-then-slept double-unmap: if a tag's physical memory is released via unmap_and_release (e.g. stale KV cache freed to make room for a new model) and then a full sleep() is called, the current sleep() implementation iterates all pointer_to_data entries and calls unmap_and_release again on the already-released handles → CUDA_ERROR_INVALID_VALUE. We hit this in production when combining selective memory management with full sleep cycles.

2. CUDA graph pool VMM slowdown: VMM operations (sleep/wake_up) take 3-5x longer when a CUDA graph capture pool is present on the device. Our measurements on 32B TP=8:

   • Without graph pool: ~2.0s
   • With graph pool: ~6-9s

   This isn't a bug per se, but it's a significant behavioral cliff that users will hit in production. Might be worth a test that asserts sleep/wake timing doesn't regress by more than Nx when enforce_eager=False.

3. Cross-tag wake ordering: waking "weights" before "kv_cache" is safe, but waking "kv_cache" first can trigger flash_attn to access remapped-but-uninitialized weight memory if a forward pass is accidentally dispatched between the two wake calls. A test asserting that partial wake + forward → clean error (not segfault) would be valuable.

Happy to contribute these as additional test cases if they fit the scope.

I see, if needed please find me at slack via aoshen524@gmail.com or add my wechat a18102550020, thanks.

[Ronald1995]

it seems like if r is normal, we need to fail the testcase, but it doesn't make it here.

[aoshen02]

Thanks for the headsup!