[GirasoleY]

Summary

Async KV loads reserve the blocks they still need to finish, but the scheduler only honored that reservation when admitting another async load. Ordinary local admissions and subsequent running chunks could consume the reserved headroom. The full-sequence admission check also ignored reserved_blocks.

This change:

• applies in-flight prefill reservations to both waiting and running allocations;
• excludes the request that owns the reservation when it resumes;
• includes reserved blocks in the full-sequence fit check; and
• adds a regression test where a local request would otherwise consume an async load's remaining capacity.

The reservation is only nonzero while async loads or chunked prefills are in flight, so normal scheduling is unchanged.

Why this is not a duplicate

I searched open PRs for async KV reservation, scheduler deadlock, and reserved_blocks.

• [BugFix] Resolve multiple async kv load deadlock #44560 introduced reservation-based admission between async loads. This PR closes the remaining gap where non-async allocations ignore that reservation.
• [Bugfix][V1] Let completed async KV loads promote and run under saturation #42568 changes max_num_seqs accounting for async loads; it does not protect block reservations.
• [BugFix] Don't strand parked async-KV-load requests behind an unschedulable queue head #45406 scans past an unschedulable queue head to promote a completed parked load; it addresses a different post-admission queue-stranding state.

Validation

Focused lifecycle test file:

.venv/bin/python -m pytest \
  tests/v1/kv_connector/unit/test_remote_prefill_lifecycle.py -q

10 passed, 17 warnings in 6.82s

Repository hooks on the modified files:

pre-commit run --files \
  vllm/v1/core/kv_cache_manager.py \
  vllm/v1/core/sched/scheduler.py \
  tests/v1/kv_connector/unit/test_remote_prefill_lifecycle.py

All hooks passed.

I also ran a local two-GPU DSv4-Pro-shaped stress test with DP=2, EP=2, DeepGEMM MegaMoE, dummy weights, a deliberately small KV pool, one delayed async external-KV load, and heavily imbalanced requests. All 8 requests completed without a collective stall. On the original target branch, the same-config A/B reduced peak logical KV usage from 94.2% to 87.2% and completion time from 28.91s to 20.82s. Current upstream main with this patch completed in 20.78s with 87.8% peak KV usage.

This test validates the reservation-pressure behavior; it does not claim that every production collective hang has the same root cause.

AI assistance

OpenAI Codex was used to investigate the scheduler path, draft the patch and regression test, and run the validation. I reviewed every changed line and am responsible for the change.

[claude]

Claude Code Review

This pull request is from a fork — automated review is disabled. A repository maintainer can comment @claude review to run a one-time review.

[ivanium]

LGTM. Thanks for the fix!

[ivanium]

Actually I may need to test this further so hold up temporarily

[ivanium]

Actually there might be some bugs. I'll investigate further

[ivanium]

This applies to all requests and seems to be buggy and cause stuck in some of my tests.

[njhill]

Thanks @GirasoleY! Actually the current behavior is intentional - it optimistically allows use of the "reserved" blocks, with the assumption that more will usually be freed by the time they are actually needed (or else we resort to preemption).

We need to enforce the reserved blocks for other async loads to avoid deadlocks.

See this comment in the original PR:

Note: we apply this throttle to async kv loads only, rather than chunked-prefill requests in general, since the latter are naturally throttled by batch capacity and are also already eligible for preemption.

I'm not saying we definitely shouldn't make this change but It would be good to understand the motivation.

[mergify]

This pull request has merge conflicts that must be resolved before it can be
merged. Please rebase the PR, @GirasoleY.

https://docs.github.com/en/pull-requests/collaborating-with-pull-requests/working-with-forks/syncing-a-fork