Skip to content

[KV Offload] Pass ReqContext to touch(), complete_load(), and complete_store() - #41366

Merged
orozery merged 1 commit into
vllm-project:mainfrom
ronensc:req-context
May 10, 2026
Merged

orozery merged 1 commit into
vllm-project:mainfrom
ronensc:req-context

Conversation

@ronensc

@ronensc ronensc commented Apr 30, 2026 •

Copy link
Copy Markdown
Contributor

Purpose

Follow up on #39185

OffloadingManager already accepts req_context: ReqContext on the three "initiation" methods (lookup, prepare_load, prepare_store) but not on the three "completion/maintenance" methods (touch, complete_load, complete_store).
This PR closes that gap so the full interface is consistent and future manager implementations can act on per-request context during completion callbacks.

Test Plan

pytest -s -v tests/v1/kv_offload/cpu/test_manager.py

Test Result

tests/v1/kv_offload/cpu/test_manager.py::test_already_stored_block_not_evicted_during_prepare_store[lru] PASSED
tests/v1/kv_offload/cpu/test_manager.py::test_already_stored_block_not_evicted_during_prepare_store[arc] PASSED
tests/v1/kv_offload/cpu/test_manager.py::test_cpu_manager PASSED
tests/v1/kv_offload/cpu/test_manager.py::TestARCPolicy::test_basic PASSED
tests/v1/kv_offload/cpu/test_manager.py::TestARCPolicy::test_t1_to_t2_promotion PASSED
tests/v1/kv_offload/cpu/test_manager.py::TestARCPolicy::test_eviction_with_load PASSED
tests/v1/kv_offload/cpu/test_manager.py::TestARCPolicy::test_adaptive_target PASSED
tests/v1/kv_offload/cpu/test_manager.py::TestARCPolicy::test_t1_t2_eviction_policy PASSED
tests/v1/kv_offload/cpu/test_manager.py::TestARCPolicy::test_ghost_list_bounds PASSED
tests/v1/kv_offload/cpu/test_manager.py::TestARCPolicy::test_touch_ordering PASSED
tests/v1/kv_offload/cpu/test_manager.py::TestARCPolicy::test_failed_store PASSED
tests/v1/kv_offload/cpu/test_manager.py::TestARCPolicy::test_full_scenario PASSED
tests/v1/kv_offload/cpu/test_manager.py::test_filter_reused_manager PASSED

Essential Elements of an Effective PR Description Checklist
  • The purpose of the PR, such as "Fix some issue (link existing issues this PR will resolve)".
  • The test plan, such as providing test command.
  • The test results, such as pasting the results comparison before and after, or e2e results
  • (Optional) The necessary documentation update, such as updating supported_models.md and examples for a new model.

@claude claude Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Claude Code Review

This pull request is from a fork — automated review is disabled. A repository maintainer can comment @claude review to run a one-time review.

@gemini-code-assist gemini-code-assist Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code Review

This pull request updates the KV offloading interface and its implementations to include a ReqContext parameter in the touch, complete_load, and complete_store methods. These changes ensure that per-request context is consistently passed through the offloading manager, with corresponding updates applied to the scheduler and test suite. I have no feedback to provide as there are no review comments to evaluate.

@orozery orozery added the ready ONLY add when PR is ready to merge/full CI is needed label Apr 30, 2026
@mergify

mergify Bot commented Apr 30, 2026

Copy link
Copy Markdown
Contributor

Hi @ronensc, the pre-commit checks have failed. Please run:

uv pip install pre-commit>=4.5.1
pre-commit install
pre-commit run --all-files

Then, commit the changes and push to your branch.

For future commits, pre-commit will run automatically on changed files before each commit.

Tip

Is mypy failing?
mypy is run differently in CI. If the failure is related to this check, please use the following command to run it locally:
# For mypy (substitute "3.10" with the failing version if needed)
pre-commit run --hook-stage manual mypy-3.10

@mergify

mergify Bot commented May 4, 2026

Copy link
Copy Markdown
Contributor

Hi @ronensc, the pre-commit checks have failed. Please run:

uv pip install pre-commit>=4.5.1
pre-commit install
pre-commit run --all-files

Then, commit the changes and push to your branch.

For future commits, pre-commit will run automatically on changed files before each commit.

Tip

Is mypy failing?
mypy is run differently in CI. If the failure is related to this check, please use the following command to run it locally:
# For mypy (substitute "3.10" with the failing version if needed)
pre-commit run --hook-stage manual mypy-3.10

@mergify

mergify Bot commented May 4, 2026

Copy link
Copy Markdown
Contributor

Hi @ronensc, the pre-commit checks have failed. Please run:

uv pip install pre-commit>=4.5.1
pre-commit install
pre-commit run --all-files

Then, commit the changes and push to your branch.

For future commits, pre-commit will run automatically on changed files before each commit.

Tip

Is mypy failing?
mypy is run differently in CI. If the failure is related to this check, please use the following command to run it locally:
# For mypy (substitute "3.10" with the failing version if needed)
pre-commit run --hook-stage manual mypy-3.10

@ronensc
ronensc force-pushed the req-context branch 14 times, most recently from 6059286 to 1135332 Compare May 6, 2026 11:28
@ronensc
ronensc force-pushed the req-context branch 17 times, most recently from e76581f to 22406d0 Compare May 10, 2026 06:15
@mergify

mergify Bot commented May 10, 2026

Copy link
Copy Markdown
Contributor

Hi @ronensc, the pre-commit checks have failed. Please run:

uv pip install pre-commit>=4.5.1
pre-commit install
pre-commit run --all-files

Then, commit the changes and push to your branch.

For future commits, pre-commit will run automatically on changed files before each commit.

Tip

Is mypy failing?
mypy is run differently in CI. If the failure is related to this check, please use the following command to run it locally:
# For mypy (substitute "3.10" with the failing version if needed)
pre-commit run --hook-stage manual mypy-3.10

Signed-off-by: Ronen Schaffer <ronen.schaffer@ibm.com>
@orozery
orozery merged commit e175192 into vllm-project:main May 10, 2026
66 checks passed
@ronensc
ronensc deleted the req-context branch May 10, 2026 13:09
weifang231 pushed a commit to weifang231/eb-vllm that referenced this pull request May 13, 2026
…e_store() (vllm-project#41366)

Signed-off-by: Ronen Schaffer <ronen.schaffer@ibm.com>
mfylcek pushed a commit to mfylcek/vllm that referenced this pull request May 19, 2026
…e_store() (vllm-project#41366)

Signed-off-by: Ronen Schaffer <ronen.schaffer@ibm.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

kv-connector ready ONLY add when PR is ready to merge/full CI is needed v1

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants