Repository navigation
[KV Offload] Pass ReqContext to touch(), complete_load(), and complete_store() - #41366
Conversation
There was a problem hiding this comment.
Code Review
This pull request updates the KV offloading interface and its implementations to include a ReqContext parameter in the touch, complete_load, and complete_store methods. These changes ensure that per-request context is consistently passed through the offloading manager, with corresponding updates applied to the scheduler and test suite. I have no feedback to provide as there are no review comments to evaluate.
|
Hi @ronensc, the pre-commit checks have failed. Please run: uv pip install pre-commit>=4.5.1
pre-commit install
pre-commit run --all-filesThen, commit the changes and push to your branch. For future commits, Tip Is
|
|
Hi @ronensc, the pre-commit checks have failed. Please run: uv pip install pre-commit>=4.5.1
pre-commit install
pre-commit run --all-filesThen, commit the changes and push to your branch. For future commits, Tip Is
|
|
Hi @ronensc, the pre-commit checks have failed. Please run: uv pip install pre-commit>=4.5.1
pre-commit install
pre-commit run --all-filesThen, commit the changes and push to your branch. For future commits, Tip Is
|
6059286 to
1135332
Compare
e76581f to
22406d0
Compare
|
Hi @ronensc, the pre-commit checks have failed. Please run: uv pip install pre-commit>=4.5.1
pre-commit install
pre-commit run --all-filesThen, commit the changes and push to your branch. For future commits, Tip Is
|
Signed-off-by: Ronen Schaffer <ronen.schaffer@ibm.com>
…e_store() (vllm-project#41366) Signed-off-by: Ronen Schaffer <ronen.schaffer@ibm.com>
…e_store() (vllm-project#41366) Signed-off-by: Ronen Schaffer <ronen.schaffer@ibm.com>
Purpose
Follow up on #39185
OffloadingManageralready acceptsreq_context: ReqContexton the three "initiation" methods (lookup,prepare_load,prepare_store) but not on the three "completion/maintenance" methods (touch,complete_load,complete_store).This PR closes that gap so the full interface is consistent and future manager implementations can act on per-request context during completion callbacks.
Test Plan
Test Result
Essential Elements of an Effective PR Description Checklist
supported_models.mdandexamplesfor a new model.