[kv_offload] Merge FilterReusedOffloadingManager and add lifecycle hook to OffloadingManager#41727
[kv_offload] Merge FilterReusedOffloadingManager and add lifecycle hook to OffloadingManager#41727hickeyma wants to merge 3 commits intovllm-project:mainfrom
FilterReusedOffloadingManager and add lifecycle hook to OffloadingManager#41727Conversation
…and add OffloadingManager.request_finished This commit implements Tasks 5 and 6 of vllm-project#33689: - Task 5: Move `reuse_manager.py` into `cpu/manager.py` - Task 6: Add `request_finished` lifecycle hook to `OffloadingManager` Signed-off-by: Martin Hickey <martin.hickey@ie.ibm.com>
![claude[bot]](https://avatars.githubusercontent.com/in/1236702?s=60&v=4){kind=small}
![gemini-code-assist[bot]](https://avatars.githubusercontent.com/in/956858?s=60&v=4){kind=small}
There was a problem hiding this comment.
Code Review
This pull request moves the FilterReusedOffloadingManager to vllm/v1/kv_offload/cpu/manager.py and introduces a request_finished hook to the OffloadingManager interface to track async work before freeing GPU blocks. The review feedback highlights a design limitation where the lack of req_id in the prepare_store method makes the new hook difficult to implement effectively. Additionally, there are suggestions to fix a garbled docstring and to ensure the shutdown method is properly delegated in the decorator class.
| self.events.clear() | ||
|
|
||
|
|
||
| class FilterReusedOffloadingManager(OffloadingManager): |
There was a problem hiding this comment.
I was actually thinking to remove FilterReusedOffloadingManager and instead copy its logic inside CPUOffloadingManager
There was a problem hiding this comment.
@orozery Do you want it to be done in this PR or in a separate PR?
Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com> Signed-off-by: Martin Hickey <martin.hickey@ie.ibm.com>
Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com> Signed-off-by: Martin Hickey <martin.hickey@ie.ibm.com>
2 participants
Purpose
This commit implements Tasks 5 and 6 of #33689:
reuse_manager.pyintocpu/manager.pyrequest_finishedlifecycle hook toOffloadingManagerTest Plan
CI passes
Tests pass
CI passes