Repository navigation
[Refactor] mem_cache: [11/N] extract KVCache and BaseSWAKVPool into pool/base.py - #35647
alphabetc1 wants to merge 4 commits into
Conversation
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: e6bbc4efc0
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
| from sglang.srt.mem_cache.pool.base import ( | ||
| unwrap_write_loc, | ||
| ) |
There was a problem hiding this comment.
Update the mocked module tree for the new pool import
When test/manual/minimax_m3/test_npu_memory_pool.py runs _load_npu_memory_pool_module(), it stubs sglang.srt.mem_cache as a plain ModuleType and provides only the old memory_pool.unwrap_write_loc. Executing this newly added import therefore raises ModuleNotFoundError: 'sglang.srt.mem_cache' is not a package before any of the standalone NPU pool tests run. The loader needs to stub sglang.srt.mem_cache.pool.base and its unwrap_write_loc symbol, or otherwise load the real package.
Useful? React with 👍 / 👎.
e6bbc4e to
2e551e7
Compare
…the write-loc helpers into mem_cache/pool/base.py
…mem_cache/pool/base.py
…write_loc on pool.base in the NPU memory pool manual test
c0bcea7 to
0dc4468
Compare
Motivation
Part of #25371. Mechanical Move. First PR of the
mem_cache/pool/layer: create the package and move the base layer every device pool derives from into it, so later family moves have a target.Modifications
Rebuilt on current
main. Four commits, each labeled by kind:add the mem_cache pool packagemem_cache/pool/__init__.pyextract KVCache and the write-loc helpersKVWriteLoc,_check_dense_loc,window_write_locs,layer_window_locs,unwrap_write_loc,write_loc_is_physical,KvBufferDesc,KVCache,GBfrommemory_pool.pyintopool/base.py; every importer repointedmove BaseSWAKVPoolBaseSWAKVPoolfrombase_swa_memory_pool.py(deleted) intopool/base.py; every importer repointedstub unwrap_write_loc on pool.basetest/manual/minimax_m3/test_npu_memory_pool.pyfakessglang.srt.mem_cache.pool.basefor the NPU pool's new importNo re-export shim: all importers (83 files) import from
pool.basedirectly, as thecheck-indirect-importshook requires.memory_pool.pyimports back the six names it still uses; ruff drops its now-unused imports (abc,KW_ONLY,IdSpace,PlainPoolRouter).Accuracy Test
Mechanical move, so the bar is byte-level equality:
mainby themechanical-refactor-verifyprimitives (cut each def/class verbatim, author only the new module's import header, repoint imports), then pre-commit and a byte diff against the commit. The chain verifier reportsPASSfor both proofs andHUMAN_REVIEWfor the two small non-mechanical commits above.mlx,torch_npu, MUSA flash-attn, XPU kernels) onmainand on this branch.pool.baseimports first without a cycle.test/registered/unit/mem_cache/,test/registered/unit/disaggregation/and every changed test file, same command onmainand on this branch: 5115 passed / 19 failed / 1 collection error on both, with identical failing test IDs (pre-existing onmain).Benchmark and Profiling Results
Not applicable -- no runtime behavior changes.
Checklist
🤖 Generated with Claude Code
CI States
Latest PR Test (Base): ✅ Run #43675877392
Latest PR Test (Extra): ❌ Run #43675877334
Latest PR Test (AMD ROCm 10): ❌ Run #43675877496
Tip
Base CI only covers the core cases, and its workload may not exercise your change. Use
/rerun-testto run the tests your change touches, or copy this prompt for an agent: