Repository navigation

DeepSeek-V4.1-Flash: LMCache tier offload through the MP server (vLLM plugin) - #2598

Merged

valarLip merged 67 commits into

mainfrom

perzhang/dsv41-lmcache-031

Nov 9, 2026

Merged

DeepSeek-V4.1-Flash: LMCache tier offload through the MP server (vLLM plugin)#2598
valarLip merged 67 commits into
mainfrom
perzhang/dsv41-lmcache-031

Commits on Oct 26, 2026
- DeepSeek-V4.1-Flash: LMCache tier offload through the MP server (vLLM plugin)
  
  zgplvyou
  and
  claude
  committed
  f0610f5
- dsv41 offload: serve checkpoint-backed local hits; drop debug instrumentation
  
  zgplvyou
  and
  claude
  committed
  26f5c0c
- dsv41 offload: address PR review (preemption fence, tail saves, ride sources, slot ownership)
  
  zgplvyou
  and
  claude
  committed
  d10a698
- recipe(dsv41): HBM-then-tier results, DSpark LMCache on/off, theory vs measured
  
  zgplvyou
  and
  claude
  committed
  9c04c16
- test(dsv41): sort the decode-keep test's imports (ruff I001)
  
  zgplvyou
  and
  claude
  committed
  96927f7
- dsv41 offload: look up only the extent a load can end on (#2642)
  
  authored and
  zgplvyou
  committed
  95c687d
- style(dsv41): black after rebasing onto main
  
  zgplvyou
  and
  claude
  committed
  08d6753
- mp worker: release finished transfers on the step thread, while its stream is idle
  
  zgplvyou
  and
  claude
  committed
  807d20b
- dsv41: restore checkpoints when the cursor read is deferred
  
  zgplvyou
  and
  claude
  committed
  13bae6f
- dsv41 checkpoints: store decode keeps at the commit, from the build's rows
  
  zgplvyou
  and
  claude
  committed
  6fdb823
- mp worker: destroy IPC resources on the step thread once nobody else holds them
  
  zgplvyou
  and
  claude
  committed
  a7d323e
Commits on Oct 27, 2026
- Merge main 481c2bfd4 into the V4.1 LMCache offload branch
  
  zgplvyou
  and
  claude
  committed
  bece476
- test(lmcache-mp): skip the submit_request fallback test without its protocol module
  
  zgplvyou
  and
  claude
  committed
  0dcb360
- dsv41: keep the plugin's checkpoint restore and warmup exemption out of core
  
  zgplvyou
  and
  claude
  committed
  b845e89
Commits on Oct 28, 2026
- dsv41 offload: address review (vLLM 0.33 hit cap, save retirement, fence, pool sizing)
  
  zgplvyou
  and
  claude
  committed
  ac8bcbd
Commits on Oct 29, 2026
- dsv41 offload: address follow-up review (MP stalled saves, layout override, fence deadline)
  
  zgplvyou
  and
  claude
  committed
  a6ed491
- dsv41 offload: give the plugin test fixtures the connector's new fields
  
  zgplvyou
  and
  claude
  committed
  428fd67
- deepseek_v41: keep vLLM plugin rules out of the model validator
  
  zgplvyou
  and
  claude
  committed
  dd4a59a
- deepseek_v41: drop the plugin check from the model validator
  
  zgplvyou
  and
  claude
  committed
  52cbd1e
- dsv41 offload: keep vLLM's knowledge out of the shared offload code
  
  zgplvyou
  and
  claude
  committed
  65cd127
- docs: describe what the V4.1 plugin admits now
  
  zgplvyou
  and
  claude
  committed
  6e7ba5f
- dsv41 offload: address review round 9
  
  zgplvyou
  and
  claude
  committed
  fee2de2
- tests: let the index-plane parity test's QuantizationConfig stub take keywords
  
  zgplvyou
  and
  claude
  committed
  9b2cdf9
Commits on Oct 31, 2026
- Address round 10 review: V4.1 plugin guards and per-step lookup budget
  
  zgplvyou
  and
  claude
  committed
  4c07290
- plugin state checkpoints: one defaults helper and one runner wrapper
  
  zgplvyou
  and
  claude
  committed
  57f8757
- mp lookup: check the client once instead of adapting every request
  
  zgplvyou
  and
  claude
  committed
  69e2ebc
- mp worker: take one last look at pending loads in get_finished
  
  zgplvyou
  and
  claude
  committed
  7512caf
- mp worker: reuse step events instead of creating and destroying one per step
  
  zgplvyou
  and
  claude
  committed
  22021a6
- plugin V2 runner: read checkpoint block ids from host rows, not a table D2H
  
  zgplvyou
  and
  claude
  committed
  831f5c8
- V4.1 plugin: one owner for the prefill tail and one prefill-chunk clipper
  
  zgplvyou
  and
  claude
  committed
  71b69db
- V4.1 plugin: let V2 profile CUDA-graph memory instead of skipping by name
  
  zgplvyou
  and
  claude
  committed
  0763613
- mp worker: one native-state registration and block-id path for every worker
  
  zgplvyou
  and
  claude
  committed
  9cc401a
- V4.1 plugin: carry the state checkpoint as a native-state image
  
  zgplvyou
  and
  claude
  committed
  c4721c2
- V4.1 plugin: a finished request saves to its last checkpointed boundary
  
  zgplvyou
  and
  claude
  committed
  d8e827d
- V4.1 plugin: say which side enforces the tier hit grain
  
  zgplvyou
  and
  claude
  committed
  c4569ad
- vLLM plugin offload: one BoundaryPlanner base for the KDA and V4.1 planners
  
  zgplvyou
  and
  claude
  committed
  8eaf726
- vLLM plugin offload: drive every boundary planner through one loop per hook
  
  zgplvyou
  and
  claude
  committed
  69fbd74
- vLLM plugin offload: build each planner next to its class, and refuse two
  
  zgplvyou
  and
  claude
  committed
  9da531f
- recipe: DeepSeek-V4.1 nvfp4 does not start on vLLM 0.31.1
  
  zgplvyou
  and
  claude
  committed
  de2c4dd
- Merge origin/main into the DeepSeek-V4.1 LMCache offload branch
  
  zgplvyou
  and
  claude
  committed
  ee65d81
- plugin V2 host rows: replace a resumed row, arm only when kept; CPU-safe test
  
  zgplvyou
  and
  claude
  committed
  414de75
- tests: use list for the empty runner-class stub (ruff PIE807)
  
  zgplvyou
  and
  claude
  committed
  65e2907
- mp worker: record the step event only for a step that reaches the server
  
  zgplvyou
  and
  claude
  committed
  105730d
Commits on Nov 1, 2026
- V4.1 plugin: store decode keeps at the next build, before it stages
  
  zgplvyou
  and
  claude
  committed
  0687f4b
- plugin V2 host rows: keep them only on runners that read them
  
  zgplvyou
  and
  claude
  committed
  6da7026
- test: give the proxy factory contract the decode-keep store site
  
  zgplvyou
  and
  claude
  committed
  3587050
- V4.1 plugin: build the step from the host where the host is exact
  
  zgplvyou
  and
  claude
  committed
  7cb4bf9
- V4.1 plugin: defer the cursor read only while serving synchronously
  
  zgplvyou
  and
  claude
  committed
  f5d9788
Commits on Nov 2, 2026
- offload: address the core review of 11-02
  
  zgplvyou
  and
  claude
  committed
  b6f6685
- offload: report native saves that land during the forward that step
  
  zgplvyou
  and
  claude
  committed
  53e32bd
- V4.1: build the pool geometry beside the model config
  
  zgplvyou
  and
  claude
  committed
  413df1b
- offload: run native ATOM's MP transfers on the step thread again
  
  zgplvyou
  and
  claude
  committed
  3593a34
Commits on Nov 3, 2026
- offload: address the second pass of the 11-02 core review
  
  zgplvyou
  and
  claude
  committed
  7081137
- Merge main 498133ed9 into the V4.1 LMCache offload branch
  
  zgplvyou
  and
  claude
  committed
  9e70cbc
- offload: native ATOM drops finished MP futures as main does
  
  zgplvyou
  and
  claude
  committed
  97c8dde
- offload: keep native V4.1's LMCache namespace as main computes it
  
  zgplvyou
  and
  claude
  committed
  85d55d8
- offload: one MP transfer thread for both frontends, and scheduler fixes
  
  zgplvyou
  and
  claude
  committed
  0132de1
- offload: drop handed-back MP futures while the stream is idle
  
  zgplvyou
  and
  claude
  committed
  4b589d8
Commits on Nov 4, 2026
- offload: the plugin's polling worker as a constructor flag, and review fixes
  
  zgplvyou
  and
  claude
  committed
  45355d2
- offload: swap the transfer thread's error under its lock
  
  zgplvyou
  and
  claude
  committed
  a278334
Commits on Nov 5, 2026
- offload: native MP paths as main, one floor record, and the 11-05 review
  
  zgplvyou
  and
  claude
  committed
  6a08af2
- tests: wire two plugin tests to the plugin's dense scheduler
  
  zgplvyou
  and
  claude
  committed
  d7619fe
Commits on Nov 6, 2026
- offload: settle the polling thread's shutdown, failure and stalls
  
  zgplvyou
  and
  claude
  committed
  ba157dd
- offload: deadlines from the forward thread, and the 11-06 review
  
  zgplvyou
  and
  claude
  committed
  ce7dd45
- offload: never fail-stop a finished transfer from the forward thread
  
  zgplvyou
  and
  claude
  committed
  4b0dc75
Commits on Nov 9, 2026
- offload: remove the forward thread's deadline check
  
  zgplvyou
  and
  claude
  committed
  ce1187d
- Merge origin/main into perzhang/dsv41-lmcache-031
  
  zgplvyou
  and
  claude
  committed
  f259bbb

Commits

Commits on Oct 26, 2026

Commits on Oct 27, 2026

Commits on Oct 28, 2026

Commits on Oct 29, 2026

Commits on Oct 31, 2026

Commits on Nov 1, 2026

Commits on Nov 2, 2026

Commits on Nov 3, 2026

Commits on Nov 4, 2026

Commits on Nov 5, 2026

Commits on Nov 6, 2026

Commits on Nov 9, 2026