Skip to content

fix(mooncake): defer cache publication for remote loads and concurrent puts - #232

Open
XingLiu1 wants to merge 4 commits into
mainfrom
liuxing/mooncake-remote2h-deferred-insert
Open

XingLiu1 wants to merge 4 commits into
mainfrom
liuxing/mooncake-remote2h-deferred-insert

Conversation

@XingLiu1

@XingLiu1 XingLiu1 commented Jul 23, 2026 •

Copy link
Copy Markdown
Collaborator

Summary

  • propagate per-block Mooncake transfer results through workers, schedulers, multi-node handles, and batched callbacks
  • defer CPU/SSD radix publication for REMOTE2H until completion, publishing only the longest contiguous successfully loaded prefix
  • keep Mooncake PUT allocations detached until graph completion, then fresh-rematch, reuse any concurrently published prefix, insert only the missing suffix, and mark it ready in the same tree transaction
  • use only contiguous ready local blocks as H2DISK/H2REMOTE sources, pin existing CPU/SSD anchors during transfer, and allow a full CPU hit to retry a missing Mooncake upload without another D2H
  • reject stale radix insertion anchors before mutation so duplicate or overlapping writers cannot replace an existing child and orphan physical blocks

Why

The previous flow published an unready radix node before transfer and marked it ready afterward. A partial Mooncake load failure could therefore expose blocks that were never loaded successfully.

Moving PUT insertion into the completion callback fixes that failure mode, but a callback must not reuse the match captured during planning. Two duplicate or overlapping PUTs can both reserve blocks before either finishes; if the later callback inserts with that stale match, it can replace an existing radix child and leak the displaced allocation.

Publication is now serialized by the cache-tree lock. Each callback rematches the local tree, recycles duplicate staging, handles concurrent splits, inserts only the unmatched suffix, and marks it ready before releasing the lock. The radix layer also rejects stale anchors before any split or child mutation, leaving block ownership with the caller.

User impact

  • partial Mooncake loads no longer publish invalid cache entries
  • concurrent duplicate PUTs converge on one owner without block leaks
  • overlapping PUTs preserve shared prefixes and both branches after a split
  • unready local blocks are never used as Mooncake upload sources
  • CPU/SSD staging, SWA slots, anchor locks, and P2P publication follow the final committed node

Tests

  • targeted CUDA/c_ext-stubbed unit tests: 45 passed
  • covered duplicate PUTs in both completion orders, overlapping splits, CPU+SSD fresh-rematch and loser recycling, ready-only source selection, CPU full-hit remote retry, SWA duplicate cleanup, and single P2P publication
  • Python compileall: passed
  • Ruff checks for the new/modified tests and non-baseline source rules: passed
  • C++ release-mode syntax checks for base, local, and distributed radix trees: passed
  • git diff --check: passed

Full native runtime integration was not available on the local macOS host because libcudart.so.11, the compiled flexkv.c_ext, and the Mooncake SDK are unavailable.

@XingLiu1 XingLiu1 changed the title fix(mooncake): publish only successful remote load prefixes fix(mooncake): defer cache publication for remote loads and concurrent puts Jul 23, 2026
@XingLiu1
XingLiu1 changed the base branch from merge/dpsk4-mooncake-store to main July 24, 2026 02:15
@XingLiu1
XingLiu1 force-pushed the liuxing/mooncake-remote2h-deferred-insert branch from c687ca4 to e606102 Compare July 24, 2026 02:27
@XingLiu1
XingLiu1 marked this pull request as ready for review July 29, 2026 13:18
XingLiu1 added 2 commits July 30, 2026 10:27
…te2h-deferred-insert

# Conflicts:
#	flexkv/cache/cache_engine.py
#	flexkv/common/transfer.py
#	flexkv/kvtask.py
#	flexkv/transfer/transfer_engine.py
#	flexkv/transfer/worker.py
#	flexkv/transfer_manager.py
…te2h-deferred-insert

# Conflicts:
#	flexkv/kvtask.py
#	flexkv/transfer/worker.py
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant