Skip to content

Commit 1bb3393

Browse files
Jasen2201claude
andcommitted
refactor(offload): let the shared scheduler name where a load starts
main's #2332 made ChunkedOffloadSchedulerBase set seq.offload_load_start_tokens = hbm right before it creates each load operation, so every chunked offload scheduler publishes a restored prefix into the HBM prefix index. mooncake_store's _new_load_operation override set the same value (seq.num_cached_tokens, which is that hbm) and is now redundant. The dense-scheduler test no longer asserts that the attribute is absent, since main sets it there on purpose. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
1 parent 52c3f02 commit 1bb3393

2 files changed

Lines changed: 0 additions & 14 deletions

File tree

‎atom/kv_transfer/offload/mooncake_store/scheduler.py‎

Lines changed: 0 additions & 13 deletions
Original file line numberDiff line numberDiff line change
@@ -481,19 +481,6 @@ def _build_load_request(
481481
chunk_hashes=hashes[: (transfer_end // self.chunk_size) * DIGEST_BYTES],
482482
)
483483

484-
def _new_load_operation(self, seq: Any) -> LoadOperationId:
485-
operation = super()._new_load_operation(seq)
486-
# The worker restores KV straight into allocated blocks, which never
487-
# pass through `hash_blocks()`. Naming where the load starts lets
488-
# `Scheduler._mark_offload_load_ready` publish the loaded prefix into
489-
# the HBM prefix index once the load succeeds, so a later turn hits it
490-
# in HBM instead of loading it again; without it the suffix prefill
491-
# finds its parent block unhashed and cannot register its own blocks
492-
# either. Loads are dispatched only from a chunk-aligned post-allocate
493-
# HBM frontier, which is always a hash-block boundary.
494-
seq.offload_load_start_tokens = int(seq.num_cached_tokens)
495-
return operation
496-
497484
def save_abandon_timeout_s(self) -> float:
498485
"""Seconds a deferred save may sit before the engine reclaims it.
499486

‎tests/test_mooncake_store_offload.py‎

Lines changed: 0 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -1740,7 +1740,6 @@ def test_dense_scheduler_load_request_is_unchanged(monkeypatch):
17401740
assert request.token_ids == list(range(32))
17411741
assert request.chunk_hashes is None
17421742
assert request.block_ids == seq.block_table
1743-
assert not hasattr(seq, "offload_load_start_tokens")
17441743

17451744

17461745
def test_scheduler_settings(cluster):

0 commit comments

Comments
 (0)