Skip to content

[Fix] hicache: Order write-back host copies after the forward that produced the KV - #44800

Merged
hanming-lu merged 2 commits into
mainfrom
oss-sync-015db9f41c
Oct 17, 2026
Merged

hanming-lu merged 2 commits into
mainfrom
oss-sync-015db9f41c

Conversation

@metamergebot

@metamergebot metamergebot commented Oct 16, 2026 •

Copy link
Copy Markdown
Collaborator

Motivation

With --hicache-write-policy write_back, KV pages could be copied to host before the forward pass that produced them had finished. The host copy then held whatever the page contained at submit time, and a later load-back served that stale KV to another request. Two paths reached the device-to-host copy without any dependency on the forward stream:

  • Insert-time SWA window backups. Write-back defers Full and Mamba backups to eviction, but the tree core still published an unbacked SWA window when a request's nodes were inserted. That backup was queued while the request's own forward could still be writing the window.
  • The blocking write-back drain. Eviction submits everything queued in the tick on the device-to-host stream with no fence, and an evicted SWA page can belong to a checkpoint whose producing forward has not completed.

Modifications

  • UnifiedTreeCore (Python) and unified_tree_core.rs (Rust): under write-back, insertion never returns a BackupKV action, so SWA windows are no longer published at insert time. The incremental-component check no longer special-cases SWA under write-back.
  • HiCacheController.start_writing(wait_for_forward=...): a new keyword-only flag. When set and a write_fence_stream is configured, the device-to-host stream waits on it before the copies are submitted. The scheduler sets write_fence_stream to the model runner's forward stream next to the existing load_fence_stream.
  • UnifiedRadixCache.writing_check(write_back=True) passes wait_for_forward=True on the blocking drain. Write-through and the asynchronous write-back flush are unchanged, and the default keeps every other caller as it was.
  • Tests:
    • test_hicache_write_back_ordering.py (new): a checkpointed request whose forward is still writing on a separate stream; evicting an older SWA window of that request must copy the values the forward wrote, not the pre-forward contents.
    • test_unified_hicache_regressions.py: the blocking drain waits for the fence stream before copying both a tick's queued backup and the eviction victim.
    • test_unified_radix_cache_unittest.py and the Rust crate tests: write-back insertion of backed Full nodes publishes no SWA window and leaves no pending ack; the Mamba pending-ack eviction test now queues its window backup explicitly instead of relying on insert-time publication.

Accuracy Tests

  • cargo test for rust/sglang-radix-tree (--no-default-features, as pr-test-rust-workspace.yml runs it): 931 passed, 0 failed.
  • On one CUDA GPU, with both SGLANG_UNIFIED_RADIX_TREE_CORE_BACKEND=rust and =python: test_hicache_write_back_ordering.py, test_unified_hicache_regressions.py::TestTransferStreamOrdering, and the two touched cases in test_unified_radix_cache_unittest.py pass.
  • Without the runtime changes, test_hicache_write_back_ordering.py fails on both backends: the host copy holds the pre-forward contents instead of what the forward wrote.
  • No numerics change: the fix only orders existing copies after their producer and removes an early backup that eviction performs later.

Speed Tests and Profiling

The new wait is a single wait_stream on the blocking write-back drain, which already blocks on every ack; the asynchronous paths do not take it.

Checklist


CI States

Latest PR Test (Base): 🚫 Run #38999355934
Latest PR Test (Extra): ❌ Run #38999355753
Latest PR Test (AMD ROCm 10): 🚫 Run #38999355752

@metamergebot metamergebot added the run-ci CI: run the baseline test suite on this PR label Oct 16, 2026
@github-actions github-actions Bot added hicache Hierarchical Caching for SGLang unified-radix-cache labels Oct 16, 2026
@github-actions

This comment has been minimized.

@metamergebot metamergebot changed the title [HiCache] Order write-back host copies after the forward that produced the KV [Fix] hicache: Order write-back host copies after the forward that produced the KV Oct 16, 2026
@metamergebot metamergebot added the bypass-fail-fast CI: a failing job no longer aborts its siblings (lint still gates) label Oct 16, 2026
@metamergebot

Copy link
Copy Markdown
Collaborator Author

/rerun-test test/registered/unit/mem_cache/test_unified_post_capture_sizing.py

@github-actions

github-actions Bot commented Oct 17, 2026 •

Copy link
Copy Markdown
Contributor

🚀 1-gpu-5090 (1 test): ❌ View workflow run

cd test/ && python3 registered/unit/mem_cache/test_unified_post_capture_sizing.py

metamergebot and others added 2 commits October 16, 2026 20:28
…d the KV

Under `--hicache-write-policy write_back`, two paths copied KV pages to
host before the forward that wrote them had finished, so a later load-back
could serve stale KV to another request:

- Insert-time SWA window backups. Write-back defers Full and Mamba
  backups to eviction, but the tree still published an unbacked SWA
  window when a request's nodes were inserted. That copy was queued while
  the request's own forward could still be writing the window.
- The blocking write-back drain. Eviction submits everything queued in
  the tick on the device-to-host stream with no dependency on the
  forward stream, and an evicted SWA page can belong to a checkpoint
  whose producer has not completed.

Stop publishing SWA windows at insert time under write-back, in both the
Python and the Rust tree cores, and give the blocking drain a dependency
on the scheduler's forward stream, the same stream the load-back H2D path
already fences on (`load_fence_stream`). Only a non-empty blocking drain
waits; write-through and the asynchronous write-back flush are unchanged.

Co-authored-by: xiezhq-hermann <xiezhq-hermann@users.noreply.github.com>
…backups

Write-back insertion no longer publishes the SWA window backup; it is
deferred to eviction. These four tests covered the removed behavior.
@metamergebot

Copy link
Copy Markdown
Collaborator Author

This port clears our bar on 3a845c1339303d645f51035348aab2db2eb4e013: every base-a and base-b job is green.
base-c and the accelerator suites are reported but not gated, because they run
on scarce hardware and upstream merges through them routinely.

gate 23✓✗1 lint✓ build 7✓✗1✂3 A✓ B✓ C 0/15⏳ other 8✓✗1 · accel ✗amd,npu,xpu

@fei-xx @merrymercy @xiezhq-hermann — could one of you take a look and merge it if you are happy? Nothing else is waiting on our side.

@hanming-lu
hanming-lu merged commit f75232c into main Oct 17, 2026
160 of 193 checks passed
@hanming-lu
hanming-lu deleted the oss-sync-015db9f41c branch October 17, 2026 08:31
detain pushed a commit to detain/sglang that referenced this pull request Oct 19, 2026
…oduced the KV (sgl-project#44800)

Co-authored-by: metamergebot <metamergebot@users.noreply.github.com>
Co-authored-by: xiezhq-hermann <xiezhq-hermann@users.noreply.github.com>
(cherry picked from commit f75232c)
(cherry picked from commit cb8cbe7902c86f7f23c66fd0500331b07ab6732d)
detain added a commit to detain/sglang that referenced this pull request Oct 19, 2026
… test on a visible device

`test_queued_full_backed_parent_owns_its_ack_ahead_of_a_queued_child` arrived with
7e9dda1 (commit in 323c1c07eb) without a device guard, so it added ONE new
failure node to test_unified_radix_cache_unittest.py (1300F -> 1301F at CVD=9):
`_build()` creates a real cache/host pool, which raises
`RuntimeError: No accelerator ... or platform plugin is available`.

Uses the file's own in-class convention — its 4 fixture-based siblings already
carry `@unittest.skipUnless(torch.cuda.is_available(), "cache fixtures need CUDA")`
(added in stack entry 199's test fixes). Node-list check of the whole file after
the gate: identical to the base at 1275 unique FAILED nodes, with the 2 expected
renames from sgl-project#44800 (`test_write_back_swa_publish_leaves_an_ancestor_pending_under_another_ack_alone`
-> `test_write_back_defers_swa_window_backup_until_eviction`, x2 parametrizations).

(cherry picked from commit f2e627253a604823a56839aeb76b3877ae63d8ec)
jumanzii pushed a commit to fractalyze/sglang that referenced this pull request Oct 19, 2026
Resolve against sgl-project#44800: start_writing fences every device-to-host copy on write_fence_stream, so the wait_for_forward flag and the separate load_fence_stream wait are no longer needed.
detain added a commit to detain/sglang that referenced this pull request Oct 29, 2026
…20 commits since 12dc623 @ merge-sgl-project#4)

Conflicts: 167 files (149 content / 14 add/add / 4 modify/delete) at the settled
309 tip d28e7cd -- dossier predicted 146 +/-25; the +21 shift is entries
275-309 landing qsa-ring/hicache-ctrl/gdn/moe/llguard deltas into the conflict
zone. By class: A 13 (auto-takes-upstream, dedupe-verified), B 80, C 56, D 18.
102 TODO(merge5) re-plant/fold markers planted: B1 x15, B2 x29, B3 x12, B4 x23,
B5 x23. Re-plant stock: 117 files under /tmp/opencode/merge5-dossier/replant/
(76 fork-hunks patches, 19 upstream-side patches, 14 AA fork copies, 8
preremove snapshots) -- sessions 2/3 re-plant from real code, not memory.

D-list outcomes (must-not-auto-resolve, handled personally):
- sgl-project#45289-vs-sgl-project#44858 (entry 301): VERIFIED, not re-resolved -- the gate/up-order
  final form is in origin/main (5b1dbc2 in range); scales test taken as
  upstream oracle, fork 191-line version saved to replant/ for B5 diffing.
  sgl-project#44858's call-site shape CANNOT survive (our base never had it).
- entry-230 dispatcher vs sgl-project#44993/sgl-project#44994: took upstream structure in
  token_dispatcher/standard.py; published-value CONTRACT lives upstream via
  get_moe_runner_backend() in the LoRA-EP guard; fork's warn-once mismatch
  diagnostic re-plants (B5 marker). UNION GATE: standard_dispatcher 16P (was
  12P, suite grew), model_overrides 121P/1S/210sub (was 112P/1S/181sub),
  fold probe 6P/2F (upstream added a runner_backend arg to the predicate --
  B5 re-pins the probe).
- entry-219 gauge names vs sgl-project#43727 return + sgl-project#44968: PER-POOL NAMES KEPT as the
  serving schema -- hicache_host_pool_{used,total}_tokens{pool} dataclass
  fields, Gauge registrations and labels(pool=) emission re-restored over the
  #4697x structure; hicache_host_pool_evicted_tokens_total already survived
  auto-merge; producer dict restored by union in metrics_reporter. ANTI-NEEDLE
  SATISFIED: old mamba-only gauges are NOT the serving schema.
- entry-221 eager-direct add/add: DEDUPED to upstream (bodies identical except
  est_time 15->9); MEASUREMENT DEBT RECORDED: prefill t/s must be re-measured
  post-B4 before trusting the est_time.
- entry-168 D2H fence vs sgl-project#44800: the fence is ALREADY UPSTREAM (both
  load_fence_stream + write_fence_stream with the load-thread-corruption
  comment in origin/main's cache_controller.py) -- nothing re-planted; the two
  real regions resolved as import-union + comment wording (merge sgl-project#4 stale-tail
  pattern avoided). Fork hunks still saved to replant/ (belt).
- entry-209 clamp + 216/218 ordering + 288-295 hicache-ctrl-era vs #46970x KV
  rewrite: ADOPTED-UPSTREAM across unified_radix_cache/hicache_storage/
  hybrid_cache_controller/memory_pool/kv_cache_builder; EVERY fork hunk
  extracted to replant/*.fork-hunks.patch; B3 re-plant markers planted.
  sgl-project#45703's short-full-match seam arrives with upstream.
- sgl-project#45090 SGLANG_ENABLE_UNIFIED_RADIX_TREE: REMOVAL TAKEN (fork had already
  moved it to _DeprecatedEnv; now gone). grep: ZERO .py readers repo-wide;
  2 upstream-owned docs jsx snippets still export it (deepseek-v4, glm-5.2 --
  upstream's own debt). OPERATOR-OWED CHECK: confirm no /root harness exports
  it (unreadable from this seat).
- sgl-project#39727 unified L3 key: VERIFIED OPT-IN -- merged hicache_storage.py keeps
  the DEFAULT legacy suffix composition _<model>_<tp_rank>_<tp_size>
  (content-anchor: HiCacheFile._init_config_suffix, raise-guarded when
  _unified_chunk_spans is set). ANTI-NEEDLE: no silent key-scheme switch;
  warm /root/hicache_store contents stay addressable.

Modify/delete (4, all victims of upstream sgl-project#45672 test prune + URC split):
- test_unified_radix_cache_unittest.py: REMOVED (adopted split) -- its bare
  unified_tree_core_inspector imports are unreachable post-sgl-project#45236 re-home;
  fork source + hunks saved; B2 marker planted in the new core test.
- test_fused_moe_triton_config.py: REMOVED -- imports common_utils, absent
  from both sides post-sgl-project#45672; the 2 live entry-3/235 tests re-plant in B2.
- test_scheduler_batch_is_full_latch.py: REMOVED -- fork delta was 3 stub-attr
  lines documenting sgl-project#46643's absorbed arms; upstream deletion correct.
- test_qsa_kv_pool.py: KEPT -- fork's 147 added lines are entry-187/200
  sharding + 285-287 QSA-ring tests with NO upstream replacement;
  qsa_kv_pool.py sides converged byte-identical; collects clean. B2 marker.

Cascading silent-auto-merge repairs (classes not in the conflict set but
broken by auto-merge + union splices -- all found by gates, now fixed):
kernels/ops/activation/__init__.py (dead _CUDA registration dup), kernels/ops/
diffusion/__init__.py (5-tuple _SPECS entry), kernels ops embeddings/moe/
attention/mamba __init__ (KernelSpec fields removed upstream: description/
format_signature stripped, CapabilityRequirement->CUDA_ONLY), base_runner/
eager_runner cp_extend + shared_read_event imports retargeted per upstream's
"no imports in runner_utils/__init__" contract, and 4 fork helpers grafted
back onto their take-upstream homes because merged code/tests still import
them (memory_pool.spec_intermediate_bytes_per_req, kv_cache_builder.
_adjust_hicache_size_for_draft + _device_pool_size_per_token,
hybrid_cache_controller.local_restorable_mask, validation_hook.
validate_replayssm_spec_algorithm, arg_groups/moe_hook b12x validator).
flashinfer_cutlass.py flipped to FORK-OWNED (b12x route is fork-only, 760 vs
95 lines; upstream sgl-project#44993 hunks = B5 fold patch saved).

Silent-semantics notes: sgl-project#46127/sgl-project#44912 riders arrived via per-PR picks (union);
sgl-project#45520 (entry 302) is STILL OPEN upstream -- entry-230 fail-safe layer stays
as entry 303 records; if a future sync brings it, re-apply the fail-safe reads.
qwen4_exp holder-contract fused (ctor takes optional batch).

Gates at commit: import sglang OK (CVD=9, prod venv); check_registered_tests/
check_no_bare_pytest_main/check_no_registered_tests_in_package rc=0; ruff F821
+E9+format clean on the resolution set; collect-only over unit/{mem_cache,
model_executor,distributed,layers/moe} = 4337 nodes, 3 ERRORS: 2x cargo/torch-
sys libtorch probe (REPRODUCED IDENTICALLY AT THE PRE-MERGE TIP -- env issue,
not merge damage) + test_hicache_async_ready_sync.py (entry-253-era attribute
gone with the URC rewrite -- EXPECTED pre-B3).

sessions 2/3 owed: B2 test re-home, B3 mem_cache+rust re-plant (+hicache-ctrl
re-ports), B4 scheduler/admission, B5 moe/quant/dispatcher, B6 environ+C0a+
CAVEATS re-pin. PROD BOOT FORBIDDEN until B3 completes (209 clamp + 218
ordering re-plant).
detain added a commit to detain/sglang that referenced this pull request Oct 29, 2026
…),199,217,218,219,253,290d,293,294

Re-plants the fork-hunks patch (1013 lines, the deepest file) onto the
#46970x-rewritten URC. B3 marker consumed in unified_radix_cache.py.

What the merged tree had ALREADY absorbed (deduped, not duplicated — per-hunk
verdicts recorded): the whole deadline/stop-policy family (_StoragePrefetchWait
msgspec/PollTimer, _start_storage_prefetch_wait, _prefetch_deadline_expired
REPLACING _prefetch_timeout_check_linear_func, _mark_prefetch_deadline_expired,
the reason="deadline" bookkeeping, belief-heal suppression on deadline, the
_invalidate_absent_from_hit_query move to the drain head + KV-verdict guard +
deadline gate, the release_aborted_request / finish_storage_prefetch_admission
wait pops, and the _should_backup_after_insert write-back half of sgl-project#44800
(unified_tree_core.py:1447). The sgl-project#45703 short-full-match seam did NOT
obsolete entry 209's clamp (it lives controller-side; B3-2 gate 17P).

Re-planted here:
- entry 294 (sgl-project#45795): the FULL deferred ready-count sync family
  (_deferred_ready_sync_group gate ladder incl. the runtime-attach fallback,
  _check_hicache_events_deferred one-op-per-step on its own gloo group,
  _consume_deferred_ready_sync with the digest MIN/-MAX assert,
  _deferred_consume_count, _drop_deferred_ready_sync) + the check_hicache_events
  branch + the reset() settle. This CLEARS the test_hicache_async_ready_sync
  collection error (the B3-caused 1err in the merge gates).
- entries 199+218: the batched write-through backup family VERBATIM from the
  patch (_NodeBackupSpec, _queue_write_through_backup, _drain_queued_backups,
  _ancestors_first incl. the sgl-project#40363 delta walked-set, _is_unbacked_node,
  _parent_backed_up, _release_queued_backup, _backup_queued_batch,
  _build_node_backup_spec, _backup_specs, _merge_backup_transfers, the
  node_ids= ack-list through _execute_kv_backup into the controller write
  (B3-2), the flush_pending_backups guard, and the writing_check(write_back)
  drain + start_writing(wait_for_forward=True) — the ONE-fence ordering rule).
  _batched_backup reads SGLANG_ENABLE_HICACHE_BATCHED_BACKUP, DEFAULT FALSE
  (upstream's 0ab8654 default-ON stays NOT taken, entry-218's record).
- entry 293 (sgl-project#46451): the opt-in host-capacity WAIT: fields + init_hicache
  reads with the nonnegative-finite ValueError, wait_full, the retry cadence
  (host_capacity_deadline/next_retry/retries — fields planted on the operation
  in B3-2), allow_partial=not wait_full, pending_storage_hits park/drain with
  identity snapshots, the info.operation-is-not-operation late-query guards,
  and the revoke/alloc-already-owned guards. DEFAULT 0.0 = byte-identical.
- entry 290(d): _alloc_prefetch_aux_staging now sizes to alloc_len, the
  endpoint the KV span ACTUALLY got (mamba-host-full prod boots stop declining
  prefetches a smaller endpoint would fit).
- entry 219 producer: _record_host_pool_evictions on evict_host (before
  _apply_eviction_result) feeding hicache_host_pool_evicted_tokens_total{pool}.
- entry 292 (sgl-project#43110): _build_sidecar_transfers resolves derived pools through
  merge_source_transfers (every destination, not matching_sources[0]; the two
  AssertionErrors it subsumes are structurally guaranteed by the resolver).
- entry 30 rider: load_back unwinds (drop both locks, report miss) when any
  component PREPARED-REJECTS (the sgl-project#39830 poison/degrade contract the mamba
  component sets -- component side rides B3-5's mamba.py cluster).
- entry 253: the per-call-site file_storage_path fold DELETED here (it moved
  into _generate_storage_config, every attach path, B3-1-verified).
- test_chunked_publish_release_disjointness.py (entry 307's tripwire):
  re-anchored CacheInitParams(tree_components=...) -> tree_slots=tree_slots_of(...)
  per the upstream slots refactor; the tree_core_backend='rust' SUBFAILED line
  is the documented cargo/torch-sys ENV class (fails identically at base).

ACTUALS (CVD=9, prod venv, PYTHONPATH=python, per-file, damage -> after):
  test_hicache_async_ready_sync          1err -> 7P/0F   ** B3-caused error CLEARED **
  test_hybrid_storage_sparse_allocation  12P/1F -> 13P/0F/14sub  ** entry-290(d) closed **
  test_storage_prefetch_lifecycle                -> 16P/0F/13sub
  test_hicache_trailing_endpoint (gate)          -> 17P/0F (unmoved by this cluster; controller-side B3-2)
  test_buffer_mode_sidecar                        -> 6P/0F/6sub
  test_chunked_publish_release_disjointness 4P/2F -> 4P+1sub/1F (the F now the rust-cargo ENV class; the python-backend half RE-ANCHORED green)
  test_mamba_path_state_cap                      -> 5P/5S (skips = honest, env)
  test_decode_retraction_backup             2P/2F -> 2P/2F (both F = No-CUDA ENV class, byte-identical at base; file is B2's per the resolution log)
  test_hicache_mamba_sizing                      -> 30P/0F/17sub (unmoved)
  URC topic dir (unified_radix_cache/*.py): 1056F/40P/199S -> 1056F/40P/199S
    UNMOVED — 100% of the errors are 'No accelerator ... available' from the
    upstream kits' fixtures.py build_fixture calling get_device() (entry 310's
    "GREEN, honest skips" claim does NOT reproduce at this seat; the battery's
    own per-file numbers (core 27P/465F etc.) match the damage totals exactly;
    this is a kit no-CUDA skip DEFECT = B2/kit scope, recorded not fixed here).
Lint: ruff format + F821/E9 clean on both files; py_compile + import OK.
No GPU executed; no push.
detain added a commit to detain/sglang that referenced this pull request Oct 30, 2026
…en at upstream-new InsertParams site (B3-7 rust re-plant fallout, first-compiling seat)

The operator's cargo/maturin-style release build at /home/my/sglang/sglang was
the FIRST build in the merge era to get past torch-sys, and it turned entry
311 / B3-7's "compilation = ENV DEBT" (the CAVEATS + entry-310 unverified-cargo
marker) into three hard, reproducible compile errors. Reproduced at this seat
BEFORE any edit (same 3, same lines):

  error[E0599] no method named `parent_node_id` found for struct
               `TreeCoreBinding<Core>`  -> src/python_bindings.rs:3253  (x2)
  error[E0063] missing field `mamba_value_seqlen` in initializer of
               `InsertParams<'_, _>`    -> src/unified_tree_core.rs:2105

ROOT CAUSE 1 -- parent_node_id: WRONG CFG BLOCK, not a missing method.
The core side is fine and correct (unified_tree_core.rs:4283: arena.resolve ->
node.try_parent() -> parent's external id, Ok(None) on a root -- matching the
interface doc "The parent node id, or None for the root" and its pure-python
twin, and deliberately using try_parent rather than the panicking parent()).
What is wrong is WHERE the binding-side delegate was replanted: B3-7 put
TreeCoreBinding::parent_node_id inside the `#[cfg(feature = "inspection")]`
impl (python_bindings.rs:2314-2324) while the pyclass wrapper that exposes it
to Python (3252, emitted by tree_core_binding!) is UNCONDITIONAL. History
settles which side is authoritative:
  * `git log -S parent_node_id -- rust/sglang-radix-tree` shows the method is
    FORK-ONLY -- it does NOT exist at the MERGE sgl-project#5 base (ae6ee52) at all, so
    there was no upstream rename to track and nothing to "take from a snapshot".
  * The fork's own pre-merge copy (2f31a7e, entry-199 era) had it in the
    ALWAYS-ON impl (nearest impl header at line 1030; the two cfg(inspection)
    markers are far above at 328/482).
  * The shipped extension (python/.../rust_tree_core/mem_cache...so, Oct 28)
    exposes `parent_node_id` and contains NO `inspect_contains_node`, i.e. a
    NON-inspection build that still published the method.
  * The consumer is production, not tests: unified_radix_cache.py:4388/4393/4408
    (the hash-chain walk) -> rust_tree_core/adapter.py:1082 get_parent_node_id
    -> binding.parent_node_id, and it is an abstractmember of the tree-core
    INTERFACE (unified_cache/unified_tree_core_interface.py:226) with a pure-
    python twin at unified_tree_core.py:583 -- so BOTH backends must serve it,
    and the python-side name is get_parent_* while the rust binding is
    parent_node_id (the sibling adapter.py inspect_get_parent_node_id kwarg fix
    B3-7 made is the other side of this same seam).
FIX: move the delegate into the always-on accessor impl beside its siblings
node_backuped/is_root (1824/1830), with a comment recording why it must not be
gated. Verified equivalently: the fresh non-inspection build again exposes
`parent_node_id` and contains no `inspect_*` name.

ROOT CAUSE 2 -- mamba_value_seqlen: a genuine OMITTED FIELD, semantics read
from the guard it feeds. insert_overlap_step_ (2105) is an upstream-new / B3
re-plant construction site that copies 12 of InsertParams' 13 fields from the
walk state and drops exactly this one. Filling it with `state.mamba_value_seqlen`
is the only self-consistent choice and matches BOTH sibling sites verbatim
(2058 in insert_walk_step_, 2251 in insert_commit_step_; the field is seeded on
the walk state at 1891 from the caller's params). A `None` here would have
COMPILED and SILENTLY DISARMED the entry-196/sgl-project#38815 alignment guard: the
consumer components/mamba.rs:174-183 returns early -- declining to stamp --
whenever params.mamba_value_seqlen is Some(seqlen) and key.atom_len()+
K::IS_BIGRAM != seqlen, which is precisely "a leaf truncated to a shorter
component boundary is not the position the donated state was produced at".
Since this same params carries state.mamba_value, pairing it with a None seqlen
would assert "produced nowhere in particular" about a real donated tensor.
Call-chain honesty note, measured not assumed: at this site the params is passed
ONLY to update_component_on_insert_overlap, which mamba does NOT override (only
swa.rs:436 does, reading prev_prefix_len/swa_evicted_seqlen), and the binding is
not reused later in the function -- so today the omission is compile-breaking and
LATENT-semantically, not yet an observable misstamp. Fixed at the root anyway.

ROOT CAUSE 3 (found by VERIFYING, not by the operator's report) -- the crate's
TEST TARGET did not compile either, so the era's cargo-test pin was never
provable: src/tests/components/swa.rs::
write_back_reinsert_defers_swa_backup_while_write_through_publishes carried a
half-converted replant: an `assert!(... matches!(action, CacheAction::BackupKV(_)))`
on the RETIRED enum architecture (CacheAction is not in scope -- E0433) plus a
STRAY ORPHAN LINE `.any(|action| action.as_any().is::<BackupKV>())` with no head,
between two statements (E0061-class syntax error). The converted expression was
the right one and is the file-wide idiom (used verbatim 3x in
tests/components/mamba.rs and by the filter_map four lines below); the authority
version (a7d333d, sgl-project#44800) proves the test's intent was: write-back branch
asserts NO BackupKV, write-through branch downcasts them and asserts
backups == vec![vec![leaf]]. FIX: assert converted to as_any().is::<BackupKV>()
and the orphan removed -- NO test weakened, none #[ignore]d.

VERIFICATION (this seat can compile, so it was verified, unlike every prior
claim in this era). Build environment is the crate README's documented recipe --
NOTHING installed, NO venv touched:
  LIBTORCH_USE_PYTORCH=1 LIBTORCH_BYPASS_VERSION_CHECK=1 \
  CXXFLAGS="-std=c++20 -include $PWD/torch_compat.h" \
  LD_LIBRARY_PATH=$(python -c 'import torch,pathlib;print(pathlib.Path(torch.__file__).parent)')/lib
(plain `cargo check` without LIBTORCH_USE_PYTORCH dies in torch-sys's build
script with "Cannot find a libtorch install"; the venv's activate exports NO
LIBTORCH/LD_LIBRARY_PATH, so the README env IS the equivalent of the operator's
setup -- recorded because it is the missing ingredient for all future rust debt.)

  cargo check -p sglang-radix-tree --features python-extension          -> exit 0 (was 3 errors)
  cargo check ... --features 'python-extension inspection'              -> exit 0 (no duplicate defn)
  cargo clippy -p sglang-radix-tree --features python-extension         -> clean
  cargo fmt --check                                                     -> rc 0 (crate is fmt-gated)
  cargo test -p sglang-radix-tree --locked     (dev profile)            -> 944 collected
        test result: ok. 943 passed; 0 failed; 1 ignored
     (re-run after `touch` to prove the test target really recompiled; the ONE
      ignored test is upstream's own pre-merge #[ignore] from a77283f /
      sgl-project#37290 -- insert_host_panics_on_a_colliding_page, documented
      "unconstructible today ... the assert is defensive-only". Not added here,
      not a suppressed failure.)
     ==> TRUE POST-MERGE RUST PIN: 943P/0F/1ignored, REPLACING the long-quoted
         "932P/0F" of the pre-merge era.

OPERATOR'S EXACT COMMAND, RUN VERBATIM, NOW GREEN:
  cargo rustc --lib --manifest-path ../rust/sglang-radix-tree/Cargo.toml \
    --release --features 'pyo3/extension-module python-extension' --locked \
    --crate-type cdylib
    -> exit 0; "Finished `release` profile [optimized] target(s) in 55.20s"
       (also re-run green from the other CWD the relative manifest path admits: 51.97s)
    -> artifact target/release/libmem_cache.so, 3,321,640 bytes, and it embeds
       `parent_node_id` while containing NO `inspect_contains_node`.

ALSO ANSWERS THE OPEN PYTHON-SIDE QUESTION: python/sglang/srt/mem_cache/
rust_tree_core/extension.py runs `bindings = load_tree_core_extension()` at
MODULE IMPORT, and rust_extensions/loader.py:402-407 shells out to EXACTLY
`cargo build --release --locked --package sglang-radix-tree --features
python-extension` -- the command named in the mem_cache dir's 2 collection
errors and in the draft_gate 4F. That feature set has NO inspection, i.e. those
python-side failures WERE this crate break, not independent env debt. Re-running
that loader command here: exit 0, "Finished `release` profile ... in 9.65s".
=> The python pins for those suites WILL MOVE on the operator's next full build;
   deliberately NOT re-run or re-pinned in this commit (one-line forward note in
   CAVEATS entry 319 instead).

Files: 3 source files, +29/-8. No artifact committed, no target/ churn, the
shipped extension binary untouched, nothing installed, NOTHING PUSHED.
Co-reported with the operator's verbatim failure lines.
detain added a commit to detain/sglang that referenced this pull request Nov 11, 2026
…r/URC markers -- 3/18 census consumed, 3 gates in one

CASCADE (start_writing(wait_for_forward) TypeError class, 68 distinct nodes):
upstream sgl-project#48012's NEW fixture hicache_write_back_fixtures.InstantWriteBackController
mirrored the pre-sgl-project#44800 no-kwarg surface while the fork's writing_check drain
(entries 199/218: the batched-backup flush=False + ONE fenced start_writing)
calls the real API. Def-site fixes, no fence weakened:
- fixture start_writing gains the keyword (instant copies -> accepted-ignored,
  documented);
- test_write_back_eviction_batching: record_submission wrapper FORWARDS the
  flag (it measures the very wait, so swallowing it would understate the
  fence); _FencedController gains load_fence_stream = forward (the fork's
  unconditional entry-168 fence reads it first; prod: both streams are the
  same forward stream) -- also silences the 16-test Rust TreeCore panic class
  whose "victims not demoted" assert was the aborted submission downstream.

MARKERS (3): managers/cache_controller.py:1202 ADOPTED -- the dropped
set_hicache_transfer_done_event write line is superseded by upstream
c24285b(sgl-project#48008 per-band page ledger)+2cbb66f563(sgl-project#48012): record_host_transfer
publishes the same signal per-page BOTH directions (load twin writes=True).
unified_radix_cache.py:627 ADOPTED -- the buffer-mode threshold floor
(resolve_prefetch_threshold) was DELIBERATELY deleted upstream by
dfaca56(sgl-project#46934, restores sub-window SWA L3 hits); window sizing now lives
per-allocation in BufferModePipeline; the per-call-site file_storage_path fold
is consistent with the entry-253 funnel (cache_controller.py:948, intact).
unified_radix_cache.py:3917 RE-PLANTED -- entry-293's cadence gate (park, do
not re-enter allocate_storage_hit/evict_host inside the retry window) + the
expired no-retry arm were dropped; wait_full folded into upstream's
comp_xfers allow_partial rule (min_tokens form already equivalent).

URC context guard: _deferred_ready_sync_group read get_memory() unguarded --
upstream's NEW _arm_tick CPU/spawn fixtures (their restructured timeout tests,
sgl-project#44614) reach it without a published server config. Now published-check
(schedule_policy.py:615 idiom): default-off CPU path, production unchanged.

Deadline-anchor retargets (fork B3-9 family vs upstream's sgl-project#44614 vote-before-
poll restructure; every assertion preserved): _vote_prefetch_deadlines drives
the verdict before the expiry polls; failed-arm polled inside-budget so the
storage failure stays the SOLE cut reason; fixture cc gains the sgl-project#44342
liveness attrs + pp_prefetch_command_thread; _arm_tick stub returns the fork's
5-tuple (health bit True); release_aborted_request -> finish(ABORT) per
8898429(sgl-project#48491).

Battery actuals (CPU, CVD=9, prod venv, PYTHONPATH=wt/python):
- the 5 cascade files 68F/39P -> 3F/70P (survivors = the sgl-project#48491/pp-thread/
  budget-anchor fixes committed here; after them): test_storage_prefetch_
  lifecycle ** 31P/31sub ** (entry-311 pin 28P/31sub, 295's >=23P/28sub
  EXCEEDED); decode_retraction_backup 6F + chunked_publish 2F = No-CUDA
  device class (env, reported).
- combined gate run: trailing_endpoint + batching + parity + async_ready_sync
  + staged_write_back = 89P/1S/28sub 0F.
Gates: ruff repo ruleset+format clean on touched files; registration trio rc=0.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

bypass-fail-fast CI: a failing job no longer aborts its siblings (lint still gates) hicache Hierarchical Caching for SGLang run-ci CI: run the baseline test suite on this PR unified-radix-cache

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants