Repository navigation
misc: use pip cache purge and add unit test ci - #871
Merged
Merged
Conversation
timethink
pushed a commit
to timethink/sglang
that referenced
this pull request
Mar 9, 2025
efschu
pushed a commit
to efschu/htsglang
that referenced
this pull request
Aug 26, 2026
…o the guard can arm sgl-project#718/sgl-project#847 built every part of the rebind except one: the 'tp' staging pin was assembled with a SINGLE entry, PoolName.KV. On a hybrid model the live tier carries KV *and* MAMBA, so `check_pool_coverage` computed `missing={MAMBA}` and refused -- correctly, on every cutover. A refused rebind leaves the sgl-project#718 device tier DISARMED, `load()` returns None, every read-through misses, and every prefix a cutover retracted is recomputed in full. Measured on the W40 sgl-project#857 acceptance boot (boot_w40_857strict_0825_2342.log): 60 `sgl-project#719 HiCache rebind refused`, ZERO arms, and `#cached-token: 0` on all 243 prefill batch lines -- one bucket, no exceptions. THE PRECONDITION WAS ALREADY WRITTEN DOWN, in the guard's own docstring: "A phase host tier has to be built with the FULL POOL SET before this rebind can arm; until then the sgl-project#718 disarm is the correct state and a read-through miss is the correct cost." This is that precondition, met. Not a new finding -- sgl-project#856 answers whether the retraction is necessary (it is: carry moves Req scheduling metadata, never KV bytes, and sgl-project#856 deliberately retired the movers), and sgl-project#718/sgl-project#847 already named this remedy. REFUSAL CONVERSION, NOT GUARD DELETION. `check_pool_coverage` is untouched. It must stop firing because its precondition is MET, never because it was removed, and the test asserts BOTH directions -- a full pool set arms, a narrowed one still refuses. Neither assertion alone can tell a fix from a disarm. DERIVED FROM THE BOUND TIER, not from the model config: the set that must be covered is whatever the READER names, which is the same quantity the guard compares. Reading the config would be a second opinion about one fact. MIRRORS `build_hybrid_mamba_stack` rather than re-deriving it -- same primitives, same layer mappings, same transfer_layer_num rule. The controller is deliberately NOT reused: this pin needs a host VIEW, and a second HybridCacheController would be a second writer against one device pool. Both entries are rebuilt in the hybrid case. The KV-only pin used an identity map over range(layers), which is right while KV is the only entry and wrong the moment a second pool shares the transfer index space -- the two maps collide at index 0. SIZING IS PER-SLOT, NOT PER-GB, and it is the one place the mamba half must not copy the KV half. MambaPoolHost reads host_size in GB only when > 0, else `device_pool.size * ratio`. The KV pin's GB figure is derived from a token count; mamba is allocated per request slot. Ratio 1.0 with host_size 0 mirrors the device pool, which is what a phase-matched staging pin means. The mamba half is a NAMED HOST-LEDGER POST, priced from what was allocated rather than from the intention. An unpriced pinned pool is what the ledger exists to prevent. A pool set this builder cannot mirror (SWA, indexer, DeepSeek) is logged as an ERROR at the cause and still refused by the guard -- named, not swallowed. THE CHECK (sgl-project#871, third scope item): `advance_fence_blind_streak` + FENCE_BLIND_STREAK=4, aggregating the EXISTING `persisted_nothing` instrument. One empty fence is legitimate; every empty fence means the canonical store can never populate, and that shows up only as latency. Gated on `released`, mirroring sgl-project#719's busy gate at the stale-gate streak: a fence over an empty tree is correct to persist nothing, so counting quiet cutovers would build a crying-wolf alarm out of the instrument written to replace one. NO SECOND COUNTER -- a parallel "recomputed prefix tokens" counter would measure what this and #cached-token already measure between them. Extracted as a pure function so it is falsifiable without booting. A guard whose logic can only be exercised by booting is a guard that ships unexercised, which is the failure mode this ticket is about. CLASS (unchanged, carried from the sweep): a store whose only writer is a lifecycle event another mechanism systematically preempts. Retention is finish-only; the cutover retracts before finish; the store can never populate, so every recovery path reading it can never fire. TEST RESULTS test_phase_tier_full_pool_set_871.py (new): 15 passed. FALSIFIED IN THREE DIRECTIONS BEFORE THE GREEN WAS CLAIMED, count gate held on each: guard removed (disarm) -> 3 failed, and they are exactly the refusal tests. 3 extracted == 3 in summary. guard fires unconditionally -> 6 failed, exactly the arming tests. 6 == 6. streak loses its `released` gate -> 1 failed, the idle test. 1 == 1. restored -> 15 passed. Targeted regression set (7 files: sgl-project#847 writer, sgl-project#783 fence x3, sgl-project#856 empty wave, counters, and the new file): 97 passed, 0 failed, 2 subtests passed. Count gate 0 extracted. Hermetic, CUDA_VISIBLE_DEVICES verified EMPTY at the PROCESS (/proc/<pid>/environ), not at the command. AN EXISTING TEST CAUGHT A REAL BUG OF MINE and the first run of that set was 3 failed: the new HOST-LEDGER line read `tp_host.entry_map` directly, but that writer is driven in tests by stand-ins where HostPoolGroup itself is patched. Fixed with the getattr discipline this module states at its other probes -- an instrument may never be the thing that breaks a boot. Lint: ruff 0 before and after on phase_flip_runtime.py and the new test. phase_flip_boot.py reads 1 before AND after -- a pre-existing F401 on `pack_into_arena`, present in HEAD, not mine. codespell clean. Flag-off path verified byte-identical: without --phase-flip-rebind-hicache the writer still returns {} and allocates nothing. NOT CLAIMED, AND DELIBERATELY NOT GUESSED: no metal run. Whether the kv+mamba pin can actually be ALLOCATED on this box, what its real HOST-LEDGER post comes to against the 16G floor, and whether #cached-token becomes non-zero are decidable only on hardware. They belong to a boot window. The sgl-project#857 acceptance instance was left running and untouched throughout -- it is the standing proof and the operator declared it taboo.
efschu
pushed a commit
to efschu/htsglang
that referenced
this pull request
Aug 26, 2026
Two subtractions, one justification, only one of them checked:
scheduler.py `_pending_now -= _seam_transport_now` UNCONDITIONAL
scheduler.py `_seam_serviceable_now = ...` gated on the TP phase
AND on
seam_transport_premise_holds
Both rest on the same claim -- that a seam re-admission is cheap flip transport
rather than real workload, because "their prefixes are served by read-through
from the canonical store". #861j verified that claim for the EXISTENCE term and
never backported the verification to the ECONOMICS term twelve lines above it.
WHEN THE PREMISE IS FALSE those tokens are not transport at all: they are a full
cold prefill of real work, and deducting them tells the policy that work does
not exist. The consumer that pays is the one arm still reading RAW pending --
the sgl-project#677(a) blocked-admission stall escape, whose threshold is
`pending > pp_exit_tokens`. A deflated pending holds a genuine stall below its
own escape, which is the wedge that escape was written to end. Delayed escape,
not a wrong answer: `demand_prefill_tokens()` takes
max(pending, admissible - serviceable) and `admissible_prefill_tokens` is not
reduced by seam transport, so with sgl-project#869 landed `_strict_holds_pp` still sees the
true backlog.
VARIANT (a) OF THE TWO I FILED, chosen because it makes the two subtractions ONE
RULE rather than removing the last raw-pending consumer and leaving the pair
disagreeing. `seam_transport_deduction` is pure and total, so both directions
are falsifiable without a scheduler, and the premise is now asked ONCE per round
and read by both terms -- one predicate, one clock, per that function's own
contract. Asking twice was its own latent defect: the debt clock can lapse
between two calls in the same round.
NOT A DISARM. A verified premise in the TP phase still deducts exactly as
before, which is the #861j/W32 behaviour where it was actually earned. The test
pins that direction too.
CLASS: a subtraction justified by a premise that is never checked -- the same
family as sgl-project#869 (a predicate that does not measure what it claims), different
root. Sibling of, not instance of.
TEST RESULTS
test_seam_transport_premise_869c.py (new): 11 passed.
FALSIFIED IN BOTH DIRECTIONS BEFORE THE GREEN WAS CLAIMED, count gate on each:
fix reverted (deduct unconditionally = today's tree) -> 7 failed, INCLUDING
the specified falsifier test_the_stall_escape_arms_on_the_true_backlog.
7 extracted == 7 in summary.
over-fixed (never deduct, blanket disarm) -> 3 failed, including
test_a_verified_premise_in_tp_still_deducts. 3 == 3.
restored -> 11 passed.
Regression set, 59 files (purity, policy, seam, transport, sgl-project#677, sgl-project#713, sgl-project#861,
sgl-project#871, #869c): 895 passed, 4 failed, 123 subtests passed. Count gate 4 extracted
== 4. All four are test_restore_never_rebuild_677, one of the three standing
card-needing modules whose hermetic count is exactly 4. No new failures.
Hermetic: CUDA_VISIBLE_DEVICES verified EMPTY at the PROCESS (/proc/<pid>/environ).
Lint: ruff 0 before and after on phase_purity.py; scheduler.py 102 before AND
after (pre-existing); new test 0. codespell clean.
Non-flip and non-seam paths are byte-identical: with no stamped re-admissions
`_seam_transport_now` is 0, and the deduction is 0 under every gate combination.
TWO BUGS IN MY OWN TEST, found by running it and reported rather than quietly
fixed. The first draft left the hand-set `pp_window_s` stopwatch enabled, so the
deflated specimen still armed -- for a different reason -- and the test would
have passed without the escape ever being consulted; it now sets pp_window_s=0
and decode_stall_slo_s=0 to isolate the sgl-project#677(a) arm. The second asserted on the
substring "stall" in the reason string, which matched
SGLANG_PHASE_POLICY_DECODE_STALL_SLO_S in an unrelated suggestion line; it now
asserts on the verdict. Same substring-matching family as the ^FAILED trap.
NOT CLAIMED: no metal run. Desk-proven only. The sgl-project#857 acceptance instance was
left running and untouched throughout.
efschu
pushed a commit
to efschu/htsglang
that referenced
this pull request
Aug 26, 2026
…oute blind over the anchor
W38 acceptance boot died on all three ranks, 2026-08-26 12:54:28Z:
IndexError: index 76997 is out of bounds for dimension 0 with size 30518
pool_host/base.py:344 assert self.slot_used[indices_cpu].all()
memory_pool_host.py:1724 return self.anchor_entry.host_pool.free(indices)
unified_radix_cache.py:2853 mem_pool_host.free(host_indices[:unclaimed_to])
HostPoolGroup.anchor_entry is fixed at construction, so the anchor does not move
inside one group -- the GROUP is rebuilt onto a narrower tier at a phase rebind.
In-flight prefetch state does not move with it: check_prefetch_progress holds
host_indices minted against the previous, wider tier and frees them after the
rebind, against a pool whose slot_used is shorter.
THE CLASS, not the instance. 322f331 (sgl-project#718/sgl-project#847) fixed the TRANSFER path via
_entry_for_transfer and did not sweep one level up. That resolver cannot cover
these: it resolves by transfer.name through entry_map, and free / alloc /
get_page_buffer_meta / get_data_page / set_from_flat_data_page receive BARE
INDICES with no name (it has exactly two callers, :1803 load and :1834 backup).
The index axis needs a range guard, not the resolver. Its own docstring asks for
this: the raise exists "so that a future producer that bypasses the resolver is
LOUD rather than wrong" -- free() is exactly such a producer.
Sibling axis of _host_binding_is_stale (sgl-project#760), which guards the POINTER axis
(device_buffers captured at construction) and left the INDEX axis unguarded.
Same time signature: sgl-project#760's specimens died three seconds after a pp_to_tp
cutover completed; this one dies on the free path instead of the write path.
free() DROPS a stray, the other four are LOUD. A dropped free is a no-op on a
tier torn down wholesale -- nothing leaks. An accessor that returns or writes a
page cannot drop: that shortens a result or skips a store and the caller
proceeds on data nobody wrote, the wrong-answer-with-no-crash outcome this
codebase ranks worse than a crash. StrayHostIndexError subclasses IndexError so
existing handlers keep working.
THIS IS A BACKSTOP, NOT THE CURE, and the ticket should not be closed on it.
The cause-side cut is one of two, both open: refuse at the rebind when the
narrower tier cannot cover what is in flight (sgl-project#871, the unmet sgl-project#718/sgl-project#847
precondition), or make ongoing_prefetch a cutover participant with a
completeness check (sgl-project#859, cutover_participants.py, present at the pin). What
this commit guarantees is only that the failure is named and survivable instead
of killing three schedulers.
Unlike the binding generation stamp -- dead code whenever the binding does not
advance -- this guard compares against the live pool's actual size and cannot
be inert.
Tests: test/registered/unit/mem_cache/test_stray_host_index_718_class.py, 13
passed. Can-fail proven by two mutations: disabling the stray filter reds 2,
making the loud refusal a no-op reds 5; restore returns 13 green.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Thank you for your contribution, we really appreciate it. The following instructions will help improve your pull request and make it easier to receive feedback. If there are any items you don't understand, don't worry. Just submit the pull request and ask the maintainers for help.
Motivation
Please explain the motivation behind this PR and the goal you aim to achieve with it.
Modification
Briefly describe the changes made in this PR.
Checklist
pre-commit run --all-filesor other linting tools are used to fix potential lint issues.