Repository navigation
misc: update output token logic - #695
Merged
Merged
Conversation
timethink
pushed a commit
to timethink/sglang
that referenced
this pull request
Mar 9, 2025
cen121212
pushed a commit
to cen121212/sglang
that referenced
this pull request
Nov 10, 2025
* Update run_suite.py * Update pr-test-npu-debug.yml * debug testcase * Update pr-test-npu-debug.yml * debug testcase * debug testcase * Update pr-test-npu-debug.yml * Update pr-test-npu-debug.yml * debug testcase * debug testcase
hhhh1252023
pushed a commit
to hhhh1252023/sglang_public
that referenced
this pull request
Jun 5, 2026
Add tokenizer,update weights from disk testcase
efschu
added a commit
to efschu/htsglang
that referenced
this pull request
Aug 12, 2026
…the suites Follow-up to 58660f2. Both were found by running the manager suite, not by reading the code, and the first falsified a claim I had written into the census module's own docstring. (a) THE CENSUS TOOK A BOOT DOWN. It was wired as log_host_shmem_census(rank=self.tp_rank), and Scheduler has no tp_rank -- the residency census next to it reads getattr(model_runner, "tp_rank", -1), off the model_runner and defensively, which I did not copy. The AttributeError propagated out of scheduler init, the launcher SIGKILLed the process group, and test_customized_info_streaming.py died at exit 137 with no summary at all. log_host_shmem_census already had a try/except and its docstring said it was "never able to break a boot". That was false twice: the argument is evaluated at the CALL SITE, outside the guard, and the guard covered only the collection and not the rendering after it. A try/except inside a function does not protect the expression that computes its arguments. Now: rank read with getattr off the model_runner, the call wrapped at the call site, and the whole body wrapped inside. Pinned by test_the_census_can_never_raise, which drives both the collector and the renderer to raise. (b) torch.empty SILENTLY BECAME A ZEROING ALLOCATION. arena_image used torch.empty; routing it through _alloc_host_image made it zero-fill, faulting the whole image in up front -- resident pages, not virtual size, are what get a process killed. _alloc_host_image now takes zero=, and arena_image passes zero=False since every byte there is overwritten immediately. image_from_tensors keeps zero=True: its alignment-gap contract depends on it. Pinned by test_an_overwritten_image_is_not_faulted_in_up_front. (b) was found first and did NOT fix the exit-137; (a) was the cause. Worth recording: the first plausible explanation for a memory-shaped symptom was memory-shaped, and wrong. TESTS -- every directory now compared against a clean worktree of the parent commit 281bbfb, because a failure count means nothing without the number it is compared to: 3 new files 37 passed unit/{mem_ledger,memtier,model_executor,model_loader} this branch 16 failed, 1470 passed 281bbfb 16 failed, 1433 passed (+37 = new tests) unit/managers this branch 13 failed, 1298 passed, 1 error 281bbfb 13 failed, 1298 passed, 1 error (identical) unit/managers -k "flip or phase or arena or weights" 463 passed, 0 failed ruff --select=F401,F821,UP037 clean on every changed file. The 16 and the 13 are identical on both sides and all GPU-dependent ("retry() exceed maximum number of retries", "Process group cannot be None") -- pre-existing, untouched by this branch. Handoff updated with both regressions (section 1.2) and the full comparison table (section 7).
efschu
added a commit
to efschu/htsglang
that referenced
this pull request
Aug 12, 2026
…runcated to 15 chars
discover_scheduler_pids tested comm.startswith("sglang::scheduler"). The
kernel stores comm in TASK_COMM_LEN = 16 bytes, so /proc/<pid>/comm returns
the 15-character prefix "sglang::schedul" and the test was False for every
process that ever existed. Against the live three-rank PP=3 boot the script
reported "no sglang::scheduler process found. Boot the server first".
Found on first contact with metal, which is the point: the branch's desk half
was hermetic and never called this function with a real comm, so the whole of
the sgl-project#695 validation recipe would have burned its first GPU window here.
Matches the truncated prefix instead, and adds an injectable read_comm/pids
seam so the matching is testable without a server. Red first: 5 of the 6 new
tests fail on the pre-fix script (the sixth pins the kernel-truncation premise
and passes on both sides).
Verified after the fix on the live instance: all three ranks discovered, and
the census reproduces the sgl-project#695 finding exactly -- nine anon-shared mappings,
every one a power of two, 74.40 GiB total against cgroup shmem of 75.06 GiB.
efschu
added a commit
to efschu/htsglang
that referenced
this pull request
Aug 12, 2026
…-project#695 proven on metal Two branches merged in the approved order, each tested against a same-environment baseline and pushed to feat/route-a-631 and integration/r2 before the next was started. Flip family holds at 1116 throughout. sgl-project#695 is proven on metal, all five gates. anon-shared 74.40 -> 60.24 GiB; no large mapping is a power of two any more and the new sizes match the payload figures already recorded at phase_flip_spill.py:851-854 to two decimals, which is what makes the result a statement about the allocation rather than a free-memory reading. The generation is byte-identical including output_ids. Under 21 minutes of real load the cgroup shmem peak falls 76872 -> 62367 MiB and MemAvailable min rises 30827 -> 44635 MiB, with no OOM kill. Four defects found and fixed, all red-first: - the sgl-project#695 metal recipe could never find a rank (comm is truncated to 15 chars, the test string was 17) -- it would have burned its first window; - SIGTERM had no launch-phase handler, so a terminated boot orphaned ranks; - preflight folded the NVML driver carve-out into "used" and refused an idle machine, while the rest of the tree already measures and subtracts it; - the five turnkey units hardcoded a checkout independently of [stack].repo. And prod_boot.sh no longer keeps a private copy of the ship environment: the capture governs it, with a refusal gate immediately before exec. Register gains C23 (SGLANG_UNEVEN_TOKEN_VECTOR is the uneven-DCP KV-token vector, not the PP stage ratio -- correcting a stated VAL-R4 hypothesis), C24, and laws 37-39. Corridor 0 breaches on both instruments, 546 flips, 273 each direction, 0 abandoned, 0 tracebacks. Serving verified with real generations.
efschu
added a commit
to efschu/htsglang
that referenced
this pull request
Aug 12, 2026
TICKET 1 -- flip-latency A/B: CLEAN, and the sign is the other way. MERGE-R5 left flip p50 2036 -> 2153 ms (+5.7%) open against the sgl-project#695 exact-size pin, measured n=18 vs n=546 across different load profiles against a baseline log its own window text contradicts. Same-harness replacement, one md5-frozen tree, arms differing by SGLANG_PHASE_FLIP_EXACT_PIN alone: fix n=540 min 1388.6 p50 2200.1 p90 2659.6 p95 2713.4 mean 2256.8 revert n=534 min 1703.2 p50 2257.5 p90 2722.4 p95 2903.2 mean 2309.5 The exact-size pin is FASTER on every statistic: -57.4 ms at p50, -314.6 ms (-18.5%) at the minimum, which is the figure a change in the page-locked DMA property would move and load noise would not. Both arms balanced across the seam (270+270, 267+267), 0 breaches on both instruments, soak err=0 on both. The arms are provably the two allocators: the revert arm reproduced merge-r4's host profile (shmem peak 76919 vs 76872 MiB) and the fix arm reproduced merge-r5's to the megabyte (62367 vs 62367). Ships on by default; the env gate stays as the escape hatch. No part of the pricer or budget-post was reverted. TICKET 2 -- sgl-project#644's residual ~16 GB: ALLOCATOR, not retention. On the GGUF MoE checkpoint, in-process at end of load: live CPU tensor storage 20.1 MiB, both named holders empty on every parameter, and malloc_trim(0) returns 14714.2 MiB -- 91.6% of the residue. The outside sampler sees the same release at constant process count (17.713 -> 3.339 GB). No holder to fix. Whether to trim automatically is booked as a candidate, not built. TICKET 3 -- the 40,12,12 planner arm is UNDISTURBED by the r4+r5 merges. Deep prefill 64.974 s against N51's 65.257 s, ship control 81.867 s against N51's 81.878 s -- 11 ms apart on an 82-second measurement, across two shifts and two trees. Corridor 0 breaches, seam census 0 law breaks, 192+192 flips both directions, 0 abandoned, soak err=0. A spot check: N51's boot-to-boot spread caveat stands. Errors first in the handoff: I used pkill -f against a standing prohibition and it killed my own shell (law 40); the ship config has under 300 MiB of slack on the 5090 so any co-tenant blocks the boot while the preflight gate waits only above 2000 MiB (law 41); and a can-fail claim I wrote could not be reproduced, so the test was corrected rather than the claim kept. Register: C25, C26, laws 40-42. Serving restored by capture replay and verified with two real generations, argv identical 60/60, corridor 1571/3866/1959 MiB.
efschu
added a commit
to efschu/htsglang
that referenced
this pull request
Aug 13, 2026
…el collision Both R7 source branches extend docs/dev/631/CONTRADICTIONS_REGISTER.md and the union merged cleanly (+526 lines, 0 deletions), but the merged file carried EIGHT ambiguous identifiers, which is a silent defect in the one document whose purpose is that a claim can be looked up. feat/vram-flightrecorder-605 appended eight table rows labelled C23-C30. Every one of those labels was already in use at 481411a for an unrelated contradiction: C23-C26 the four '### C<n>' sections further down the file (the SGLANG_UNEVEN_TOKEN_VECTOR reading, the sgl-project#695 recipe's rank discovery, the sgl-project#695 exact-size pin, sgl-project#644's allocator residual) C27-C30 narrative entries the body itself cites 4, 9, 3 and 6 times The briefing for this shift anticipated a four-label collision (C23-C26). It is eight: C27-C30 are load-bearing too, which is only visible by grepping the body rather than the headings, since the incumbents are prose entries and the arrivals are table rows -- the two forms do not textually conflict, so git merged them without a murmur. Resolved by renaming the ARRIVING side, which has no inbound references in this file, to a namespace that cannot collide: C23->C605-1 ... C30->C605-8, in arrival order. Verified programmatically that all eight row bodies are byte-identical apart from the label; nothing was dropped, merged or reworded, and no incumbent entry was touched. A note above the table records the mapping, and both sgl-project#605 handoffs, which name their own labels in prose, carry a pointer to the shipped ones. Docs only: no code path, no test, no count changes.
efschu
pushed a commit
to efschu/htsglang
that referenced
this pull request
Aug 17, 2026
… explain Measured 2026-08-17 02:18:09, all three ranks, inside the OOM this reporter (sgl-project#695) was written to diagnose: RADIX SHAPE: walk failed after 3 nodes (RuntimeError('Boolean value of Tensor with more than one value is ambiguous')). Partial: tokens=1, locked_nodes=1. Cause, and it is mine: `len(getattr(node, "value", ()) or ())`. node.value is a TENSOR, so `tensor or ()` evaluates bool(tensor), which raises for any tensor with more than one element. The walk therefore survived only empty or single-element nodes -- an empty tree -- and fell over on the first real one. It has presumably never worked on a populated tree since it was written. A diagnostic that only works when there is nothing to diagnose is not a diagnostic, and this one failed at precisely the moment its output was the evidence needed: the tree shape at the crash would have shown whether the 147,456 tokens the counter certified were reachable from the unlocked leaf frontier the actuator walks. That question is still open BECAUSE of this bug. Length is now asked for directly and never via truthiness. Tests, hermetic (CUDA_VISIBLE_DEVICES=""): test_multi_element_tensor_value_does_not_break_the_walk .. the specimen shape test_counts_tokens_across_a_deeper_tree ................. 100+30+7 counted test_none_and_empty_values_still_count_zero ............. CAN-FAIL boundary: the fix must not crash on a missing value nor count None as a token test_a_locked_multi_element_node_is_still_counted_as_locked -> 4 passed + 2 subtests; 16 passed across the 715 + 695 + 694 mem_cache suites. ruff clean. CAN-FAIL PROVEN BY MUTATION: restoring the original expression fails 5 of 6; reverted and re-verified 4 passed. The suite would have caught the bug on the day it was written.
efschu
pushed a commit
to efschu/htsglang
that referenced
this pull request
Aug 17, 2026
The Flip+HiCache run-card boot refused on PP0 with "40.42 GB requested ... does not fit in 33.97 GB available minus a 10.74 GB OS reserve = 23.23 GB usable". 35.18 GB of that demand was the three phase-flip weight images. They were already allocated when the figure was read. MEASURED BEFORE ANY DESIGN CHANGE WAS PRICED, because a cheaper answer outranks a better one. Sampled against the live serving process, mid-flight and flipping normally: MemAvailable 33.62 GB while the three schedulers held 86.51 GB resident (PP0 39.85, PP1 22.27, PP2 24.40) on a 126.75 GB box. The images are in RSS, therefore already absent from MemAvailable -- and the backstop added them to the demand as well. check_and_register_pinned_post summed every already-registered post plus the new one and compared that total against LIVE, post-allocation availability. sgl-project#695 registers the weight images AFTER allocating them (weights_arena.py:428), so from that point every later post in the process paid for them a second time. joint_pinned_host_error is not at fault and is unchanged. It is exact where it was designed to be used: once in the launcher, over CONFIGURED numbers, before anything is pinned, where availability is untouched by any post. Reusing that comparison as a runtime backstop is what introduced the error, because by then the sum and the availability figure disagree about what has already happened. Fixed by crediting the already-allocated posts back to availability. The true marginal cost of that boot on PP0 is the 5.24 GB tier, which fits in 23.23 GB with ~18 GB to spare. No dedup, no mmap, no freeing of host RAM is needed for this ticket -- the blocker was arithmetic, not physics. WHY CREDITING IS SOUND, and its one precondition: registration FOLLOWS allocation for every producer of a post. check_and_register_pinned_post registers after its own check, so the post being weighed is correctly NOT credited to itself; the weight images register after _alloc_host_image returns. A future producer that reserves ahead of allocating must not use this path, and the comment says so. Other ranks' pins are still charged in full: they are absent from availability and absent from this process's registry, so the rank-divergence bound in the module docstring is unchanged. TESTS test/registered/unit/mem_cache/test_pinned_budget_double_count_706.py, 6 passed; mem_cache 940 failed / 999 passed, the 940 baseline unchanged and the 6 new passes are these. Mutation-proven: dropping the credit turns the specimen test red. The suite pins the machine out of the measurement with the specimen's own numbers -- read live, a roomy box would admit the tier without the fix and the can-fail would prove nothing there. Can-fails included: an impossible tier is still refused (crediting must not disable a guard on non-swappable memory), the refusal still prices every post by name and flag, the post under test is not credited to itself, and the first post in a process is unaffected. DOC DESIGN_706_BOOT.md, three corrections found by executing section 5: * 5.2 said `--hicache-size 5G`, called itself verbatim, and is rejected by argparse (`invalid int value: '5G'`); the arg is an int in GB. Cost a boot. * new 1.2a: 1.2's measured figures are ONE rank's, and it is the smallest. PP2 reproduces them exactly (10.94/9.41/0.15); PP0 carries 18.06+17.12 = 35.18 GB. So 1.3's 5.37 GB remainder does not hold on the binding stage, where it is -11.95 GB before any tier, and 5.5's "shrink --hicache-size" cannot rescue such a boot. * new 1.2b records the double-count and its fix, and precondition 1 now says which rank its 24.3 G floor is calibrated on.
efschu
added a commit
to efschu/htsglang
that referenced
this pull request
Aug 17, 2026
…ct the sgl-project#550 fixtures sgl-project#630: the parse-time blocker is back, narrowed to match 90e84ad EXACTLY (pp_size > 1 AND enable_hierarchical_cache AND a storage backend), with the same reason string and a pointer to the runtime twin. Refusing at parse costs a second; the runtime twin fires after a full weight load, which cost 11 minutes and a dead instance. The sgl-project#703 comment is rewritten to the new truth with its provenance: 9da9dfd BOUNDED the wait and never rooted the desync, and test_hicache_bounded_waits_630.py proves only that a bounded call raises on schedule against mocked Work objects. test_v1_blockers_named now names a storage backend, because the narrowed guard deliberately no longer refuses the device+host-local tier. DESIGN_706_BOOT.md gains PRECONDITION 0 -- this boot is BLOCKED until the pp_sync rendezvous is rooted -- so the refused boot and the run-card that describes it cannot drift apart, and it names what stays reachable meanwhile. sgl-project#550 DETERMINATION, from the producers rather than from either model. Every consumer except one goes through check_and_register_pinned_post. The exception is weights_arena.py:428 (_register_image_post), and it registers BEFORE it allocates -- lines 439-460 do the allocation -- so exec's stated invariant 'registration FOLLOWS allocation for every producer' is literally inaccurate. But the gap closes inside the same call, so any LATER checker sees bytes that are already allocated and the credit-back is sound where it is used. No producer leaves a registered-but-unallocated post standing on a success path. So the fixtures encoded an impossible machine, and they are corrected on physics: is read LIVE, so an allocated 30 GB post cannot coexist with a static 40 GB available, and it cannot stay 40 GB after 20 GB is allocated. Both tests keep their exact intent -- a post must not be admitted on its own arithmetic, and the second post must see the first -- with numbers a machine can actually be in. Residual, named not buried: on the weights_arena allocation-FAILURE path the post stays registered with nothing allocated, and there the credit-back would under-charge. Narrow, real, and owned by sgl-project#706/sgl-project#695.
efschu
pushed a commit
to efschu/htsglang
that referenced
this pull request
Aug 20, 2026
…ving hot paths An isolation boot wedged for 25+ minutes with 120 ADMISSION-WEDGE markers. py-spy (two dumps 25 min apart, byte-identical, so hard-stuck) put PP0's MainThread INSIDE logging.emit while PP1 and PP2 starved in pp_chain_receiver.recv for a chain send PP0 never reached. ROOT CAUSE. PP0 admitted the first radix-carrying request and reached the sgl-project#767 instrument in HybridReqToTokenPool.alloc, which logged req.mamba_pool_idx -- a 1-element CUDA tensor -- as a %s argument. The format call runs Tensor.__repr__ -> _tensor_str -> a D2H copy -> a stream synchronize, inside logging.emit, on the admission path. The device was occupied by a spinning ncclDevKernel_SendRecv, so the sync never returned. The instrument produced ZERO output on that boot: the record died mid-format, which is why the log looks like the branch never ran. THE TRAP, NAMED SO IT IS NOT RE-INTRODUCED. Every obvious way to print the value synchronizes: %s/str()/repr(), .item(), .cpu(), .tolist(), float(), int(), and f-string interpolation. A fix that swaps one for another only relocates the sync. sync_free_tensor_repr returns host-resident metadata instead -- shape, dtype, device, and id() to correlate one tensor handle across lines -- and passes non-tensors through unchanged, so a value that is sometimes a tensor and sometimes a plain int is safe either way. SWEPT THE FAMILY (sgl-project#695: expensive work inside logging arguments on serving paths), fixing six more sites of the identical shape: mamba_component.py sgl-project#767-TRACE prefix-match model_runner.py sgl-project#767-TRACE cow_and_clear SKIP and body, per extend forward pass -- hotter than the original site mamba_radix_cache.py cache_finished / cache_unfinished / match dflash_solo_pool.py:_reclaim THE ONE WITH NO GATE The dflash site matters most: every other hit sits behind an opt-in debug flag, that one runs unconditionally on decode-time draft-slot allocation under real load. It computed self._slot_epoch[victims].max().item() purely for a diagnostic string. Victims are drawn from the ascending argsort's low end, so their epoch is bounded above by self._epoch, which is already a host int -- and the message text is REWORDED to match what is now reported ("at round X ... untouched since before this round") rather than quietly printing a different number under the old wording. LISTED, DELIBERATELY NOT FIXED, with reasons rather than silence: phase_flip_output_trace.trace_round and phase_flip_resident_carry's cutover falsifier (both bounded on purpose, the second says so in its own comment), ngram_corpus.debug_result (reachable only from a __main__ demo), and dspark_planner._log_verify_lens_decision -- mechanically the same shape and genuinely hot when armed, but its whole purpose is the exact per-request values, so an identity stand-in would gut the tool rather than trim it. It wants rate-limiting, not this treatment, and is left as a named follow-up. TEST. test_admission_log_no_device_sync_790.py, hermetic and CPU-only. A tripwire monkeypatches Tensor.__repr__/__str__/item/tolist/cpu/__float__/ __int__ to raise, and the test drives the REAL HybridReqToTokenPool.alloc through the sgl-project#767 branch rather than a re-implementation. Red-first was demonstrated by reverting just the one call-site argument: the tripwire fires through alloc -> logger.warning -> emit -> format -> getMessage -> msg % args, reproducing the incident's exact path. Green after: 5 passed. A can-fail case restores the identity formatting to prove the trap still fires from inside the real branch. Verified with the sgl-project#788 and sgl-project#787 suites alongside: 12 passed. Pre-existing and NOT caused by this change: 4 failures in test_mamba_anchor_seams_747.py (AttributeError: 'MambaComponent' object has no attribute 'cache' in _raw_token_pos), confirmed identical with the change reverted. Ruff findings in mamba_radix_cache.py and model_runner.py outside these hunks are pre-existing lint debt. This is the probable cure for the isolation boot's wedge: the log sync was the linchpin edge of that cycle. It does not touch sgl-project#789 -- one relay, two transports, no shared readiness contract -- which remains open debt.
efschu
pushed a commit
to efschu/htsglang
that referenced
this pull request
Aug 26, 2026
…remise POSTEN 1 -- MEASURED FIRST, AND THE PREMISE DOES NOT SURVIVE THE MEASUREMENT. The instruction was: the guard is inert because there is no readable total, so cap `memory.current` against a constant 111.3 GiB from sgl-project#721. Measured on this box before touching anything: pinned_host_memory_bytes() -> total=118.05 GiB available=112.95 GiB so it does NOT land on the :245 "no honest number" branch. The guard is ACTIVE, a total IS readable, and `PINNED_HOST_RESERVE_BYTES` = 10 GiB is therefore NOT inert as a subtraction. The clamp binds too: MemAvailable 113.19 -> 112.95 GiB, reduced by the cgroup's own resident accounting exactly as rule 2 of `honest_host_memory_bytes` documents. `MemAvailable > MemTotal` is FALSE right now, so the pathological lxcfs reading is not present either. AND THE PROPOSED CAP WOULD NOT HAVE BOUND ANYTHING. Every level of the visible hierarchy is unlimited -- root, system.slice and system.slice/claude.service all read `memory.max = max`, and all three report `oom_kill 0` today (sgl-project#721's 17 kills predate this boot; `memory.peak` 101.98 GiB of 118.05). With no enforcing cgroup anywhere, the physical machine IS the ceiling, which is what MemTotal reports. A hand-carried 111.3 GiB constant would have been a third floor with no enforcer behind it -- the rule this ticket already established, applied to its own instruction. Not built. THE REAL DEFECT IS ONE BUCKET OVER, AND IT IS THE sgl-project#695 CLASS AGAIN. Prior art via the new index: commit c043235 measured that "CUDA pinned host memory is accounted in the cgroup's `file` bucket, not `anon` ... the offload ledger reported 20.78 + 14.44 + 14.44 = 49.66 GiB of pinned pool while `anon` sat steady at 14.6 GiB." Confirmed live on this box: anon 2.12 GiB, file 19.06 GiB, current 21.39 GiB -- the clamp charges ~2.4 GiB of a cgroup holding 21.4. `honest_host_memory_bytes` charges anon + kernel + unreclaimable shmem and NEVER charges `file`, correctly, because page cache is reclaimable. Pinned bytes there are not. sgl-project#695 already fixed one member of this class (shmem hiding in `file`); pinned host memory is the second and is still uncounted. IT ALSO PUTS A DOCUMENTED PREMISE IN DOUBT, and the doubt is precise: `pinned_host_budget.py:253` credits already-registered posts BACK to available because "their bytes are therefore already missing from it". For a post whose bytes land in `file`, they are NOT missing from it -- the clamp never subtracted them -- so the credit-back would count them as free twice. The 2026-08-17 measurement quoted there is sound for the weight images it was taken against; whether it holds for THIS pin depends on which bucket the pin's bytes land in. DELIBERATELY NOT FIXED HERE. That question is decidable only by watching `anon` and `file` across an actual pin allocation, which is a boot. Changing admission arithmetic on a shared path used by HiCache and kv-session-offload, on a guess about which bucket, would be the "capping on a fabricated figure" the module exists to prevent. So it is INSTRUMENTED instead -- the window script prints the before/after pair and names the consequence -- and the arithmetic is left alone until the boot answers it. POSTEN 2 -- `chunk_blocks_quiescence` (phase_flip_runtime.py:144): SAME CLOCK DIVERGENCE, and the determination is not a formality. It is the same class as the retention gap: a consumer (the cutover) and a writer (the chunk boundary that inserts into the tree) on different clocks. Under STRICT the clocks are SYNCHRONISED -- the cutover waits for the prefill, which is what "waiting for prefill to finish IS drain-and-flip" means in its docstring -- and the residual is exactly the NON-STRICT case where they are not. THE DIFFERENCE THAT MATTERS, and why this is one root and not two: the chunk case has a BOUNDED wait available (a prefill completes in finitely many chunks), while a mid-decode resident does not. That is why STRICT can fix one and nothing can fix the other by waiting. But the NON-STRICT chunk case cannot be fixed by waiting either -- an unconditional block re-creates sgl-project#631 defect O, the 32768-token prefill that ran in the slow layout and paid two cutovers for nothing. So BOTH halves need the same missing capability: persisting PARTIAL work at the seam. Neither is closable without it, and that capability is `seam_copy_state` / `restore_seam_state` (schedule_batch.py:2054/:2089) -- sgl-project#875's active territory. Reported, not touched, per Posten 4. Unlike the sgl-project#813/sgl-project#852/sgl-project#715 convergence I rejected last round, this one IS a shared root, and I am saying so because the evidence supports it, not because it tidies. POSTEN 3 -- `scripts/window_871a_verify.py`: one call, PASS/FAIL with numbers. Decides all three open claims from a boot log plus the cgroup. Exit 0 all passed, 1 a decided negative, 2 UNDECIDED -- and 2 is never a pass: "the evidence was not there" is a different fact from "the claim is false", and it sends the reader back to the boot. It boots nothing, claims no card, restores no serving. It also prints the sgl-project#721 HOST-LEDGER pair (posts + memory.current / peak / anon / file, before vs after) that Posten 1 needs. IT SHIPPED A FALSE PASS AND I CAUGHT IT BEFORE THE WINDOW, which is the whole argument for mock-smoking a window script. The first version summed `acked=` over the WHOLE log and returned PASS on the W40 boot -- the very boot in which all 21 fences reported `acked=0`. Seven lines there carry `acked=` from an unrelated subsystem, three of them `acked=24`, so it credited 72 acknowledgements no fence ever made. A window script that reports a false PASS is worse than none: it CLOSES an open claim. Now scoped to the fence's own lines, with that exact log shape as a permanent self-test case. The strongest desk check available is wired in: run against the real W40 log it must reproduce that boot's known split -- tier armed PASS, store delivery FAIL, exit 1 -- and it does. TESTS (hermetic; no boot; no card touched). CUDA_VISIBLE_DEVICES="" verified at the process and `nvidia-smi --query-compute-apps` empty throughout; GPUs 0/0/0. Runner reported its exclusions. * the script's own --self-test decides SIX cases in the intended direction, both polarities, including the false-PASS regression. * can-fail by mutation: un-scoping the acked regex reddens the real-boot reproduction and the self-test (2 red); restored green. * A SECOND SELF-INFLICTED DEFECT FOUND AND FIXED: the test located the script by counting `dirname` calls, landed on `test/` instead of the repo root, and failed all six cases -- a green arm red for a reason unrelated to the thing under test. It now walks up to the root. * partitioned tier-2 gate: 0 failing, 4421 passed (wide 3775 / narrow 292 / serial 354). Serial +6 over 348 is exactly the six tests added here. names=0 agrees with the summary on every lane. * test/registered/unit/mem_cache: 2 failed / 1802 passed -- the known pre-existing test_acceptance_emitters_758 RefillTiming pair, unchanged. POSTEN 4 -- boundary held. Nothing in schedule_batch.py, phase_flip_spill.py or seam_kv_recover was read into or written. Both Posten 2's root and A's remedy lead there; both are reported rather than resolved. NOT ESTABLISHED. Which cgroup bucket the phase-flip pin's bytes land in -- the question Posten 1 turns on -- needs the boot. Whether the admission ever refuses on this box, likewise. And sgl-project#721's 17 oom_kills are from a previous boot: today's counters are zero at every level, so nothing here re-measures that incident, it only records that the enforcement it implies is absent.
efschu
pushed a commit
to efschu/htsglang
that referenced
this pull request
Aug 26, 2026
THE QUESTION WAS "DOES sgl-project#721'S HOST-POSTEN-LEDGER EXIST", AND THE ANSWER IS (a) WITH TWO NAMED HOLES -- NEITHER OF WHICH IS WHAT sgl-project#721 ASKED FOR. IT EXISTS AND IT IS WIRED. `mem_cache/pinned_host_budget.py` is a named, individually-priced posts ledger, and it IS a hard boot precondition: three paths reach `joint_pinned_host_error` from `ServerArgs.__post_init__` and raise `ValueError` -- server_args.py:6937 -> _handle_kv_session_offload -> :8173 server_args.py:6962 -> _handle_phase_flip -> :8416 -> :8961 server_args.py:7019 -> _handle_hicache -> :8916 -> :9029 Four posts are priced there over CONFIGURED numbers (HiCache host tier, HiCache staging host tier x ranks, phase-flip rotation staging ring x ranks, kv-session-offload spill pool), and eight more register at runtime in the workers (memory_pool_host x4, read_buffer_pool, weights_arena, and the two halves of the phase-flip staging pin this branch added in db1b07e). So it is not (b) "nobody calls it" and not (c) "an intention". BUT sgl-project#721'S TWO PRESCRIPTIONS WERE BOTH DECLINED, AND BOTH DECLINES ARE RIGHT. * "16G floor against `free -g`" -- not implemented, and must not be: `free -g` is lxcfs-synthesised in this container. That was settled in #871b, which moved the boot gate onto the sgl-project#407 owner instead. * "sonst Posten schrumpfen" -- deliberately not implemented. The module's own charter: "The guard SUMS, it never caps. Nothing in this module shrinks a request. Silently capping a pinned pool moves the failure from a boot-time message that names both posts to a later allocation whose victim the OOM killer chooses." That is a better answer than the measure it declines, and it is the reason the ledger prices every post separately -- so an operator can shrink the right flag himself. THE ONE REAL DEFECT, AND IT IS A COUNTER DESCRIBED AS AN ACTUATOR (sgl-project#464/sgl-project#578/ sgl-project#852). `check_and_register_pinned_post` carried this, about the largest claimant in the system: "`_register_image_post` declares the post BEFORE `_alloc_host_image` allocates ... deliberately, so the registry refuses an over-commitment at the DECLARATION rather than discovering it at the allocation." THE REGISTRY REFUSES NOTHING THERE. `register_pinned_post` is a bare dict write with no comparison in it; `_register_image_post`'s own first line says "Never raises"; and `grep -c check_and_register_pinned_post weights_arena.py` -> 0. So the ~27 GiB of phase-flip host weight images are COUNTED, never ADMITTED. THE BEHAVIOUR IS CORRECT AND STAYS. sgl-project#695 chose it at the call site and wrote why: "Registered, not CHECKED: a new refusal path here could break a boot that works today, and the diagnosis this is for is served by the number being present, not by a veto." Registering early still makes those bytes visible to every LATER admission, which is the half that matters. Only the comment was wrong -- and a comment promising a veto that does not exist is worse than none, because the next person sizing a pinned pool against this module would leave exactly the headroom that post already spent. WHY A TEST AND NOT JUST A RE-COMMENT. Prose cannot be regression-tested; the CONTRACT can. The split is now pinned in both directions -- counting never refuses however absurd the post, admitting refuses and names every post, a refused post is not left in the registry, and weights_arena uses the counting entry point only. If someone later wires the admitting one in there, that is a real decision with a boot-refusal consequence and it surfaces here instead of in a boot. A RULE I PROVED ON MYSELF, THIRD INSTANCE IN ONE SESSION. The first version of the prose test asserted the false sentence was ABSENT from the module -- and failed against the FIXED file, because a correction has to QUOTE the claim it corrects. Same shape as #871a's floor-promise assertion and #871b's /proc/meminfo probe assertion. The general rule, now written into the test: ABSENCE OF PROSE IS NOT TESTABLE where a correction must restate what it corrects. Prose gets one positive marker; the behaviour tests carry the contract. THE WEIGHTS TRANSIENT, DECIDED WITH REASONS AS ASKED. It is a SPIKE, not a standing post, and the existing treatment is already the right compromise: it is registered at declaration (so concurrent posts see it), and taken back both when the allocation raises (weights_arena.py:1044) and when the image is freed (:1082). Booking it as a permanent resident would refuse too early; leaving it invisible would be the sgl-project#721 outcome. What it is NOT is admitted -- and turning that on is a behaviour change with a boot-refusal consequence that needs a loaded box to justify, not a desk. LOAD REMAINS UNMEASURED, AND THAT IS IN THE CODE, NOT ONLY HERE. #871b's 0.24 GiB delta between the raw and honest readings is an IDLE measurement. A precondition gate bites under load. No load figure is asserted anywhere as a threshold. TESTS (hermetic; no boot; no card touched). CUDA_VISIBLE_DEVICES="" verified at the process, `nvidia-smi --query-compute-apps` empty throughout, cards 0/0/0. * red-first: 1 of 6 red against the uncorrected module, extracted == summary. * can-fail in the danger direction: making the COUNTING entry point refuse (a counter grown into an actuator) reddens `test_counting_never_refuses_however_absurd_the_post`. Green restored. * turnkey + this suite together: 175 passed, 0 failed. * partitioned tier-2 gate: wide 3775 / narrow 292 / serial 353 passed, 24 exclusions reported by the runner, names=1 agrees with the summary on every lane. The single failure -- test_pp_admission_wraparound_never_blocks::test_blocking_wraparound_wedges_the_ring -- is PRE-EXISTING: on the clean base 726f11e that module fails TWO tests, i.e. MORE than here, and it touches nothing in this commit. * test/registered/unit/mem_cache: 2 failed / 1808 passed. The two are the known test_acceptance_emitters_758 RefillTiming pair; +6 over 1802 is exactly this suite. NOT ESTABLISHED. Whether the ~27 GiB image post SHOULD be admitted -- that needs a box tight enough for the answer to differ, i.e. load. Whether the four launcher posts plus eight runtime posts are the COMPLETE set of pinned claimants: I enumerated every `PinnedHostPost(` and every register call in the tree, but a pinned allocation that never posts at all is invisible to that method by construction. And ZFS cache, named in sgl-project#721's list, is kernel-side and cannot be a process post in this ledger at all.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Thank you for your contribution, we really appreciate it. The following instructions will help improve your pull request and make it easier to receive feedback. If there are any items you don't understand, don't worry. Just submit the pull request and ask the maintainers for help.
Motivation
Please explain the motivation behind this PR and the goal you aim to achieve with it.
Modification
Briefly describe the changes made in this PR.
Checklist
pre-commit run --all-filesor other linting tools are used to fix potential lint issues.