Repository navigation
fix: init readthedocs support - #784
Merged
Merged
Conversation
timethink
pushed a commit
to timethink/sglang
that referenced
this pull request
Mar 9, 2025
efschu
pushed a commit
to efschu/htsglang
that referenced
this pull request
Aug 21, 2026
…rdict on the wrong one
The acceptance verdict had no phase detection. It sampled one window and called
that the verdict. A phase-flip boot binds its KV pool in the PP prefill layout
-- that is where the id space is min-reduced across ranks and where the binding
rank's bracket runs down to its arming floor -- while the TP decode layout
inherits that id space through the canonical KV page, sizes nothing of its own,
and has released its arena tail, so its free column reads gibibytes higher.
On 2026-08-20 the sample window fell entirely inside TP. Rank 2 was graded at
5475 MiB while at rest in PP it sat at 3232 MiB against a 3226 MiB arming
floor: six MiB of slack. The boot was graded 2243 MiB away from the number that
binds it, on a layout that sizes nothing.
The sampler now buckets by the layout that was live when each sample was taken,
reading it from the presence marker the runtime already publishes
(phase_flip_presence.publish_active_phase / read_active_phase). The rule lives
in corridor_guard.phase_corridor_verdict so the script cannot drift from it:
* the sizing phase decides;
* other phases are reported as notes -- a card resting high in TP is the
layout releasing its own arena tail, which is it working, not a sizing
defect, and failing a boot for that is the instrument's error;
* a near-OOM reading fails in ANY phase, because that one is about survival
rather than efficiency;
* over-band in the sizing phase fails, under-band warns, per the operator's
asymmetric ruling;
* a window that never observed the sizing phase returns NO-VERDICT and exits
non-zero. An absent measurement must not be readable as a pass.
Replayed against the boot the defect was found on: the TP-only window that
produced the confident 2026-08-20 FAIL now returns NO-VERDICT, and once the PP
window is included the verdict names GPU0 and GPU1 as giving away 5094 and 3833
MiB in the phase that sizes the pool while exonerating GPU2, the binder, which
is correctly filled at its arming floor. That is the stranding the bug report
describes, now pointed at the right ranks.
Also folds the last duplicated constant: the script's local near-OOM literal
now comes from corridor_guard.NEAR_OOM_MIB.
Tests: 8 further cases in test_corridor_arming_credit_784.py, red first,
covering the sizing-phase identity, an unsampled sizing phase, over/under band
in each phase, and near-OOM in a non-sizing phase. 33 pass in that suite, 134
across it plus test_corridor_threshold_pair_656, test_arming_floor_funding_662
and test_nvml_carve_out_602. ruff check/format and codespell clean.
efschu
pushed a commit
to efschu/htsglang
that referenced
this pull request
Aug 21, 2026
The derivation is graded PASS on all three ranks at error 0.00 MiB against the same boot's measured totals (boot_735_tail785, commit fa7d387), so it is now allowed to reach the solve. Three sites, all on the cold path only: - _maybe_price_cold_arena_tail sets arena_fixed_bytes on a COLD record from this boot's own layouts. A non-cold record, or a cold one that already carries a tail, is returned untouched -- this adds a number where there was none and never replaces a measurement. - the arming floor read the draw only when reserve.active, and active requires an id-space ANCHOR a cold record cannot have. The anchor is genuinely absent; the tail is not, once derived. arming_draw_bytes already returns exactly the arena term when no per-leg measurement exists, so the condition widens rather than the arithmetic changing. This is what separates a cold arming floor of 1523 MiB from the real 3226 on rank 2. - the cold solve passed a literal 0 for solve_pool_tokens' arena_fixed_bytes. Its own log said "the fixed and arena terms are NOT charged"; the arena half of that sentence is no longer true and the log now says which term is charged and which is still missing. The warm path does not move: a measured record already carries the tail, and the derivation agrees with it exactly, so nothing about the currently serving configuration changes size. What changes is the FIRST boot of any configuration, which previously priced rank 2's tail at zero, solved 755649 tokens and OOMed when the NEXTN draft weights landed. That is the two-boot cliff, and it is the reason a configuration change silently sized against the previous configuration. Tests: 7 new, including the can-fail guards -- a rank whose PP layout is the smaller one must be charged nothing rather than a negative tail, and the charge must be the SUBTRACTION rather than layout_pp itself (10789 instead of 2215 MiB would size a rank to almost nothing while looking correctly priced). 46 passed across the sgl-project#784/sgl-project#785 family. ruff clean. The 51 failures in the mem_cache+managers sweep are pre-existing: the four named files run 12 failed / 10 passed both on this branch and on a detached worktree at the merge base 332f083.
efschu
pushed a commit
to efschu/htsglang
that referenced
this pull request
Aug 25, 2026
…ark transport batches W37-G (boot_w37g_0825_1326.log, pin 0be4ff4) proved the #861j doors open on metal -- SEAM TRANSPORT ADMITTED x3, TP decode formed at bs 5-6 with accept ~2.4-2.7, per-rid tokens strictly rising -- and exposed the next gap: all 27 TP transport batches recomputed cold (#cached-token: 0, the sgl-project#783 dark-radix shape defeating the premise at runtime) while the conformance count stayed 0, because layout_conformance.work_layout_verdict -- built after W37-D's 258 identical batches -- had ZERO callers. Desk-written- never-executed, in the instrument meant to falsify exactly this shape. * metrics_reporter: _note_work_layout wired into report_prefill_stats before the logging-rank gate (counters are rank-local); phase from the same authority as the phase label, strict from purity_of, fully try/except'd. Decode side deliberately not wired -- named in the docstring, not omitted silently. * scheduler: the admission that rides the exemption stamps batch.is_seam_transport from the same transport_only local that spends the seam stamps, so the claim cannot drift from the admission. * layout_conformance: the restore excuse is now cached_tokens > 0 (was AND new_tokens <= 0, which would have flagged every legitimate read-through -- a real restore still computes >=1 token per request). The defect signature is cached == 0, W37-D and W37-G both. Red-proven in a temp worktree at HEAD: the cold-transport batch through the REAL report_prefill_stats counts 0 ("the detector for exactly this shape has no caller") and the build-site pin finds no transport mark; green on this tree. Can-fails: genuine partial restore stays silent, ordinary prefill-in-tp still flags, prefill-in-pp never flags. Tests: test_tp_decode_formation_861j.py 13/13 (dark-radix arm added), scoped impact 130/130 (conformance/economy/band/forward-pass/exemption/ servability), ruff delta 0, codespell delta 0. Full serial managers battery: 8 failed / 4320 passed / 2 skipped / 356 subtests in 722.41s (/tmp/861k_gate2.log), cards VISIBLE -- the Tier-2 managers gate is not hermetic and CVD="" flips its verdict in both directions. Failure set identical BY NAME to the standing 8: staging_reserve_631 x4, rank_prefill_log::TestCollectiveClock x1, hisparse_unit x2, draft_cuda_graph_removal x1. Unexpected names: none. WURZEL. Klasse: desk-written-never-executed -- an instrument built, unit- tested against hand-made inputs, and never wired to the path it judges, so it reports green by never running; test-only coverage is what HIDES it. Geschwister, swept tree-wide by AST over python/sglang/srt (143 instrument- shaped defs, references counted outside the def line), sweep validated in both directions (it lists work_layout_verdict with metrics_reporter.py masked and goes silent with it wired): 10 members with zero production callers. Three are load-bearing and all in managers -- * progress_liveness.py: the WHOLE sgl-project#699 module has zero production importers (only its test file), including thrash_verdict (#861e), which has exactly one reference repo-wide, its own def line, and no test at all. That is the detector for "decode advances, completions do not" -- the W37-D/W37-G shape and the sgl-project#857 acceptance discriminator. * corridor_guard.phase_corridor_verdict (sgl-project#784): 7 test call sites, zero production callers; the constants are imported widely, the verdict is not. Zukunfts-Check: OWED and specified, not built here -- the sweep becomes a registered test that fails when an instrument-shaped def has no production caller, with an allowlist carrying an owner and a reason per dark entry. Filed as #861l so this commit stays the wiring change. Full list and file:lines in /spinning/gpu-arb/NOTE_861fg.md.
efschu
pushed a commit
to efschu/htsglang
that referenced
this pull request
Aug 25, 2026
…ument surface
#861k wired one dark instrument. This closes its CLASS.
THE ROOT IS NOT "no check existed". test/registered/unit/test_unwired_features_421.py
already carries both the doctrine ("a reachability assertion is the only kind
of test that can see the gap between 'the unit works' and 'the product uses
it'") and the AST machinery. Its hole is its SHAPE: it is ENUMERATIVE -- one
hand-written pin per feature that the sgl-project#421 audit had already found. Every
instrument built after that audit is outside its reach BY CONSTRUCTION, which
is exactly how four of them went dark under a fully green suite:
work_layout_verdict (#861k), thrash_verdict (#861e), phase_corridor_verdict
(sgl-project#784), and the whole progress_liveness module (sgl-project#699).
This adds the EXHAUSTIVE complement: one AST pass over python/sglang, every
instrument-shaped def (147 of them), dark = the name appears nowhere in
production outside its own def line. The dark set must EQUAL a declared
allowlist, so a newly-dark instrument fails a gate run instead of waiting for
a metal boot to not-report something. Both files stay: sgl-project#421 pins named
absences with per-feature reasoning, this pins the membership of the set.
12 declared entries, each with a reason that must open with DARK (the class,
with a named owner) or EXEMPT (not an instrument) -- itself asserted. Three
EXEMPT (an upstream fp8 debug helper; two offline GGUF audit entry points,
one of which names its test caller in its own docstring). Nine DARK, filed
with owners, the two load-bearing ones being:
* progress_liveness.thrash_verdict (#861e) -- before this gate it had ONE
reference repo-wide, its own def line, and no test either. Its docstring
describes W37-D verbatim ("decode_steps STRICTLY INCREASING, completions
FLAT"), i.e. the W37-G shape and the sgl-project#857 acceptance discriminator. Its
whole module is dark with it: nothing in production imports
sglang.srt.managers.progress_liveness, so sgl-project#699's assess(),
sample_from_scheduler() and build_liveness_is_active() are dark too --
the module written because "/health answers is the process up, not is
work moving" and "the watchdog is blind to admission wedges BY
CONSTRUCTION" was never installed. The job is today done out-of-process
by serving-liveness-monitor.sh: one payload, two movers, the in-process
one inert. Reconciliation owed with the wiring.
* corridor_guard.phase_corridor_verdict (sgl-project#784) -- 7 test call sites, zero
production callers. The module's CONSTANTS are imported in four places,
which is what makes it read as live; its verdict is computed by nobody.
THE GATE WAS TESTED AGAINST A KNOWN STATE IN BOTH DIRECTIONS BEFORE ITS
NUMBERS WERE USED, AND IT FAILED THE FIRST TIME. Indexing only ast.Name and
ast.Attribute, it called three LIVE checkers dark --
Scheduler._check_layout_policy_conformance, _check_layout_conformance and
PhaseFlipRuntime._census_ownership_audit -- all three reached through this
tree's deliberate getattr(self, "<name>", fallback)(...) idiom, where the
callee name exists only as a string literal. A gate that accuses live code is
worse than no gate. String constants now count as references, and the idiom
is pinned by a synthetic dispatch case so the defect cannot return.
Direction check: the sweep lists work_layout_verdict with metrics_reporter.py
masked and goes silent with it wired -- delta exactly one name.
Can-fail proofs are executable, not prose: four synthetic-tree cases (unwired
reported / wired silent / same-module caller counts as wired / string
dispatch counts as wired) plus a guard-the-guard assertion that the walk
found >100 instruments at all. The gate also fired unprompted on its first
real run, on jit_kernel/tests/kv_canary/_invariants.py::assert_all -- fixed
in the PATH filter ("/tests/", which the sgl-project#421 helper's "/test/" walks past)
rather than by an allowlist entry, because that file is not production source
and allowlisting it would have recorded a fiction.
Tests: 8/8 new, plus test_unwired_features_421.py 6/6 unaffected (14 passed
in 89.9s). No production file touched, so the #861k managers battery
(8 failed / 4320 passed, standing set by name) still stands. ruff, codespell,
black, isort: clean.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Thank you for your contribution, we really appreciate it. The following instructions will help improve your pull request and make it easier to receive feedback. If there are any items you don't understand, don't worry. Just submit the pull request and ask the maintainers for help.
Motivation
Please explain the motivation behind this PR and the goal you aim to achieve with it.
Modification
Briefly describe the changes made in this PR.
Checklist
pre-commit run --all-filesor other linting tools are used to fix potential lint issues.