Skip to content

fix: init readthedocs support - #784

Merged
zhyncs merged 1 commit into
sgl-project:mainfrom
zhyncs:fix
Jul 28, 2024
Merged

zhyncs merged 1 commit into
sgl-project:mainfrom
zhyncs:fix

Conversation

@zhyncs

@zhyncs zhyncs commented Jul 28, 2024

Copy link
Copy Markdown
Contributor

Thank you for your contribution, we really appreciate it. The following instructions will help improve your pull request and make it easier to receive feedback. If there are any items you don't understand, don't worry. Just submit the pull request and ask the maintainers for help.

Motivation

Please explain the motivation behind this PR and the goal you aim to achieve with it.

Modification

Briefly describe the changes made in this PR.

Checklist

  1. Ensure pre-commit pre-commit run --all-files or other linting tools are used to fix potential lint issues.
  2. Confirm that modifications are covered by complete unit tests. If not, please add more unit tests for correctness.
  3. Modify documentation as needed, such as docstrings or example tutorials.

@zhyncs
zhyncs merged commit 27f5ce0 into sgl-project:main Jul 28, 2024
@zhyncs
zhyncs deleted the fix branch July 28, 2024 06:55
timethink pushed a commit to timethink/sglang that referenced this pull request Mar 9, 2025
efschu pushed a commit to efschu/htsglang that referenced this pull request Aug 21, 2026
…rdict on the wrong one

The acceptance verdict had no phase detection. It sampled one window and called
that the verdict. A phase-flip boot binds its KV pool in the PP prefill layout
-- that is where the id space is min-reduced across ranks and where the binding
rank's bracket runs down to its arming floor -- while the TP decode layout
inherits that id space through the canonical KV page, sizes nothing of its own,
and has released its arena tail, so its free column reads gibibytes higher.

On 2026-08-20 the sample window fell entirely inside TP. Rank 2 was graded at
5475 MiB while at rest in PP it sat at 3232 MiB against a 3226 MiB arming
floor: six MiB of slack. The boot was graded 2243 MiB away from the number that
binds it, on a layout that sizes nothing.

The sampler now buckets by the layout that was live when each sample was taken,
reading it from the presence marker the runtime already publishes
(phase_flip_presence.publish_active_phase / read_active_phase). The rule lives
in corridor_guard.phase_corridor_verdict so the script cannot drift from it:

  * the sizing phase decides;
  * other phases are reported as notes -- a card resting high in TP is the
    layout releasing its own arena tail, which is it working, not a sizing
    defect, and failing a boot for that is the instrument's error;
  * a near-OOM reading fails in ANY phase, because that one is about survival
    rather than efficiency;
  * over-band in the sizing phase fails, under-band warns, per the operator's
    asymmetric ruling;
  * a window that never observed the sizing phase returns NO-VERDICT and exits
    non-zero. An absent measurement must not be readable as a pass.

Replayed against the boot the defect was found on: the TP-only window that
produced the confident 2026-08-20 FAIL now returns NO-VERDICT, and once the PP
window is included the verdict names GPU0 and GPU1 as giving away 5094 and 3833
MiB in the phase that sizes the pool while exonerating GPU2, the binder, which
is correctly filled at its arming floor. That is the stranding the bug report
describes, now pointed at the right ranks.

Also folds the last duplicated constant: the script's local near-OOM literal
now comes from corridor_guard.NEAR_OOM_MIB.

Tests: 8 further cases in test_corridor_arming_credit_784.py, red first,
covering the sizing-phase identity, an unsampled sizing phase, over/under band
in each phase, and near-OOM in a non-sizing phase. 33 pass in that suite, 134
across it plus test_corridor_threshold_pair_656, test_arming_floor_funding_662
and test_nvml_carve_out_602. ruff check/format and codespell clean.
efschu pushed a commit to efschu/htsglang that referenced this pull request Aug 21, 2026
The derivation is graded PASS on all three ranks at error 0.00 MiB against
the same boot's measured totals (boot_735_tail785, commit fa7d387), so
it is now allowed to reach the solve.

Three sites, all on the cold path only:

- _maybe_price_cold_arena_tail sets arena_fixed_bytes on a COLD record
  from this boot's own layouts. A non-cold record, or a cold one that
  already carries a tail, is returned untouched -- this adds a number
  where there was none and never replaces a measurement.
- the arming floor read the draw only when reserve.active, and active
  requires an id-space ANCHOR a cold record cannot have. The anchor is
  genuinely absent; the tail is not, once derived. arming_draw_bytes
  already returns exactly the arena term when no per-leg measurement
  exists, so the condition widens rather than the arithmetic changing.
  This is what separates a cold arming floor of 1523 MiB from the real
  3226 on rank 2.
- the cold solve passed a literal 0 for solve_pool_tokens'
  arena_fixed_bytes. Its own log said "the fixed and arena terms are NOT
  charged"; the arena half of that sentence is no longer true and the log
  now says which term is charged and which is still missing.

The warm path does not move: a measured record already carries the tail,
and the derivation agrees with it exactly, so nothing about the currently
serving configuration changes size. What changes is the FIRST boot of any
configuration, which previously priced rank 2's tail at zero, solved
755649 tokens and OOMed when the NEXTN draft weights landed. That is the
two-boot cliff, and it is the reason a configuration change silently
sized against the previous configuration.

Tests: 7 new, including the can-fail guards -- a rank whose PP layout is
the smaller one must be charged nothing rather than a negative tail, and
the charge must be the SUBTRACTION rather than layout_pp itself (10789
instead of 2215 MiB would size a rank to almost nothing while looking
correctly priced). 46 passed across the sgl-project#784/sgl-project#785 family. ruff clean.
The 51 failures in the mem_cache+managers sweep are pre-existing: the
four named files run 12 failed / 10 passed both on this branch and on a
detached worktree at the merge base 332f083.
efschu pushed a commit to efschu/htsglang that referenced this pull request Aug 25, 2026
…ark transport batches

W37-G (boot_w37g_0825_1326.log, pin 0be4ff4) proved the #861j doors open
on metal -- SEAM TRANSPORT ADMITTED x3, TP decode formed at bs 5-6 with
accept ~2.4-2.7, per-rid tokens strictly rising -- and exposed the next gap:
all 27 TP transport batches recomputed cold (#cached-token: 0, the sgl-project#783
dark-radix shape defeating the premise at runtime) while the conformance
count stayed 0, because layout_conformance.work_layout_verdict -- built
after W37-D's 258 identical batches -- had ZERO callers. Desk-written-
never-executed, in the instrument meant to falsify exactly this shape.

* metrics_reporter: _note_work_layout wired into report_prefill_stats
  before the logging-rank gate (counters are rank-local); phase from the
  same authority as the phase label, strict from purity_of, fully
  try/except'd. Decode side deliberately not wired -- named in the
  docstring, not omitted silently.
* scheduler: the admission that rides the exemption stamps
  batch.is_seam_transport from the same transport_only local that spends
  the seam stamps, so the claim cannot drift from the admission.
* layout_conformance: the restore excuse is now cached_tokens > 0 (was
  AND new_tokens <= 0, which would have flagged every legitimate
  read-through -- a real restore still computes >=1 token per request).
  The defect signature is cached == 0, W37-D and W37-G both.

Red-proven in a temp worktree at HEAD: the cold-transport batch through the
REAL report_prefill_stats counts 0 ("the detector for exactly this shape
has no caller") and the build-site pin finds no transport mark; green on
this tree. Can-fails: genuine partial restore stays silent, ordinary
prefill-in-tp still flags, prefill-in-pp never flags.

Tests: test_tp_decode_formation_861j.py 13/13 (dark-radix arm added),
scoped impact 130/130 (conformance/economy/band/forward-pass/exemption/
servability), ruff delta 0, codespell delta 0. Full serial managers
battery: 8 failed / 4320 passed / 2 skipped / 356 subtests
in 722.41s (/tmp/861k_gate2.log), cards VISIBLE -- the Tier-2 managers gate
is not hermetic and CVD="" flips its verdict in both directions. Failure
set identical BY NAME to the standing 8: staging_reserve_631 x4,
rank_prefill_log::TestCollectiveClock x1, hisparse_unit x2,
draft_cuda_graph_removal x1. Unexpected names: none.

WURZEL. Klasse: desk-written-never-executed -- an instrument built, unit-
tested against hand-made inputs, and never wired to the path it judges, so
it reports green by never running; test-only coverage is what HIDES it.
Geschwister, swept tree-wide by AST over python/sglang/srt (143 instrument-
shaped defs, references counted outside the def line), sweep validated in
both directions (it lists work_layout_verdict with metrics_reporter.py
masked and goes silent with it wired): 10 members with zero production
callers. Three are load-bearing and all in managers --
  * progress_liveness.py: the WHOLE sgl-project#699 module has zero production
    importers (only its test file), including thrash_verdict (#861e), which
    has exactly one reference repo-wide, its own def line, and no test at
    all. That is the detector for "decode advances, completions do not" --
    the W37-D/W37-G shape and the sgl-project#857 acceptance discriminator.
  * corridor_guard.phase_corridor_verdict (sgl-project#784): 7 test call sites, zero
    production callers; the constants are imported widely, the verdict is
    not.
Zukunfts-Check: OWED and specified, not built here -- the sweep becomes a
registered test that fails when an instrument-shaped def has no production
caller, with an allowlist carrying an owner and a reason per dark entry.
Filed as #861l so this commit stays the wiring change. Full list and
file:lines in /spinning/gpu-arb/NOTE_861fg.md.
efschu pushed a commit to efschu/htsglang that referenced this pull request Aug 25, 2026
…ument surface

#861k wired one dark instrument. This closes its CLASS.

THE ROOT IS NOT "no check existed". test/registered/unit/test_unwired_features_421.py
already carries both the doctrine ("a reachability assertion is the only kind
of test that can see the gap between 'the unit works' and 'the product uses
it'") and the AST machinery. Its hole is its SHAPE: it is ENUMERATIVE -- one
hand-written pin per feature that the sgl-project#421 audit had already found. Every
instrument built after that audit is outside its reach BY CONSTRUCTION, which
is exactly how four of them went dark under a fully green suite:
work_layout_verdict (#861k), thrash_verdict (#861e), phase_corridor_verdict
(sgl-project#784), and the whole progress_liveness module (sgl-project#699).

This adds the EXHAUSTIVE complement: one AST pass over python/sglang, every
instrument-shaped def (147 of them), dark = the name appears nowhere in
production outside its own def line. The dark set must EQUAL a declared
allowlist, so a newly-dark instrument fails a gate run instead of waiting for
a metal boot to not-report something. Both files stay: sgl-project#421 pins named
absences with per-feature reasoning, this pins the membership of the set.

12 declared entries, each with a reason that must open with DARK (the class,
with a named owner) or EXEMPT (not an instrument) -- itself asserted. Three
EXEMPT (an upstream fp8 debug helper; two offline GGUF audit entry points,
one of which names its test caller in its own docstring). Nine DARK, filed
with owners, the two load-bearing ones being:
  * progress_liveness.thrash_verdict (#861e) -- before this gate it had ONE
    reference repo-wide, its own def line, and no test either. Its docstring
    describes W37-D verbatim ("decode_steps STRICTLY INCREASING, completions
    FLAT"), i.e. the W37-G shape and the sgl-project#857 acceptance discriminator. Its
    whole module is dark with it: nothing in production imports
    sglang.srt.managers.progress_liveness, so sgl-project#699's assess(),
    sample_from_scheduler() and build_liveness_is_active() are dark too --
    the module written because "/health answers is the process up, not is
    work moving" and "the watchdog is blind to admission wedges BY
    CONSTRUCTION" was never installed. The job is today done out-of-process
    by serving-liveness-monitor.sh: one payload, two movers, the in-process
    one inert. Reconciliation owed with the wiring.
  * corridor_guard.phase_corridor_verdict (sgl-project#784) -- 7 test call sites, zero
    production callers. The module's CONSTANTS are imported in four places,
    which is what makes it read as live; its verdict is computed by nobody.

THE GATE WAS TESTED AGAINST A KNOWN STATE IN BOTH DIRECTIONS BEFORE ITS
NUMBERS WERE USED, AND IT FAILED THE FIRST TIME. Indexing only ast.Name and
ast.Attribute, it called three LIVE checkers dark --
Scheduler._check_layout_policy_conformance, _check_layout_conformance and
PhaseFlipRuntime._census_ownership_audit -- all three reached through this
tree's deliberate getattr(self, "<name>", fallback)(...) idiom, where the
callee name exists only as a string literal. A gate that accuses live code is
worse than no gate. String constants now count as references, and the idiom
is pinned by a synthetic dispatch case so the defect cannot return.
Direction check: the sweep lists work_layout_verdict with metrics_reporter.py
masked and goes silent with it wired -- delta exactly one name.

Can-fail proofs are executable, not prose: four synthetic-tree cases (unwired
reported / wired silent / same-module caller counts as wired / string
dispatch counts as wired) plus a guard-the-guard assertion that the walk
found >100 instruments at all. The gate also fired unprompted on its first
real run, on jit_kernel/tests/kv_canary/_invariants.py::assert_all -- fixed
in the PATH filter ("/tests/", which the sgl-project#421 helper's "/test/" walks past)
rather than by an allowlist entry, because that file is not production source
and allowlisting it would have recorded a fiction.

Tests: 8/8 new, plus test_unwired_features_421.py 6/6 unaffected (14 passed
in 89.9s). No production file touched, so the #861k managers battery
(8 failed / 4320 passed, standing set by name) still stands. ruff, codespell,
black, isort: clean.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant