Skip to content

Format - #593

Merged
Ying1123 merged 1 commit into
mainfrom
format
Jul 5, 2024
Merged

Format#593
Ying1123 merged 1 commit into
mainfrom
format

Conversation

@Ying1123

@Ying1123 Ying1123 commented Jul 5, 2024

Copy link
Copy Markdown
Contributor

No description provided.

@Ying1123
Ying1123 merged commit dc1b8bc into main Jul 5, 2024
@Ying1123
Ying1123 deleted the format branch July 5, 2024 17:06
timethink pushed a commit to timethink/sglang that referenced this pull request Mar 9, 2025
efschu pushed a commit to efschu/htsglang that referenced this pull request Aug 5, 2026
efschu pushed a commit to efschu/htsglang that referenced this pull request Aug 6, 2026
…priced per rank into the sgl-project#593 reserve and sgl-project#602 corridor late terms — inherited 70 MiB honestly labelled CALIBRATED with a measured replacement path via the flight recorder; communicator_groups declaration corrected both directions with an AST contract test against parallel_state; runbook 16.8 reserve semantics + 16.9 fill side documented OPEN)
efschu pushed a commit to efschu/htsglang that referenced this pull request Aug 13, 2026
…ng the ledger

TASK 1 (weights on the pin path) and TASK 4 (residual recalibration).

COMPLETENESS CHECK, red-first. reconcile.completeness_failures() names any
card whose model-weights term is priced at zero or absent. Run against the
real shipped dump ledger_1464299-1786612548.json it fails on all three cards,
which is the red the brief asked for:

  model weights (shards) on GPU 0/1/2: priced at 0 MiB. This is the PIN PATH
  signature -- pinning --rank-gpu-memory-mib skips the planner that computes
  the shard vector, so the term is built and dumped as a zero rather than
  refused.

A REFUSAL PASSES THE CHECK. Only a silent zero fails. That distinction is the
whole point: a zero is indistinguishable from a price.

WEIGHTS ON THE PIN PATH. _build_card_ledgers now accepts weight_mib_per_rank
and history; both default to None and None reproduces the old behaviour
exactly, so the sgl-project#593 sizing paths are byte-identical. Only the observation
ledger passes them -- a sizing path must not change what a boot reserves as a
side effect of an instrument being armed.

PIPELINE PARALLELISM RETURNS None, NOT A NUMBER. The ship configuration is
pp_size=3 / tp_size=1 with --pp-layer-ratio 28,20,16, so the split is a LAYER
split, and per-layer bytes are not uniform on a hybrid Qwen3.5/3.6 checkpoint
(interleaved GDN and full attention). Solving the measured episode-1 posts
13674/8325/9293 MiB for a uniform per-layer cost gives 488/416/581 MiB per
layer -- the model does not close, so it is not shipped. None reaches the
ledger as no term at all, which the completeness check then names.

HISTORY INTO THE LEDGER. build_card_ledgers(history=...) gives measured-over-
inherited precedence for the two terms the boot history can speak to, and
refuses rather than averages a wide one:
  hardware residual  band charged at its HIGH, superseding the window-
                     2026-08-06 probe (664 -> 902 MiB on the 5090)
  load transient     REFUSED and UNBOUNDED; the inherited 70 MiB is
                     explicitly NOT used as a fallback
A refused residual band keeps the probe calibration: a wide history is a
reason to distrust a summary of the history, not to discard a direct
measurement.

Tests: 15 new across test_history_into_ledger_605.py (8) and
test_pin_path_weights_605.py (7), all red first. The byte-identical-default
test is the one protecting every existing caller.
Suite: 531 passed, 1 failed (inherited test_communicator_group_contract_612,
also failing at base cd71ec3). black 26.1.0 clean; ruff 0 on every touched
file, same as base.
efschu added a commit to efschu/htsglang that referenced this pull request Aug 17, 2026
… the sizer must NOT claim it

VERDICT: no dead reservation found. The gap is the arming floor (deliberate)
plus min-rule residue (structural), so this is the labeled accounting table the
brief asks for in that case, not a forced fix.

Per-rank, from the sgl-project#704/sgl-project#707 instrument chain rather than modelled:

  rank  budget  weights  mamba    rest  holdback  allowed   used   slack
  PP0   31800  16064.5   916.5  14819.5  6690.1   8129.5  5964.7  2164.8
  PP1   18800  10061.8   654.3   8083.4  3562.7   4520.7  4260.5   260.2
  PP2   19800  10699.8   523.3   8576.3  5167.9   3408.4  3408.4     0.0

Total slack 2,425 MiB, and the shape is the finding: it is almost all on PP0,
and the BINDER PP2 has exactly zero.

HOLDBACK is deliberate, established in sgl-project#707: the pool is capped so the resting
free column still holds the arming floor. Sizing below it is the boot that holds
the corridor and never flips (sgl-project#656 E/G). Not claimable.

SLACK is structural and still not claimable BY THE SIZER. The pool is one global
token count, so a non-binding rank cannot spend its extra allowance without the
binder moving -- PP0's 2,164.8 MiB is not booked-and-untouched, it is
unreachable at this cut. A sizer that claimed it would hand the pool tokens PP2
cannot back, and PP2 is the rank that OOMs. That is the sgl-project#593-family direction
the brief warns about, so I wrote NO red-first test for it: the falsifier would
have to assert a wrong answer.

WHERE THE CAPACITY ACTUALLY IS: the slack is a property of the CUT, and that is
already solved on another strand. sgl-project#702 rev5 found the incumbent [28,20,16] is
not pool-optimal -- the binding rank switches, so moving layers OFF the binder
raises the pool, and [30,18,16] gives +20 percent AND 1.11x pipelined prefill.
Claim it by rebalancing the cut, not by relaxing a reserve.

WHAT I DO NOT CLAIM: the ticket's 2.0 / 5.7 / 3.7 GiB per card is a DIFFERENT
operating point from the instrumented boot, and the instruments that would
attribute it per-term only exist on boots carrying 2a6305d / 5f3e61f /
f55c1a8. Mapping this table onto those three numbers is the cross-boot
arithmetic this strand has already retracted twice, so I did not. Ask for the
next boot to capture the three instrument lines at that operating point and the
table re-runs against it directly -- a window item.

One thing that CAN be said without it: against the measured arming floors
(1728 / 1825 / 2467 MiB) the per-card free exceeds the ~1024 MiB corridor target
BY CONSTRUCTION, because on a flip-enabled boot the arming floor is the binding
level, not the corridor. Reading the gap against 1024 overstates it on every
rank.

Docs only. No sizer change, deliberately.
arathi-hlab added a commit to arathi-hlab/sglang that referenced this pull request Nov 4, 2026
sgl-kernel-xpu#593 moves Dockerfile.xpu_kernel from Miniforge to a uv venv
at /opt/venv. This workflow builds that Dockerfile from the kernel release
branch and then activates the conda env, so the first release branch cut
after sgl-project#593 would fail at `. /miniforge3/bin/activate`.

Activate /opt/venv when the image has it, else fall back to the conda env
so older release branches still build. Assert the active Python matches
the matrix version so a wrong env fails fast.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant