Repository navigation
docs: update doc - #713
Merged
Merged
docs: update doc#713
Conversation
timethink
pushed a commit
to timethink/sglang
that referenced
this pull request
Mar 9, 2025
cen121212
pushed a commit
to cen121212/sglang
that referenced
this pull request
Nov 10, 2025
* Update test_utils.py * Update test_utils.py * Update test_utils.py * Update run_suite.py * Update run_suite.py * Uncomment test_original_logprobs.py in test suite * Update pr-test-npu-debug.yml * Fix variable name from exit_code to ret_code * Update run_suite.py * Update run_suite.py * Update daily-build-test-npu-innersource.yml * Update run_suite.py * Increase test timeout from 60 to 120 minutes * Update daily-build-test-npu-innersource.yml
efschu
pushed a commit
to efschu/htsglang
that referenced
this pull request
Aug 17, 2026
… is not the bug
MEASURED: a TEN-token prompt waited 31.64 s to first token. Sampled every 2 s
across the whole wait -- 0 running, 1 queued, mamba_available 3 (never zero,
0/16 samples), KV hundreds of thousands free -- while the policy logged
"BOTH BLOCKED ... 0 req resident, 22 tok pending". An 8-arm run put every TTFT
between 11.87 s and 62.65 s with NOT ONE arm under 3 s, so this is the serving
floor rather than an outlier.
THE CANDIDATE I CARRIED IN IS DEAD. I suspected the message's pending count and
the simulation's disagreed. They cannot: _idle_locked_inputs receives ONE
pending_tokens argument and passes the SAME value to both _layout_admits calls,
and the policy message prints the same figure. No denominator divergence exists.
AND THE SIMULATION IS CORRECT. Replaying _layout_admits with exactly the
measured numbers returns pp=True / tp=False, so _idle_locked_inputs would have
returned (nothing_can_run=True, target_can_admit=True) and the policy would have
ARMED THE FLIP. The refusal is therefore not a bad rule; the values the
simulation reads in-process differ from what /metrics reports, and no amount of
external sampling can show which -- I already spent a measurement window
proving that. The terms have to be printed where they are computed.
So this commit adds the instrument, not a fix. One line, on the refusal branch
only (both layouts declined), rate limited to 5 s because the state persists
for tens of seconds and healthy rounds must stay silent. It names every term the
verdict was computed from: phase, running_bs, pending_tokens, both layout
verdicts, post_evict_rows, allocator availability, mamba slots, chunk size --
and reports an accessor that RAISES as "RAISED <type>" rather than silently
becoming 0, since a raising probe is one of the two ways slots can read zero.
Tests, hermetic (CUDA_VISIBLE_DEVICES=""):
test_pp_admits_the_measured_idle_state ..... the complaint in one assertion
test_tp_correctly_refuses_the_same_state ... counterweight: tp MUST refuse,
else nothing_can_run is false and the refusal would be correct
test_the_pair_would_have_armed_the_flip .... (True, True) on the real state
starved pool / no state slot / no pending work still refuse -- the refusal
must stay reachable for its genuine causes
test_double_refusal_prints_every_term
test_silent_when_the_target_can_admit ...... CAN-FAIL: a diagnostic that
narrates healthy rounds is noise, and noise is how a signal gets filtered
-> 8 passed; 118 passed + 7 subtests across 713 + 708 + 701 + 677 + 689 +
flip-runtime. ruff clean in the changed region.
ADJACENCY, named not absorbed (sgl-project#689 window formation, sgl-project#677 window economics):
both suites are in the run above and pass. This touches neither -- it adds no
branch to the arming decision and changes no input to it.
efschu
pushed a commit
to efschu/htsglang
that referenced
this pull request
Aug 17, 2026
…ts is unreachable Pre-approved review amendment: _post_evict_rows() was the one undefended term inside the diagnostic's logger call, and a raise there would kill the scheduler round the line exists to observe -- the sgl-project#715 failure mode exactly. It now uses the same try/except-RAISED pattern as the other two probes. BUT THE MUTATION SAYS THE CONCERN IS UNREACHABLE, AND I WILL NOT DRESS THAT UP. My first test for it broke the accessors underneath _post_evict_rows and passed 10/10 against a deliberately UNDEFENDED call site -- a vacuous test, caught only because I mutation-check every can-fail. The reason: _post_evict_rows swallows its own accessor exceptions internally, so it cannot raise. Forcing a raise means patching the bound method, which then explodes inside _layout_admits (also a bare call) before the diagnostic is ever reached -- testing neither. So: the hardening stays, as cheap insurance against a future edit that makes _post_evict_rows raise, and because a uniform pattern is worth more than a justified exception. It is NOT claimed to fix a reachable defect today, the unreachable arm is removed rather than left as a passing test that proves nothing, and the limit is written into the test class docstring so the next reader does not re-derive it. The mamba probe IS reachable and remains tested: a raising accessor must be NAMED as "RAISED <type>", never silently read as 0, because zero-by-exception and zero-by-measurement are different states and only one is a defect. Tests, hermetic: 10 passed + 1 subtest; and the 677/689/708 adjacency suites pass alongside. ruff clean.
efschu
pushed a commit
to efschu/htsglang
that referenced
this pull request
Aug 17, 2026
ruff F401 on the previous commit: removing the unreachable _post_evict_rows arm orphaned its Scheduler import. Caught by running ruff, not by reading -- which is why the validator runs before every commit rather than after the ones that look risky.
efschu
pushed a commit
to efschu/htsglang
that referenced
this pull request
Aug 17, 2026
…e-key its own pin Merging Slot-3's sgl-project#713 admission-intake work surfaced two defects of mine, both of which I had already learned once in this same series. 1. THE STAND-IN TRAP, third time. sgl-project#708 wired the policy input as a direct call, self._uniform_kv_available(). Slot-3's scheduler STAND-INS carry only the fields the policy reads, so the merge produced nine failures with "'S' object has no attribute '_uniform_kv_available'". _idle_locked_inputs and the both-blocked relief were both given getattr defaults earlier for exactly this reason; I then wrote new code without applying my own lesson. Now getattr(self, "_uniform_kv_available", lambda: None)() -- a stand-in without the probe has MEASURED NOTHING, and "not measured" is a state the sgl-project#708 branch already reports honestly, so the degradation is free. 2. MY OWN PIN PUNISHED THE FIX. test_scheduler_passes_kv_available_tokens asserted the literal string "kv_available_tokens=self._uniform_kv_available()" and so went red on the legal refactor above -- the same way the sgl-project#698 pin broke when that call was made defensive. Re-keyed on the IDENTIFIER plus the parameter name rather than the call form. A pin must survive a legal refactor of the thing it pins, or it punishes the repair instead of the regression. Also recorded honestly: I pushed the merge before reading its test result, because I chained the push after pytest with ';' rather than '&&'. The branch carried nine failures for a few minutes. Sequence the push behind the tests. Tests: 190 passed + 21 subtests across admission-intake, phase-policy, 708, 713, 699 and flip-runtime. ruff clean in the changed region.
efschu
pushed a commit
to efschu/htsglang
that referenced
this pull request
Aug 17, 2026
… directions MEASURED (sgl-project#713 quantisation table, 06:19). Demand-PULL already worked: C1 arrived 06:19:06.56 and was served by the tp_to_pp at :09 -- 3.14 s, one seam plus overhead. What failed was the step AFTER. The box flipped back at :12, so C2, arriving 06:19:09.71 just as PP began, was not served until :15: arm arrive first token TTFT served by C1 06:19:06.56 06:19:09.71 3.14 tp_to_pp @:09 C2 06:19:09.71 06:19:15.52 5.81 MISSED :12, waited @:15 C3 06:19:15.52 06:19:15.63 0.11 landed ON the :15 flip The layout left on a timer while the prefill that pulled it was still unserved, so TTFT quantised to whole cycles -- 0.1 / 3.1 / 5.9 s, nothing between. The same arriving-tokens signal that PULLS a cutover must also HOLD it until the work it pulled for is served. One rule, two directions, not two rules. LEVER (a) IDLE DWELL IS DELIBERATELY NOT BUILT, on the coordinator's recorded call, because the measurement removed its target: the idle box does not churn. Pre-merge showed ONE flip across 10 idle minutes; post-merge ZERO across 90 s idle (0 running, 0 queued). My earlier "10 flips in 30 s on an idle box" was measured while my own arms were arriving -- the box was not idle and I called it idle. A gate with no measured trigger is dead weight. BOUNDED BOTH WAYS, because an unbounded hold is a starvation bug wearing a fix's clothes -- it trades the prefill side's starvation for the decode side's: (a) hold cannot starve decode: past LAYOUT_HOLD_MAX_ROUNDS the layout releases even with prefill unserved, and says EXHAUSTED; (b) pull cannot preempt an unserved prefill: while a hold is live in PP the verdict stays a hold, whatever the decode queue looks like. The both-sides tie (work pending in PP and TP at once) is routed to the sgl-project#677 economic comparison rather than to the timer, and is bounded by the same rounds. Bound = 8 rounds, derived not round: a cutover costs ~2.6 s of seam (sgl-project#690 measured 2.56-2.59 s), so 8 rounds holds the layout well under the cost of leaving it. Holding longer than it costs to leave is never worth it. SAFETY PRECEDENCE: never mid-flip (a cutover in progress owns the layout); never against an unfunded seam (a pull that cannot pay is an abandon). Tests, hermetic (CUDA_VISIBLE_DEVICES=""): the C2 specimen holds; pp releases when nothing pends (can-fail: the hold must not become a permanent stay); tp pulls; mid-flip and unfunded outrank demand in both phases; both-sides names the tie; the bound releases at the limit and NOT before (can-fail: early release evaporates the C2 fix the moment any decode appears); degenerate inputs -> 14 passed + 17 subtests; 185 passed + 24 subtests with phase-policy, 708, admission-intake and flip-runtime. ruff clean. CAN-FAIL PROVEN BY MUTATION IN THREE DIRECTIONS: never-holds fails 13 of 14, unbounded-hold fails 1, ignores-unfunded fails 2. Reverted and re-verified. NOT WIRED into decide() yet -- this is the decision rule. Wiring changes live flip behaviour and wants its own review; the acceptance (quantisation table collapses) can only be measured once it is wired, so that is the next step and not a claim of this commit.
efschu
pushed a commit
to efschu/htsglang
that referenced
this pull request
Aug 17, 2026
…cy was blind to
_pending_prefill_tokens saw three sources -- the waiting queue, the chunked
slot, and the recv batch -- and nowhere else. An ADMITTED request has LEFT the
waiting queue, so unless it is the chunked_req it is invisible, and the policy
reads 0 prefill pending while holding exactly that work. The arm states the
contradiction in its own words:
IDLE-LOCKED: no batch of either work class can be built in the pp layout
(1 REQ RESIDENT, 0 TOK PREFILL PENDING)
MEASURED (specimen D2, 2026-08-17): arrived 06:53:59.344, INSIDE the :59->:01
PP window with ~1.7 s of PP remaining -- ample for a ten-token prompt. Invisible
at the :01 arm, first token not until :04.879: a full ~5.5 s cycle late. So
pp_to_tp armed CORRECTLY by its own inputs and took the layout from work it
could not see. The defect was the inputs, not the rule.
SAME EXTENT LOGIC AS THE CHUNKED TERM, deliberately -- origin_input_ids minus
the filled prefix. No new notion of progress is introduced, and extend_range is
already set at admission, so an admitted-unprefilled request carries one.
UNKNOWN PROGRESS COUNTS AS ZERO, and the direction is chosen rather than
incidental. Over-counting would keep pending above 0 forever, pull and hold the
policy toward PP permanently, and starve decode -- bounded only by the SLO cap.
Under-counting merely restores today's behaviour for that request. A missing
extend_range is therefore treated as fully prefilled.
Tests, hermetic (CUDA_VISIBLE_DEVICES=""):
test_a_resident_unprefilled_request_is_counted ......... (a) the D2 shape
test_a_partially_prefilled_resident_counts_only_the_remainder
test_a_fully_prefilled_resident_counts_zero ............ (b) STARVATION GUARD
test_unknown_progress_counts_as_zero_not_as_a_whole_prompt
test_the_chunked_req_is_not_double_counted ............. (c) no double count
test_waiting_queue_and_resident_sum
test_no_resident_batch_is_harmless
test_a_broken_resident_entry_cannot_break_the_round
-> 8 passed; 211 passed + 17 subtests across 713, admission-intake,
phase-policy, 677 lever, idle-locked-arm and flip-runtime. ruff clean.
CAN-FAIL PROVEN BY MUTATION IN BOTH DIRECTIONS: removing the fix fails 3 of 8,
and making it count unconditionally (the starvation direction) also fails 3 of
8. Reverted and re-verified green after each.
Production diff is 12 functional lines, wrapped so an observation can never
raise into the scheduler loop.
efschu
pushed a commit
to efschu/htsglang
that referenced
this pull request
Aug 17, 2026
… acceptance lacked The sgl-project#713(a) boot acceptance passed every criterion and then the CONTROL build, which does not contain the fix, passed them all too. A criterion the control also satisfies cannot fail, so its passing said nothing about the patch. This adds the arm that is expected to fail, and pins it with tests. WHAT THE RECORDED LOGS SHOW (16 rotations, 2026-08-17) An IDLE-LOCKED flip ping-pong. Each layout certifies the other as the escape: the tp side reads the request as not resident with tokens still pending, the pp side reads it as resident with nothing pending, so both arm toward the other. Measured alternating runs: 72 arms / 299 s, 12 / 31 s, 10 / 27 s (twice), 3. This is the missing explanation for the sgl-project#713 TTFT quantisation. The unit of delay is one whole cutover -- median 2864 ms tp_to_pp and 2772 ms pp_to_tp over 486 flips -- and the 06:19 table's 0.1 / 3.1 / 5.9 s levels are how many cutovers the request sat through. That table was taken during a 27 s run. Trigger: an idle tp layout into which a small prefill lands as the resident count reaches 0. Pending 22 and 81 sustained runs; 63, 178, 564 bounced once or twice; every arm with pending >= 69,710 never chained. Termination: never a client disconnect -- runs end when enough requests accumulate that a decode batch with #running-req > 0 finally executes. Mechanism, from phase_policy.py: the IDLE_LOCKED branch sits above the min-dwell check and bypasses it on the strength of a written claim that it "CANNOT OSCILLATE ... after the flip the target runs by premise". The premise fails because the policy evaluates on the first round after a cutover, before the carried request is re-admitted (_idle_locked_inputs is gated on _round_built_nothing, which a just-cut-over layout trivially satisfies). The target is observed in its empty transient and armed straight back, with the only damper bypassed. WHAT IS REFUTED, AND STATED AS SUCH Not one request re-prefilled per half-cycle. The Prefill batch shape is identical every cycle (22 new-token, seqlen 23) but the rid differs every time -- six distinct requests, not one six times. Whether the fresh rid is a symptom of carry dropping identity or simply the next request hitting the same trap is left open rather than guessed. The defensible claim is the weaker one: on an idle box a small prompt costs a layout round trip, and at 06:53:56 tp had already decided it could do the work itself ("pending prefill 22 tok <= N=7004, running it in tp"), ran the prefill, and armed anyway. Also corrected: the 81 -> 639 -> 1078 -> 1572 -> 1941 growth (peak 2016) is a multi-tenant backlog pile-up, 3-4 requests carried per cutover across 138 rids -- not one request growing. The report prints the pending sequence rather than one number so the two shapes stay distinguishable. THE HARNESS CAN COUNTERFEIT ITS OWN RESULT, so it is stopped from doing so Runs are opposite-direction arms within RUN_MAX_GAP_S. Since each small prompt costs about one round trip, shots fired faster than that window chain into a single long "run" that is purely the injector's cadence. The first draft defaulted to 6 s against an 8 s window and would have manufactured exactly the result it was built to test. The gap is now guarded, defaulted to 25 s, and the test reads the real parser default instead of restating it. Live mode is a window ticket and refuses to run without a gpu-arb claim, so import and dry-run cannot reach the server by accident. TESTS test/srt/test_d2_injector_pingpong_713.py, 16 passed, ruff clean. Offline dry-run over a recorded excerpt reproduces the known 12-arm / 31 s run with the signature. Mutation-proven three ways, each caught: signature and -> or -> 1 failed drop the origin-rank guard -> 1 failed default gap 25 s -> 6 s -> 1 failed The origin-rank test was rewritten after the first version proved vacuous: all 13 fixture arms are PP0 already, so deleting the guard left the suite green. It now runs against synthetic multi-rank lines. NOT CLAIMED: that this is what sgl-project#713(a) fixed. On the tp side of the loop running_bs is 0, so the sgl-project#713(a) resident term contributes nothing there. sgl-project#713(a) stays "deployed, payoff pending targeted proof"; this is a different and larger defect that happens to produce the symptom sgl-project#713 was opened on.
efschu
added a commit
to efschu/htsglang
that referenced
this pull request
Aug 17, 2026
…d conflict map PREPARATION ONLY. No branch pointer was moved and nothing was merged. Every conflict is MEASURED by trial merges in a throwaway worktree (throwaway/merge-train-probe, removed after measurement), not predicted. Built from git rather than from the handed list, and git contradicted the list twice: * fix/713-admission-intake is ALREADY in the serving line (+0 commits, 0 files) -- not a train item, and scheduling it would be a no-op step that looks like progress. * fix/728-max-bytes-uniform points at 79216e6, byte-identical to feat/706-phase-uniform-hicache-keys, and has NO REMOTE. It must not be merged as a separate item, and its identity has to be settled first: a local-only branch is the one kind that disappears with its worktree. DIVERGENCE. The 4222976 class turned out not to be the risk it looked like: that commit is PRESENT in integration/r2 and absent only from upstream main, which is true of every fork commit. The useful axis is the merge target -- 167 commits are on the serving line and not in integration/r2, and exactly SIX of them are on no other train branch (the sgl-project#677 range c4bc982..5fed8a6, sgl-project#708 fc6f97b, and the merge 761d0d7). All six are reachable from feat/677-park-wiring, so that branch carries them; if any step drops or rewrites it, those six are the loss. The serving line is also NOT an orphaned detached head -- 5fed8a6 is exactly feat/677-park-wiring, checked, because unbranched commits on a serving head are how a train loses a fix. ORDER. All eleven candidates are independent siblings (a containment check found no branch containing another), so order is a conflict question. The measured result is two clusters and a clean set: * Cluster A, scheduler_teardown.py: all four sgl-project#673 thread-stop branches add their stop logic to the same file this lane created for sgl-project#673. They conflict with EACH OTHER, not with the trunk, so whichever lands first sets the file's shape. The barlink one also touches scheduler.py and belongs to live sgl-project#722. * Cluster B, planner/seam: fix/602-fill-side, fix/701-ledger-wiring and feat/704-prefill-ladder each rewrote seam_slope.py, planner/pp_cut.py and test_pp_cut_prefill_speed_702.py. Three lanes editing one model of the same thing -- the conflict is SEMANTIC, and resolving it by taking hunks would produce a seam model nobody designed. * The clean six-step train (621, 699, 673-lockstep, 706, 717, 677) merged in sequence with no conflicts at all and can run without either cluster. TEST MATRIX, measured on the merged probe state: mem_cache 940 failed / 973 passed and distributed 21 failed / 2716 passed, both matching the standing baselines. managers came out at 12 failed / 2274 passed against a standing figure of 14 -- and the point is that this suite's count is TRAIN-COMPOSITION-DEPENDENT: this lane's branch alone shows 4, the merged probe 12, with the extra 8 arriving from the sgl-project#677/sgl-project#713/sgl-project#631 lanes rather than from the merge. The failing classes are listed so they can be attributed, and each owner must record their own baseline before the train, or "the branch shipped it" becomes "the merge broke it". HELD OUT, with the separability question answered: 2ce1ed7 is NOT docs-only -- it changes managers/phase_flip_runtime.py, which is exactly why it earns its review boot. The two later sgl-project#441 commits touch no file it touches, so they are cherry-pickable without it; the only entanglement on that branch is be1fcec -> 2ce1ed7 (same NOTE). But 4512136 carries sgl-kernel/csrc/kvcacheio/transfer.cu, so riding it means a kernel rebuild -- its own boot-gated risk, not something to smuggle in behind a docs-and-tests framing. Also in this push: the sgl-project#568 ledger commit (79216e6), which the audit found was this lane's only unpushed work.
efschu
pushed a commit
to efschu/htsglang
that referenced
this pull request
Aug 17, 2026
…-pong A layout may be declared unable to run only after it has HAD THE CHANCE to run. An emptiness observed within idle_locked_settle_s of entering a layout is a transient, not a verdict. THE CLAIM THAT FAILED The IDLE_LOCKED branch sits above the min-dwell check and bypasses the only anti-thrash bound, justified by a comment stating "IT CANNOT OSCILLATE ... after the flip the target runs by premise, so the same condition is false there". It oscillated. Across the 16 boot rotations of 2026-08-17 the policy produced alternating runs of 72 arms / 299 s, 12 / 31 s and 10 / 27 s twice. The premise fails on WHEN it is evaluated, not on what it says. This branch is reached on the first round after a cutover, while the just-entered layout is still empty and its carried work not yet re-admitted -- Scheduler._idle_locked_inputs is gated on _round_built_nothing, which a just-entered layout satisfies trivially. The new layout is observed in its empty transient, certified unable to run, and armed straight back. That is also where sgl-project#713's TTFT quantisation came from. The unit of delay is one whole cutover (median 2864 ms tp_to_pp, 2772 ms pp_to_tp over 486 flips), so the 0.1 / 3.1 / 5.9 s levels in the sgl-project#713 tables are how many cutovers a request sat through. The 06:19 table was taken during a 27 s run. THE VALUE IS DERIVED, NOT CHOSEN Delay from "cutover complete" to the first batch the new layout builds, over 162 cutovers: 0 s for 66 of 150, 1 s for 34, 2 s for 27, 3 s for 12, thin tail to 6 s, one outlier at 30 s. 2.0 s covers 84.7 %. The p95 (4 s) is deliberately NOT used: it exceeds the 3 s min_dwell_s this rig boots with, and a settle above the dwell would make the fast escape slower than the path it exists to bypass. Log stamps are second-resolution, so this is a bound, not an optimum, and the docstring says so. FAILURE DIRECTION, AND WHAT BOUNDS IT A genuine idle lock forming within the settle of a cutover is now delayed, by at most the settle. Bounded three ways: the value is capped at min_dwell_s at the use site, so an operator cannot configure the fast escape into being the slow one; the 180 s decode-stall cap remains the backstop that released the 09:42:39 specimen; and the settle is consulted only right after a cutover, which is the only place the transient exists. sgl-project#688's escape is preserved and pinned. phase_since, not last_flip_at: the first is when THIS layout was entered and is maintained from the OBSERVED phase, so a manual POST /phase_flip restarts it too; the second is an arm stamp that is 0 until the policy has armed once. None degrades to pre-guard behaviour rather than to an infinite settle. RED-FIRST, AGAINST THE RECORDING test_idle_locked_settle_713.py replays the recorded arms through the real policy off scripts/fixtures/d2_injector_pingpong_excerpt.txt, reusing the injector's parser so harness and suite cannot disagree about what the log says. settle disabled -> 12 of 12 arms armed, the ping-pong reproduced settle enabled -> 1 of 12 armed Not zero, and that is the point: the arm that legitimately leaves a 35 s-settled idle-locked layout survives, and only the eleven arm-backs on 0-1 s-old layouts are refused. The first version of this replay asserted all twelve were transients; the data refused, one arm being 35 s past its cutover. That arm was not noise, it was the control case, and it is now the selectivity pin. Mutation-proven: disabling the guard at source turns 3 tests red. REGRESSIONS Baselined before and after by capturing the failure set with the guard reverted. 8 failures pre-exist on this branch in this area and are untouched; the guard adds none and 705 pass. The five sgl-project#677-era pins stay green because they construct PhasePolicyState() with no phase_since -- which is also why they never caught this, and their docstring now says so instead of restating the falsified invariant. sgl-project#712 TEXT REMOVED The BOTH-BLOCKED decline used to redirect to "the state-slot bound (mamba/GDN slots)". That was never measured: it was a hypothesis authored into a log string, read back out of the log and filed as a finding, and sgl-project#712 was closed as unfounded on that evidence. The line now reports only what it knows. test_both_blocked_binding_resource_708 REQUIRED that wording, so the pin is inverted rather than relaxed -- an unmeasured cause must not be named.
efschu
pushed a commit
to efschu/htsglang
that referenced
this pull request
Aug 17, 2026
… remedy THE SPECIMEN. The 2026-08-17 speed boot was ordered at cut [31,17,16] and launched with `--pp-stage-ratio 31,17,16`. The coupled derivation balances both layer families together and returned [32,16,16], moving a FULL-ATTENTION layer onto stage 0 -- 8/4/4 instead of 7/5/4 -- which shifted KV mass against a pinned 550000-token pool. The boot died in an NCCL send at 08:53:51, PP0 at 176 MiB free against a 1024 MiB corridor law, `Cuda failure 'out of memory'`. The derived split was already logged. What was missing is any statement that the REQUEST HAD NOT BEEN HONOURED, so the boot record read as the cut that was ordered and the OOM looked like a property of [31,17,16]. It was not: it is a property of [32,16,16], a cut nobody chose. sgl-project#505(a) class -- an unannounced substitution, not a wrong number. WHAT THIS DOES NOT SAY. It does not say the cut is unrealizable. It is: `--pp-attn-stage-ratio` (sgl-project#485) decouples the full-attention family from the linear one, and 7,5,4 gives exactly [31,17,16]. An earlier verdict here that "stage boundaries must be multiples of 4, so [31,17,16] cannot exist" was inferred from the COUPLED path alone and is wrong; the decoupling had already been built. So this warns and proceeds rather than refusing. The advice is computed, not hinted: the full-attention counts of the operator's own requested ranges ARE the --pp-attn-stage-ratio that realizes them, so the warning prints the vector to paste. Fires only when the scores sum to the model depth -- i.e. when the operator is spelling out layer counts rather than capability ratios. Scores are ratios by contract, so a true ratio like the incumbent 14,10,8 -> [28,20,16] must stay silent, and a test pins that it does. TESTS test/registered/unit/server_args/test_pp_stage_ratio_snap_warns_713.py, 7 passed. Pins the mechanism the advice points at (coupled snaps to [32,16,16]; attn_scores 7,5,4 realizes [31,17,16] exactly; the advised vector is derivable from the ranges), plus three can-fail cases: an honoured request, a true ratio, and an already-decoupled request must none of them warn.
efschu
pushed a commit
to efschu/htsglang
that referenced
this pull request
Aug 17, 2026
…de a solved phase layout Builds the piece the remainder determination (af2b34e) rescoped: solve the layout ONCE per phase and say where each family lands inside it. The runtime per-family flip stays closed -- it needs ~17,500 rounds to repay a whole-arena refill -- but placement inside a flip that is already happening is free. ## No second solver, which was the acceptance condition planner/boot_layout.py does not search, price or re-derive anything. It consumes a PPCutSolution that solve_pp_cut already produced and re-presents it per family. A test pins that the module's source contains no call to solve_pp_cut and no _price_stage, so the cut can only change if the SOLVER changed. The composition was cheap because the solver already carried the family axis: PPCutSolution.attention_counts is a per-stage family placement in all but name, and layer_families tags every layer. What was missing was a named shape, the other families, and one authority for the emitted flags. ## One authority for the printed remedy server_args already computes the paste-ready --pp-attn-stage-ratio when a snapped cut needs announcing (sgl-project#713, 88a0d78): "the full-attention counts of the operator's own requested ranges ARE the --pp-attn-stage-ratio that realizes them". The chooser emits that SAME quantity from the SAME field (solution.attention_counts), so the printed remedy and the solved layout cannot drift. --pp-layer-ratio comes from as_layer_ratio() VERBATIM; a test pins it byte-identical, and the mutation that re-derives it as an even split turns exactly that pin red. ## The enumeration gap: closed where funded, NAMED where not (sgl-project#253) PPCutInputs carries measured terms for exactly TWO families -- attention (attn_layer_weight_bytes, attn_layer_flops_per_token, attn_core_flops_per_token_pair) and linear (linear_layer_weight_bytes, linear_layer_flops_per_token). Both are now emitted as placements. Every other family in the sgl-project#485 law has NO input in this cost model, so none got one invented. UNFUNDED_FAMILIES records each with the measurement that would fund it, in the module that omits them rather than in a doc: kv_heads not a PP-layer family at all -- a TP-axis quantity, so per-STAGE placement is the wrong shape vocab embed/lm_head are not per-layer, so layer_families never tags them; needs measured bytes AND stage affinity experts no MoE family in this checkpoint's layer_types nonlinear_kernels folded into per-family layer FLOPs; needs a decomposed measurement linear_per_quant_lane linear_layer_weight_bytes is ONE scalar for the family; needs per-lane bytes measured from the checkpoint A test asserts every entry names a measurement, and another asserts no speculative term (expert_layer_weight_bytes, vocab_weight_bytes, ...) appears in the source. ## ABSENT is not ZERO (sgl-project#606) Families are derived from layer_families, so a family the checkpoint lacks produces no entry -- and asking for it RAISES with that reason rather than returning a zero vector. The mutation that pre-seeds both families (zero-filling a model that has only one) turns exactly that pin red. ## Refusals An INFEASIBLE solution emits no flags at all ("do not paste this layout") -- flags for a cut the solver already rejected are how an operator pastes a boot that cannot come up. A layer_families vector that does not match the cut's depth is refused as describing a different model. The phase is NAMED (PHASE_PP_PREFILL), not assumed: sgl-project#485's matrix has a decode column and this solver does not solve it. ## Tests test/registered/unit/planner/test_boot_layout_485.py -- 18, driven by the REAL solver through the existing sgl-project#485 fixture rather than a stand-in, because a stub would let chooser and solver drift (sgl-project#624). mutations, one per contract: zero-fill absent families -> 1 red (the sgl-project#606 pin) re-derive the layer ratio as an even split -> 1 red (the byte-identity pin) restored -> 18 passed test/registered/unit/planner: 2 failed / 2529 passed. Both failures are PpWithSpecEvidenceTest, the stale-register drift I diagnosed and fixed in 54ffc4d, which has not reached this base -- pre-existing and unrelated, and a merge-train item. A vacuous assertion of my own, caught by re-reading rather than by a failure: the gap-funding test asserted `assertIn("Would need", why + "would need")`, concatenating the needle onto the haystack so it could never fail. Fixed to check the actual value.
efschu
pushed a commit
to efschu/htsglang
that referenced
this pull request
Aug 17, 2026
…utover
THE CAUSAL CHAIN, measured 2026-08-17 across two boots:
the carry re-homed a request into running_batch/running_mbs[0] and left
waiting_queue untouched
-> the request existed TWICE: resident (invisible to the policy as
runnable) and queued (counted)
-> _pending_prefill_tokens summed the queue and the resident set without
excluding their intersection, billing the same prompt twice:
51,369 -> 102,307 tokens across one cutover, within rounding of 2x
-> the inflated backlog drove the flip policy past its threshold
-> six cutovers, FLIP-CARRY announcing a resident carry while the policy
read `running bs 0`, the sgl-project#699 detector reporting "1 queued, 0 running",
and the warmup generation never served.
THE FLIP CHURN WAS A SYMPTOM. Six cutovers looked like a flip defect and were
not one: the policy responded correctly to a number that was wrong. Nobody
should "fix" the churn separately.
THREE PARTS, because the defect has three surfaces:
1. STATE -- the carry now CONSUMES what it re-homes
(`_consume_carried_from_waiting_queue`). After a cutover the resident set
owns the request and the queue must not also claim it. Both edges pinned: a
request that is ONLY queued stays untouched, and a carry over an
already-consumed queue is a no-op rather than a corruption. Bookkeeping
never raises into a cutover (sgl-project#715 lesson).
2. NUMBER -- `_pending_prefill_tokens` de-duplicates at the INTERSECTION. Not a
blanket "count each rid once anywhere": a request genuinely holding budget in
two places is a real state a future reader may need to see, and a global
dedup would make that class silent the way this one was. Exactly one overlap
is excluded, and only this one. The sgl-project#713(a) resident term did not create the
duplicate state -- the carry did -- but a counter that sums two sets without
excluding their intersection is the second half, and that half is mine.
3. GUARD -- `duplicate_resident_reqs` gains the waiting queue in its universe.
It compared batches against each other and never consulted the queue, so it
reported "no duplicates" meaning "none of the kind I look for". Queue-side
hits carry a `queued:` prefix so the 2026-08-09 resident-vs-resident specimen
and this one stay distinguishable.
Plus the hardening: `_arriving_prefill_tokens` asserted "ARRIVED but not yet on
the queue" and enforced nothing. Its only inflight-bearing call site is
pre-queue so the invariant holds today, but an asserted-never-checked invariant
is exactly how this double count stayed silent, so it is checked now.
TESTS 18 passed. Mutation-proven THREE ways, each caught: removing the carry's
call site, removing the counter's dedup, and removing the guard's queue
universe.
TWO OF THOSE TESTS WERE VACUOUS FIRST, and both are worth recording:
* the carry test called the helper directly, so deleting the call from
`install_resident_set` left everything green -- the helper worked and nothing
used it. It now drives the real entry point.
* the counter test's duplicate had `extend_range=None`, which the sgl-project#713(a) term
skips by design, so the resident side contributed 0 and the intersection was
never exercised. The duplicate is now made countable on purpose.
Both were found by running the mutations rather than by reading the tests.
REGRESSION managers + mem_cache: 19 failed / 3588 passed against a captured
baseline of 19 -- ZERO new failures. Ruff: scheduler.py 94 errors before and
after, all pre-existing; the other two files clean.
efschu
added a commit
to efschu/htsglang
that referenced
this pull request
Aug 17, 2026
…ion pass Records the two-file deletion set (pin_ledger stays, kept and extended by the reconciliation), the per-hunk authority, the three incoherences the textual merge hid, and the candidate verdicts -- notably audit/stale-gates, which merged clean and was dropped because it carries a revert of sgl-project#713 that removes a PhasePolicyConfig kwarg F4-r4's sgl-project#730 tests depend on. Gate at or better than every established count; managers improved on 22.
efschu
pushed a commit
to efschu/htsglang
that referenced
this pull request
Aug 17, 2026
…- unstrangle the funder My sgl-project#744 fix has a live regression and this is the refinement. Specimen 21:46:32 on 72696b0aec: tp_to_pp refused 35x, IDLE-LOCK with 407,622 tokens pending and 0 resident, the guard reporting "no KV provider is registered" because the armed-only gate refused the rung at BOTH sites and left the provider list empty. Seam staging (PP1: needs 1269 MiB, spendable 883, rung wanted SHRINK to 126506 with slack 45526 = a 1455 MiB deficit that WAS coverable) could never be funded. THE ERROR WAS MINE AND IT WAS A DESIGN ERROR, not a slip. Seam-funding eviction of recomputable prefix rows is REQUESTED BY the flip machinery -- it is what "KV capacity is the funder" means -- so refusing the rung for the duration of a flip protects against the 21:18 crash by disabling the thing the flip is waiting on. A wholesale gate cannot be right when the protected party and the requesting party are the same machinery. THERE WERE TWO STRANGLE POINTS, not one. The armed gate was the obvious half. The second was quieter: _nothing_resident() returning False on a parked extent dropped through to the unknown-refuse branch, so even with the gate removed the rung would still have declined. Both are fixed. THE REFINEMENT. The parked extent already carries exactly the information needed to be selective, which is why this is clean rather than a compromise: rows INSIDE it are what the flip is about to pack and may not be touched; every row ABOVE it is recomputable prefix and is precisely what the funding wants. So the extent PINS THE CEILING instead of closing the rung. _parked_ceiling() returns the highest parked row id, -1 when nothing is parked, and -2 for the one case that still refuses wholesale: an UNKNOWN extent while a flip is armed, where there is no boundary to name. Both call sites take req_max = max(req_max, parked), so the evictor is handed a ceiling that cannot reach into the extent. UNKNOWN, DECIDED AND DOCUMENTED as asked: it refuses ONLY while a flip is armed. Outside a flip there is nothing parked to protect, and closing the rung there is exactly the strangle this commit removes. sgl-project#746 (the exact arm-time snapshot) is what removes this last wholesale case; until then it is one narrow, named condition rather than the whole flip window. TEST RESULTS test_evict_rung_flip_park_744.py refined to the new semantics, 19 tests + 2 subtests green. The two tests that asserted the wholesale gate are REPLACED rather than deleted -- they now assert the funding path delivers -- and the crash protection gets its own explicit test that the evictor's ceiling never reaches into the parked extent. Mutation matrix, 4 mutants, ALL KILLED, covering both directions: N1 exclusion removed (parked ignored) -> 1 failed CRASH protection dies N2 exclusion covers everything (gate back) -> 1 failed FUNDING dies N3 collecting site ignores parked ceiling -> 2 failed N4 unknown-while-armed no longer refuses -> 3 failed N1 and N2 are the (c) pair: neither over- nor under-covering survives. Pinned suites green together, 142 passed + 87 subtests: sgl-project#744, sgl-project#731 + the #731x#744 interaction pin, both sgl-project#717 suites, sgl-project#656, sgl-project#684, sgl-project#662-f4, sgl-project#713 admission intake, sgl-project#739 prefill progress. Ruff and codespell clean. PP2 SECONDARY, checked and NOT claimed as fixed. The 0 MiB shrink against an unmoved driver-free column is an already-instrumented condition (kv_backing_relief.py:1915-1927): the code detects it, names retained handles (SGLANG_FLIP_SEAM_RETAIN_HANDLES) or a missing commit chunk as the cause, keeps the cap on and returns 0. The gate refinement lets the rung PRICE again; it cannot make a retained-handle arena PAY. Necessary, not sufficient -- the retain setting needs verifying at the boot, and this commit does not pretend to close it. No boot.
efschu
added a commit
to efschu/htsglang
that referenced
this pull request
Aug 18, 2026
…ce in one harvest plan, with a proven runner Consolidated from the comp4 gate run (progress.662-F4-r5), WINDOW_TICKET_745/755, NOTE_747 par.8-9, NOTE_738/755, TICKET_727 and the operator ledger. Two hard incompatibilities shape the plan: WINDOW_TICKET_745 Arm 1 excludes the checkpoint interval that sgl-project#758's anchor-cadence observable requires (-> ARM I hicache harvest, ARM II = one flag more), and sgl-project#713's TTFT<3s needs a quiet router while every other loaded gate needs the soak backlog (-> sgl-project#713 is the idle sub-phase BEFORE the backlog, not a separate boot; the 06:44Z soak driver is the load source per the Lastprobe rule). ARM I phases: load-time (sgl-project#738 no-99G-plateau, file-backed-image reclaim), idle-quiet (sgl-project#713, health, corridor), loaded (Gates A/B/C, - sgl-project#757 race-holds, sgl-project#748 all three shapes, sgl-project#744/sgl-project#717 rung-funded flip, - sgl-project#690 refill census, sgl-project#758-2 mamba host resume, WT_745's three lines, corridor minima), teardown (image reclaim). ARM II adds --mamba-checkpoint-interval 8192 for sgl-project#758-1 anchor cadence + NOTE_747 par.8.1-8.3. SEPARATE windows named with reasons: sgl-project#727 four-boot A/B, WT_755 slots A/B (pool-geometry confound), sgl-project#755 metal retraction (mechanism not built -- nothing to measure), sgl-project#709, sgl-project#735 Step-2. sgl-project#602 and sgl-project#536/sgl-project#537 carried as HONEST unresolved slots (owner-held detail / not found with acceptance shape) rather than invented readouts. Runner run_window_ladder.sh: PASS/FAIL/UNOBS table from boot log + live server; never boots, never kills, never touches the soak driver; soak-tolerant by construction (loaded checks are log observations, the one latency check runs only in --phase idle, and choosing that phase IS the operator's quiet-router assertion). Missing-emitter cases (anchor cadence) report UNOBS, never FAIL -- absence of an instrument is not absence of the property. Mock-smoked per the desk rule, both directions: a fixture built from the comp4 specimen lines reproduces the real run's verdicts exactly (GATE-C crash, both sgl-project#748 shapes + vacuous relief, sgl-project#757 sentence = 5 FAIL; sgl-project#744/sgl-project#690 PASS; exit 1) and a clean fixture goes fully green incl. the ARM II cadence line (exit 0). bash -n clean, codespell clean. Nothing was booted.
efschu
pushed a commit
to efschu/htsglang
that referenced
this pull request
Aug 19, 2026
THE CHECK WAS IN THE WRONG PLACE, NOT MISSING. write() already refuses at ENQUEUE via the sgl-project#718 device-tier disarm. The copy happens later, and the flip rebinds in between, so a write-back queued before a cutover reaches backup_from_device_all_layer carrying a pointer table into the pool it was built from. Both crash specimens died three seconds AFTER a pp_to_tp cutover completed: 14:08:14 -> 14:08:17 (epoch 27) and 07:12:09 -> 07:12:12 (epoch 3), seven hours apart, same direction, same lag. WHY THE SHAPE GUARD COULD NEVER CATCH IT, and why its silence was misread as innocence -- by me, until the operator pushed back. Under layer_first the host layout EQUALS the device layout, so a stale binding is shape-IDENTICAL to the live one and check_shapes passes by construction. That is exactly what sgl-project#760 recorded: KV-TRANSFER-GUARD armed on all three ranks, zero transfers refused, SIGSEGV anyway. Matching shapes plus a crash puts the fault below the Python seam; a generation stamp is what tells "same shape" from "same pool". MEASURED, ONE VARIABLE. HiCache host+disk under sustained 4-way load with the flip REMOVED (plain TP3, which keeps speculation on -- PP+spec requires the flip, server_args.py:18385) survives five minutes with 0 segfaults and 0 admission wedges, and passes the REP gate outright (1 distinct/12, salted 0/6). The fault needs the cutover. STAMPED BY CONSTRUCTION, VERIFIED AT CONSUME. The stamp lives in CacheOperation.__init__ rather than at one enqueue site: an op built by any other path would otherwise be unstamped, and an unstamped op must be refused, which silently dropped legitimate write-backs and broke the staged-dispatch tests until I moved it. A stale op is dropped loudly and counted; its prefix simply misses later, the same cheap failure the sgl-project#718 disarm already accepts. Also folded in, both consequences of earlier sgl-project#767 work rather than new choices: the anchor-protection test that encoded "no interval means no anchors" is updated to the corrected premise it was measured against, and the SECOND-PASS EVICTION line drops from warning to info -- the second pass is documented as legitimate, and anchor eviction was falsified as the drift cause, so it is accounting rather than an alarm. 4 new tests, red-first (a pre-rebind stamp must be refused). mem_cache and managers suites: 4047 passed, 43 failed -- all 43 the pre-existing sgl-project#772 class (PhasePolicyConfig lacks idle_locked_settle_s after the sgl-project#713 revert), one fewer than the 44 baseline because the 747 premise test is now correct. ruff clean on both touched files (0 at HEAD, 0 now).
efschu
pushed a commit
to efschu/htsglang
that referenced
this pull request
Aug 25, 2026
d3 replaced starvation with a ping-pong. All 18 tp_to_pp armings read 'pending prefill 0 tok > 0' -- a verdict of >0 printed beside a 0 -- while two requests were DECODING; epochs 13/14 in six minutes, decode chopped mid-bundle, COMPLETIONS 0. Prefill side stayed correct (252 pp / 0 tp). TWO DEFECTS, both mine, both in the term I added: 1. TWO READS, ONE MESSAGE (sgl-project#713). The verdict came from work_exists() and the message printed pending_prefill_tokens, so the line contradicted itself and was unreadable as evidence. demand_prefill_tokens() is now computed ONCE and feeds the verdict AND the message. 2. THE WRONG UNIVERSE. work_exists() counts any queued work. The build site subtracts seam-transport tokens from pending_prefill_tokens (W32: transport is not PP workload) and my admissible term deliberately did not -- so after every cutover the re-admitted bundle read as 'unstarted prefill work' and demanded the flip away from the layout that was about to finish it. THE DISCRIMINATOR IS running_bs, NOT THE TOKEN COUNT, and it is the user's law rather than a compromise with it: aller Decode in TP, dann aller Prefill in PP. DRAIN, THEN FLIP. demand_prefill_tokens() returns 0 while decode is in flight and the full backlog when nothing is decoding. That satisfies both specimens at once -- d2 (7 queued, 0 running, 589s no first token) fires, d3 (same queue, 2 decoding) is silent -- and it is W30's own rule read forward: an arm may not destroy its own justification. Both directions use the one read, so the tp->pp arm and its pp->tp mirror cannot disagree about whether work exists. CLASS: unchanged, and this is its THIRD instance in 24h -- a counter or term that answers a different question than the decision needs (F2 blind, demand blind, demand inverted). What failed each time was the SWEEP, not the naming. FUTURE-CHECK, added as a standing rule: every existence/demand term now needs BOTH SYMMETRY PINS -- fires when work exists, silent when none does. One pin alone is exactly how a blind term and an inverted term each passed their own tests. Six pins added, including both metal specimens by name. Gate: 791 passed (purity/conformance/838/856/policy/seam/861/859), ruff clean.
efschu
pushed a commit
to efschu/htsglang
that referenced
this pull request
Aug 26, 2026
Two subtractions, one justification, only one of them checked:
scheduler.py `_pending_now -= _seam_transport_now` UNCONDITIONAL
scheduler.py `_seam_serviceable_now = ...` gated on the TP phase
AND on
seam_transport_premise_holds
Both rest on the same claim -- that a seam re-admission is cheap flip transport
rather than real workload, because "their prefixes are served by read-through
from the canonical store". #861j verified that claim for the EXISTENCE term and
never backported the verification to the ECONOMICS term twelve lines above it.
WHEN THE PREMISE IS FALSE those tokens are not transport at all: they are a full
cold prefill of real work, and deducting them tells the policy that work does
not exist. The consumer that pays is the one arm still reading RAW pending --
the sgl-project#677(a) blocked-admission stall escape, whose threshold is
`pending > pp_exit_tokens`. A deflated pending holds a genuine stall below its
own escape, which is the wedge that escape was written to end. Delayed escape,
not a wrong answer: `demand_prefill_tokens()` takes
max(pending, admissible - serviceable) and `admissible_prefill_tokens` is not
reduced by seam transport, so with sgl-project#869 landed `_strict_holds_pp` still sees the
true backlog.
VARIANT (a) OF THE TWO I FILED, chosen because it makes the two subtractions ONE
RULE rather than removing the last raw-pending consumer and leaving the pair
disagreeing. `seam_transport_deduction` is pure and total, so both directions
are falsifiable without a scheduler, and the premise is now asked ONCE per round
and read by both terms -- one predicate, one clock, per that function's own
contract. Asking twice was its own latent defect: the debt clock can lapse
between two calls in the same round.
NOT A DISARM. A verified premise in the TP phase still deducts exactly as
before, which is the #861j/W32 behaviour where it was actually earned. The test
pins that direction too.
CLASS: a subtraction justified by a premise that is never checked -- the same
family as sgl-project#869 (a predicate that does not measure what it claims), different
root. Sibling of, not instance of.
TEST RESULTS
test_seam_transport_premise_869c.py (new): 11 passed.
FALSIFIED IN BOTH DIRECTIONS BEFORE THE GREEN WAS CLAIMED, count gate on each:
fix reverted (deduct unconditionally = today's tree) -> 7 failed, INCLUDING
the specified falsifier test_the_stall_escape_arms_on_the_true_backlog.
7 extracted == 7 in summary.
over-fixed (never deduct, blanket disarm) -> 3 failed, including
test_a_verified_premise_in_tp_still_deducts. 3 == 3.
restored -> 11 passed.
Regression set, 59 files (purity, policy, seam, transport, sgl-project#677, sgl-project#713, sgl-project#861,
sgl-project#871, #869c): 895 passed, 4 failed, 123 subtests passed. Count gate 4 extracted
== 4. All four are test_restore_never_rebuild_677, one of the three standing
card-needing modules whose hermetic count is exactly 4. No new failures.
Hermetic: CUDA_VISIBLE_DEVICES verified EMPTY at the PROCESS (/proc/<pid>/environ).
Lint: ruff 0 before and after on phase_purity.py; scheduler.py 102 before AND
after (pre-existing); new test 0. codespell clean.
Non-flip and non-seam paths are byte-identical: with no stamped re-admissions
`_seam_transport_now` is 0, and the deduction is 0 under every gate combination.
TWO BUGS IN MY OWN TEST, found by running it and reported rather than quietly
fixed. The first draft left the hand-set `pp_window_s` stopwatch enabled, so the
deflated specimen still armed -- for a different reason -- and the test would
have passed without the escape ever being consulted; it now sets pp_window_s=0
and decode_stall_slo_s=0 to isolate the sgl-project#677(a) arm. The second asserted on the
substring "stall" in the reason string, which matched
SGLANG_PHASE_POLICY_DECODE_STALL_SLO_S in an unrelated suggestion line; it now
asserts on the verdict. Same substring-matching family as the ^FAILED trap.
NOT CLAIMED: no metal run. Desk-proven only. The sgl-project#857 acceptance instance was
left running and untouched throughout.
efschu
pushed a commit
to efschu/htsglang
that referenced
this pull request
Aug 30, 2026
… die Policy vergleicht == EIN FEHLENDES ARGUMENT, GEMESSEN STATT VERMUTET == `SUBCHUNK-SERVED-IN-TP` feuerte auf boot_855_1011idle NULL Mal, waehrend das sgl-project#887-Gate 11x gewaehrte und die Policy 10x tp_to_pp auf "pending prefill 51 tok > 0" armte. Alle statischen Terme lasen erfuellbar, also war Weiterlesen nicht die Antwort: ein Direktaufruf von `decide()` reproduzierte die LIVE-Zeile byte-genau mit grant=0 und lieferte bei grant=51 die Unterdrueckung -- die Policy-Haelfte ist korrekt, der Grant kam als 0 an. WURZEL: `_pending_prefill_tokens(inflight=None)` -- der eigene Docstring sagt "``inflight`` ... Passing it is what the flip policy MUST do". Die sgl-project#942-Sonde liegt auf dem Flip-Policy-Pfad und uebergab es NICHT. `recv_requests` wertet die Policy AUS, BEVOR die Requests in `waiting_queue` stehen (genau deshalb existiert der Parameter), also mass die Sonde bei einer FRISCHEN ANKUNFT 0, waehrend die Policy den echten Prompt sah. `0 < 0 < chunk` = False -> Grant 0 -> Unterdrueckung unmoeglich, und zwar fuer genau den Fall, fuer den sgl-project#887 gebaut wurde. Ein bs1-Strom macht ausschliesslich frische Ankuenfte. == DER SCHNITT: EIN OPTIONALES DURCHREICHEN, EINE REGEL == `tp_compute_fits_in_one_chunk(scheduler, inflight=None)`. Default None haelt JEDEN anderen Aufrufer byte-identisch -- insbesondere das ECHTE GATE (phase_purity.py:1093) reicht bewusst nichts durch: ein Batch-Builder kann nur bauen, was wirklich in der Queue steht. Die Fit-Regel bleibt EINE Regel (nicht dupliziert), sie beantwortet nur je nach Aufrufer die passende Frage. LECK-FAMILIE UNBERUEHRT (sgl-project#890/sgl-project#906): beide Sonden sind PURE READS, `_spend_tp_compute_chunk` wird ausschliesslich aus `prefill_blocked_here` gerufen. Zu sehen ist nicht zu verbrauchen. == FEHLERKLASSEN-PASSENDE CHECKS == * Gefaehrlichste Klasse: `inflight_reqs` nicht im Scope -> NameError, vom umgebenden `except Exception` verschluckt -> stiller No-op. AST-geprueft: es ist PARAMETER von `maybe_arm_phase_policy` (13020-13664). In Scope. * Verhalten je Aufrufer, Stand-in-Scheduler: frische Ankunft (Queue 0, inflight 51): Gate=False (UNVERAENDERT), Policy-Sonde=True <- der Fix bereits gequeued (51): beide True (unveraendert) ueber einem Chunk (9000): beide False (Refusal erhalten) Legacy-Accessor ohne Argument: True (TypeError-Fallback haelt) * Diagnose-Zeile rate-limitiert (unlimitiert: 1457 Zeilen in EINER Zelle); Zaehler-Vorlauf per Direktaufruf belegt (3 nach 3 decide()). * sgl-project#1011-Nichtregression per Direktaufruf: CASE1 refuses, CASE3 flips. == ZELLEN (gleiche Harness, greedy, 600 tok, fuehrendes Unique-Token) == Zelle vorher (bc43f0d) nachher (dieser Commit) bs1 25,09 n=5 Spr. 4,1% 72,0 n=7 Spr. 69,0% conc=4 88,64 n=3 Spr. 3,4% 134,93 n=3 Spr. 100,8% Flips/bs1-Zelle 12 auf 6 Requests 3 SUBCHUNK-SERVED 0 10 KEIN DOPPEL-PREFILL: `cached_tokens` ist in JEDEM Lauf 0, in schnellen wie langsamen -- der Grant-Pfad fuehrt keinen Re-Prefill ein (sgl-project#939 unberuehrt). EHRLICHE SCHRANKE: die Verteilung ist BIMODAL (bs1-Laeufe 67-80 tok/s, mit einzelnen bei ~23), daher die grosse Spreizung. Der Median bewegt sich stark, die Streuung ist NICHT Rauschen sondern zwei Regime -- welcher Anteil der Requests noch ins alte Regime faellt, ist offen und nicht von diesem Commit beantwortet. BELEG-STUFE: BOOT-BEWIESEN (boot_855_942c, boot_855_942cfinal).
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Thank you for your contribution, we really appreciate it. The following instructions will help improve your pull request and make it easier to receive feedback. If there are any items you don't understand, don't worry. Just submit the pull request and ask the maintainers for help.
Motivation
Please explain the motivation behind this PR and the goal you aim to achieve with it.
Modification
Briefly describe the changes made in this PR.
Checklist
pre-commit run --all-filesor other linting tools are used to fix potential lint issues.