Repository navigation
Adjust max prefix len - #980
Merged
Merged
Conversation
Ying1123
approved these changes
Aug 8, 2024
hnyls2002
enabled auto-merge (squash)
August 8, 2024 00:32
Closed
timethink
pushed a commit
to timethink/sglang
that referenced
this pull request
Mar 9, 2025
efschu
pushed a commit
to efschu/htsglang
that referenced
this pull request
Aug 28, 2026
…object) into merge/flip-window-0828
efschu
pushed a commit
to efschu/htsglang
that referenced
this pull request
Aug 28, 2026
…e mutants because the module is new sgl-project#980 shipped 1202 lines across seven files with no tests, including a new 482-line module (distributed/pp_object_recv.py). A red-first against the previous commit is the weak proof already reported twice in this window -- every arm would die on ImportError, equally red against a module whose functions are all 'pass'. So each invariant is paired with a mutant that breaks that invariant and nothing else, and the arm must go red for it. What is asserted is NOT 'does it time out'. The module's own law is that once the size header is received the payload is on the wire and must be taken off it, because a receiver that gives up mid-frame and later re-posts reads a payload AS a size and every later message on that stream is garbage. An abandoning bound is therefore worse than no bound: the unbounded wait hangs one rank, the abandoning one silently corrupts the stream. Every arm counts irecv posts -- exactly one size and one payload per object, however many steps expire and however many times the caller is handed a stall and returns. Covered: resumption across expired steps in both halves; a stall that raises AND leaves the frame resumable, with the same receive then completing; a transport error propagating as itself rather than as ObjectRecvStalled (sgl-project#734, so a dead peer stays distinguishable from a slow one); abort_after_s <= 0 restoring the exact pre-sgl-project#980 unbounded wait so the default path gains only a log line. MY OWN FIRST STRUCTURAL ARM WAS A FALSE POSITIVE, recorded in place rather than quietly rewritten. It asserted 'wait(timeout' not in the module source and went red on the module's DOCSTRING, which explains at length why it does not do that -- so the better the hazard was documented, the more surely the arm failed. Replaced by an AST walk for a real .wait(...) call carrying a timeout keyword, plus a can-fail proving that detector sees a genuine one. The prohibition matters because no runtime arm can observe it: a timed Work.wait closes the gloo pair (sgl-project#829) and would pass hermetically while killing a real boot on both sides. Test: 11 defs / 11 passed at 4ea93b8 (count probe), 1.84s, hermetic -- dist.irecv and ParkedWait stubbed in the module namespace, no gloo, no CUDA.
efschu
pushed a commit
to efschu/htsglang
that referenced
this pull request
Aug 28, 2026
…boot 8, at the mutation sgl-project#988 shipped 56 lines with no tests. Unlike sgl-project#980 this one admits a genuine behavioural red-first: the arms name no symbol it introduces (set_extend_range predates it) and assert sgl-project#965's co-derived invariant itself -- extend_range.start == len(prefix_indices), which prepare_for_extend asserts at the batch boundary. The specimen is boot 8's: PrefillAdder's host load-back grows prefix_indices in place, and before sgl-project#988 went on to the budget checks without touching extend_range, so every early return below that line left the previous visit's range behind a moved prefix. The arms drive a bail-out that happens AFTER the load-back and assert the invariant on the way out, deliberately not pinned to one specific return -- the fix's argument is that re-deriving at the mutation makes every current AND FUTURE exit inherit a consistent shape, and per-branch patches are how sgl-project#965 was paid for twice. An arm pinned to one exit goes stale the moment another is added. PICKING THAT BAIL-OUT TOOK A WRONG TURN, recorded in the file. add_one_req carries TWO near-identical 'rem_chunk_tokens is None and input_tokens >= rem_input_tokens' returns, one before the _lock_node block and one after. Configuring for the input budget hits the FIRST, so the prefix never moves and the file measures nothing -- caught only by the precondition assert, which is exactly what it is for. Nor can budget separate them: the post-load-back input_tokens is SMALLER than the pre one, so any budget tripping the second has already tripped the first. The truncation path (chunk budget 0) is reachable only after the load-back, and that is the one used. MEASURED, same file: 4ea93b8 (pre-sgl-project#988): 2 failed, 2 passed 8380555 (post) : 4 passed The reds read 'the prefix advanced to 4096 and extend_range still starts at 0' and '(0, 8422) != (4096, 4096)'. The two pre-fix PASSES are load-bearing: the prefix-preservation arm and the can-fail hold on the old tree too, so the failures are the defect and not a broken harness. Chain at 8380555: 164 defs / 164 passed across eleven suites.
efschu
pushed a commit
to efschu/htsglang
that referenced
this pull request
Sep 2, 2026
… group STOP, not a void Root (HANDOVER_1153_0902, PROVEN, pre-existing at the pin 228a66d): scheduler.py _pp_refuse_forwarded_schedule answered PPScheduleRefused ('sgl-project#791 FORWARDED SCHEDULE UNEXECUTABLE', PP1 reached 1 of the 2 rids PP0 named, batch_full_break) with a rank-local compensation: _pp_admission_pass_voided = True + emptied decision dicts -> mixin _pp_void_own_batch -> 'sgl-project#631 ROW-DELIVER BATCH NULLED slot=0 pass_voided=True' (boot_855_weg1b2 log 65000-65004) -> PP1 sent no proxy while PP0's slot stayed set -> nothing carried the void upstream (sgl-project#797 return trip: zero call sites since CUT V; sgl-project#1072 deleted the void relay) -> PP0's blocking _do_recv consumed PP2's NEXT output under this slot's label (log 65119) -> one output ahead for the rest of the boot -> the pp_to_tp arm at 21:42:05 turned the debt into an unproducible output -> sgl-project#980 ObjectRecvStalled 60 s / sgl-project#1071 PpChainRecvStalled 90 s. Same form sgl-project#1071 (169f53c) deleted for _pp_void_retracted_pass; this was the second writer of the same flag. F1 ROOT: the compensation is deleted, the detector is kept, the refusal is a group STOP. scheduler.py get_new_batch_prefill's except now does `raise self._pp_forwarded_schedule_stop(refusal) from refusal`; _pp_refuse_forwarded_schedule (flag write, dict emptying, sgl-project#971 re-home) is removed; the new _pp_forwarded_schedule_stop logs the kept 'sgl-project#791 PP-ADMISSION forwarded schedule REFUSED on rank N' line and returns a RuntimeError formatted by pp_admission_congruence .forwarded_schedule_stop_message: 'sgl-project#791 FORWARDED SCHEDULE UNEXECUTABLE STOP rank={rank} slot={slot} told=[{told}] reached=[{reached}] census={census} local={local} limiter={limiter} running_bs={running_bs} parked={parked} r2t_avail={r2t_avail} headroom={headroom} group_limit={group_limit} batch_full_setter={batch_full_setter} batch_full_at_loop_entry={batch_full_at_loop_entry}: {refusal}' Every probe is guarded (n/a on an unreadable value); the reached rids are recorded after the admission loop before any of the three raises; the batch_is_full setter site is recorded at each writer of the pass (no_allocatable_reqs_gate, count_arm, disagg_prefill_r2t_avail, add_one_req_NO_TOKEN). Group stop mechanism (existing, no new collective): the RuntimeError leaves run_event_loop; run_scheduler_process (scheduler.py 'except Exception') logs 'Scheduler hit an exception' and parent_process.send_signal(SIGQUIT) (+ killpg/kill_process_tree under SGLANG_KILLPG_ON_SCHEDULER_EXCEPTION); peers end in the bounded sgl-project#980 / sgl-project#1071 receives or the barlink dead-peer probe. F2 TRIGGER CLASS (PP0 order): on a forwarded schedule the follower's rank-local seat-count veto is not a verdict. pp_admission_congruence .rank_local_count_veto_applies(scheduled_extents) is True on PP0 and on every non-PP boot (unchanged expression) and False on a rank > 0 executing PP0's decision; the sgl-project#823 count arm, its batch_full_break, and the three pre-loop count gates of the same arithmetic (batch_full_or_empty_queue's batch_is_full half, min_free_slots_delay, no_allocatable_reqs) are gated on it. The physical allocator still refuses (NO_TOKEN -> membership refusal -> STOP naming the numbers). F3 SIBLINGS: mixin _pp_void_pass_without_upstream_launch's writer of the flag is provably unreachable (pp_upstream_void_pending returns False on every path; its final statement is `return False`) -- left with the proof in a comment. The row-authority _row_skip_plan exit now nulls a slot through _pp_null_frameless_slot, which applies the same named sgl-project#1020 'VOID REFUSED ON A LAUNCHED SLOT' guard as the void path; the guard is factored into _pp_slot_holds_unconsumed_launch(mb_id, site) and used by both sites. Matched check (error class: a follower still ends a PP0-launched pass silently): grep -rn '_pp_admission_pass_voided = True' python/sglang/srt -> only scheduler_pp_mixin.py (the unreachable sgl-project#801 writer); chain scheduler.py raise PPScheduleRefused (:11633/:11646, :11606) -> except PPScheduleRefused (:9441) -> raise self._pp_forwarded_schedule_stop (:9472) -> RuntimeError (:9623); the flag's only readers are _event_loop_pp_body (:4519 -> _pp_void_own_batch) and the scheduler.py void guard (:8955), neither reachable from the refusal. Tests (hermetic, CUDA_VISIBLE_DEVICES=""): new test_pp_forwarded_refusal_stop_1153.py (T1 STOP form + no void, T2 the count-veto helper + source pin, T3 the sgl-project#1020 guard on the frameless null, + the mixin:9389 unreachability proof): 17 passed after; on the parent ca4c6b7 (git worktree) it is a collection error (new names). test_pp_refused_pass_keeps_continuation_971.py: 12 tests that pinned the old compensation INVERTED with the withdrawal named in each docstring (24 passed after; the inverted 12 are red on the parent: 16 failed / 8 passed / 1 error across both files). Mutants: M1 flag write restored in the STOP builder -> 2 T1 red; M2a helper returns True -> 1 T2 red; M2b count-arm gate dropped in the loop -> 1 T2 red; M3 guard dropped from _pp_null_frameless_slot -> 1 T3 red. Bounded suite (7 unit/managers files + test/registered/scheduler): before 103 failed / 527 passed / 1 skipped / 27 errors; after 103 failed / 544 passed / 1 skipped / 27 errors -- per-file identical except the new file (+17). ruff check clean on all touched files; ruff format --diff hunk count unchanged vs parent (15/14/3, pre-existing) and the two test files formatted. Evidence tier: DESK-PROVEN. Boot-3 acceptance: grep -F 'sgl-project#791 FORWARDED SCHEDULE UNEXECUTABLE STOP' in the same second as 'sgl-project#791 PP-ADMISSION forwarded schedule REFUSED on rank', and zero 'ROW-DELIVER BATCH NULLED ... pass_voided=True' lines.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Thank you for your contribution, we really appreciate it. The following instructions will help improve your pull request and make it easier to receive feedback. If there are any items you don't understand, don't worry. Just submit the pull request and ask the maintainers for help.
Motivation
There is no need to adjust the
image_offset:extend_input_len< 2, there could not be an image input, so the adjust is also a dead code.This PR also adjusts
extend_input_leninto a separate function. We do not adjust the prefix indices after the request matches the radix tree, as there is a bug mentioned in #948. Instead, we adjust the token_ids to be matched.Checklist
pre-commit run --all-filesor other linting tools are used to fix potential lint issues.