Skip to content

Fix chunked prefill - #984

Merged
hnyls2002 merged 1 commit into
mainfrom
fix-chunked-prefill
Aug 8, 2024
Merged

hnyls2002 merged 1 commit into
mainfrom
fix-chunked-prefill

Conversation

@hnyls2002

Copy link
Copy Markdown
Collaborator

Thank you for your contribution, we really appreciate it. The following instructions will help improve your pull request and make it easier to receive feedback. If there are any items you don't understand, don't worry. Just submit the pull request and ask the maintainers for help.

Motivation

When the radix cache is disabled, the chunk cache would not update the prefix indices and last node, due to the API changes in #977

Checklist

  1. Ensure pre-commit pre-commit run --all-files or other linting tools are used to fix potential lint issues.
  2. Confirm that modifications are covered by complete unit tests. If not, please add more unit tests for correctness.
  3. Modify documentation as needed, such as docstrings or example tutorials.

@hnyls2002
hnyls2002 enabled auto-merge (squash) August 8, 2024 05:28
@hnyls2002
hnyls2002 disabled auto-merge August 8, 2024 05:28
@hnyls2002
hnyls2002 merged commit 6ed4e3b into main Aug 8, 2024
@hnyls2002
hnyls2002 deleted the fix-chunked-prefill branch August 8, 2024 05:28
@hnyls2002 hnyls2002 mentioned this pull request Aug 8, 2024
timethink pushed a commit to timethink/sglang that referenced this pull request Mar 9, 2025
efschu pushed a commit to efschu/htsglang that referenced this pull request Aug 28, 2026
…d becomes symmetric

Root (R7, boot 6): every void treated the ranks differently. Followers
obey the family's own doctrine ('A VoID IS A PARK, NOT A RETRACTION')
and keep prefix+pages; rank 0's batch members went through
_release_voided_request -> release_req -> reset_for_retract, which
clears prefix_indices and returns the pages. The two ends of the ring
then disagree about the same request every cycle: 169 told=0 refusals
(a FULL re-prefill of computed prefix per voided rid per cycle -- a
standing double-prefill-law violation on every void), 333 past-fill
refusals (total-token mismatch downstream of the same state), 6
duplicate-rid decisions (dedup-free tail requeue), rotating membership
that killed boots 3-6 after every earlier layer was closed.

Fix at both void release loops: pp_park_voided_batch_member -- the
member is queued UNRESET (prefix_indices, prefix KV pages, req slot,
retraction_count survive); only THIS pass's admission-side increments
are handed back: the prepared-never-run chunk via
_park_chunked_prefill_chunk(pass_allocated=True) and the admission
tree-lock ref via pp_give_back_admission_lock_ref. Dedup behind
pp_void_keeps_request (ordering load-bearing: reachability scans
running_batch, a different slot than running_mbs[mb_id]).
_release_voided_request keeps one caller: already-FINISHED members.
retract_decode/retract_all untouched -- their passes RAN.

Briefing premise falsified by the builder, recorded: keeping the
last_node lock ref would have REOPENED sgl-project#969 by a new route --
init_next_round_input re-matches with no dec_lock_ref anywhere on that
path, then admission takes a second ref. Pages-versus-claim is the
boundary, not keep-both: the prefix pages stay in the tree, the
request's claim is returned, which is the ordinary state of a queued
request. Both the wrong draft and the correction are in the docstring.

Instruments: 'sgl-project#984 VOID-PARK' per member (rid, prefix, kept_pages,
chunk_given_back, lock_ref_returned, slot, route) + per-loop census
INCLUDING the zero case, outside the sgl-project#801-spin suppression; #791b/#797d
now also print parked=/dup-skipped=. Boot 6 measured #791b 514x but
#797d 0x -- only the void-output loop is metal-proven; the census
closes exactly that blind spot next boot.

Execution proof (standing order): the rewired void-output loop IS boot
6's hot path (514 traversals); import smoke green; cold check driving
the REAL _pp_void_own_batch body: 23/23 (prefix survives, requeue,
dedup, one free = never-run chunk only, one lock-ref give-back, no
double free, finished-member path intact). Suites live in the test-agent
lane; predicted red there: test_pp_void_lock_ref_969.py encodes the
retract premise this commit retires -- its invariant (no leaked ref)
is preserved via the give-back.

Named posten, not bundled: sibling unreturned admission refs on the
#968b route and the kept chunked member; _pending_prefill_tokens_for
over-reports a parked member's backlog (honest figure: total -
len(prefix_indices)); req-pool kv_committed_len discrepancy duration now
unbounded for queued members (worth a test); sgl-project#963 prefix-floor wiring
stays its own posten.
efschu pushed a commit to efschu/htsglang that referenced this pull request Aug 28, 2026
…invariant is re-encoded, and a dead can-fail is revived

sgl-project#984 makes the void PARK rank 0's members instead of retracting them. Three
arms of the 969 suite encoded the old premise. None is deleted; each says in
place what changed and why the thing it was protecting still holds.

FIRST, THE FAILURE WAS NOT THE PREDICTED ONE. At 8be86f5 all six reds were
'TypeError: lambda got an unexpected keyword argument pass_allocated' -- the
harness stub for _park_chunked_prefill_chunk taking two positional args while
the fix passes a kwarg. A mechanical signature drift that MASKED the real
result: with the stub widened, the truth is 3 failed / 11 passed, and the
three are the premise, not the invariant.

test_the_request_is_retracted_exactly_once: asserted is_retracted and
retraction_count == 1. Now asserts NOT retracted and count 0. What the arm
protected is unchanged -- retraction_count feeds the solo-OOM abort ladder in
retract_decode, and a request must not be aborted at half the configured
retractions. A double count was the hazard when the void retracted; an
unasked count is the hazard now.

test_the_release_happens_before_the_reset -> renamed
test_the_ref_goes_back_while_the_prefix_handle_is_KEPT. Its assertIsNone(
req.last_node) encoded an ordering problem of a world where reset_for_retract
cleared the handle. sgl-project#984 removes that world and KEEPS last_node on purpose.
The invariant is untouched and is now what the arm states: the admission-side
inc_lock_ref must not outlive a pass that never ran (exactly one give-back --
not zero, a leak; not two, the sgl-project#929 underflow), while the prefix handle
survives so the next offer can report it as executed.

test_the_underflow_detector_can_actually_fail: its mutant patched
tree_cache.cache_finished_req, reached via release_req. sgl-project#984 does not take
that path, so the injected fault could no longer occur at all -- the arm
failed 0 != -1 because the mutant was never invoked. A can-fail whose mutant
became unreachable reports nothing while looking like a red someone would
'fix' by deleting the assertion. Re-aimed at pp_give_back_admission_lock_ref,
which is module-level for exactly this purpose by its own docstring.

MEASURED, same file:
  646f410 (pre-sgl-project#984): 3 failed, 11 passed
  8be86f5 (post)    : 14 passed  (14 defs, count probe)
Two of the three reds are behavioural -- 'None is not <node>' (the reset
cleared the handle) and 'True is not false' (the void retracted). The third
is an AttributeError on the symbol sgl-project#984 introduces, which is inherent to a
can-fail arm aimed at that symbol and is reported as the weaker red it is.
efschu pushed a commit to efschu/htsglang that referenced this pull request Aug 28, 2026
…e class behind both

The 797 suite went red at 8be86f5 in two arms. Neither is a regression;
both are sgl-project#984's premise arriving in a suite written against the old one.

test_a_chunk_started_this_round_is_un_started_not_kept asserted that a chunk
started this round is 'retracted like any member'. The void is now symmetric:
members are PARKED and re-queued with prefix and pages intact, and only an
already-finished request still takes the release path. The arm's actual point
is its other two assertions -- the chunk is UN-STARTED and is back in the
queue, so no rank is left uniquely mid-chunk -- and those never moved. How it
gets back to the queue is the disposal's business; that it gets there is
sgl-project#797's.

test_instr19s_raise_returns_when_the_pre_admission_value_is_blinded was a
DEAD CAN-FAIL. Its mutant blinded pp_void_keeps_request to force a retraction,
hence the reset shape, hence instr19's AttributeError. After sgl-project#984 an un-kept
member is parked rather than reset, so the mutant injected no fault at all and
the arm failed on assertTrue(chunked.retracted) with nothing having retracted
anything.

THE CLASS, since this is the second instance in one commit (the first being
test_the_underflow_detector_can_actually_fail in test_pp_void_lock_ref_969.py,
aimed at a cache_finished_req the void no longer calls): a change that
converts RETRACTION into PARKING disarms every can-fail proof whose injected
fault worked by causing a retraction. Such an arm does not report a safe
system -- it reports nothing, while looking like a red a tired reader would
'fix' by deleting the assertion.

The re-aim needed TWO neuters, and that is a result rather than a concession:
aiming at the park verdict alone left the arm red, because pp_void_keeps_request
still guards the top of the loop and continues past the chunked request before
any verdict is asked. The instr19 shape now requires defeating two independent
guards where one sufficed before. Both are neutered visibly, because a
can-fail proof must state the true depth of what it defeats rather than pick
the smallest edit that goes red.

Test: 31 defs / 31 passed at 8be86f5 (count probe), 89.51s.
efschu pushed a commit to efschu/htsglang that referenced this pull request Aug 28, 2026
…ce, so the guard gets a producer of its own

Third casualty of the same class, found by sweeping for it rather than by
waiting for it: this file was never in the chain list, so no sweep ran it.
Three of its six tests died on the stale two-positional-arg stub for
_park_chunked_prefill_chunk while the shipped call now passes pass_allocated=
-- a TypeError at scheduler_pp_mixin.py:3282, which masked the real result
entirely. Stub widened at all three sites; the truth underneath was 2 failed.

Both failures are sgl-project#984 removing this file's hazard AT ITS SOURCE, and the
file said so itself. Its precondition arm carried the warning 'if it did not,
this test no longer reproduces the boot and its green result is meaningless'.
That warning came true in the good direction: the disposal loop now asks
pp_park_voided_batch_member and PARKS an ordinary member, and only an
already-finished request reaches _release_voided_request, the only thing that
resets. The cross-slot reset that produced instr19 is no longer produced on
this path at all.

test_slot_a_disposal_reaches_slot_b_chunked_request -> renamed and inverted to
..._no_longer_RESETS_..., asserting parked-not-reset with a readable
extend_range.

test_the_retracted_chunk_is_not_carried -> renamed
test_whatever_is_left_in_chunked_req_is_READABLE. 'is None' was never the
point; surviving the next pass's unguarded dereference was. A parked request
is readable, and clearing it would discard the executed prefix for nothing.

AND THE GUARD KEEPS A PRODUCER. A guard whose only producer has disappeared
is a guard nobody exercises, and the next edit to the disposal could
reintroduce the state with no test objecting -- which is how sgl-project#798 arrived in
the first place. test_the_guard_still_clears_a_reset_shape_that_DOES_arrive
produces the reset shape through the SHIPPED decision (forcing
VOID_PARK_FINISHED down _release_voided_request) rather than by setting a
flag, asserts the shape really was produced, and then requires the clear to
fire.

Test: 7 defs / 7 passed at 8be86f5 (count probe; 6 before, +1 guard arm).
efschu pushed a commit to efschu/htsglang that referenced this pull request Aug 28, 2026
…aterialises it

Root (R9 census, boots 6-7): 506 of 513 void-causing refusals were ONE
rid failing the sgl-project#791 fill-length clause by ONE token -- rank 0 alone
holds a sampled output token that never crossed the tp_to_pp seam
(sgl-project#631 OUTTRACE: PP0 n=1 tail=[25], followers n=0), so 8447 vs 8446;
rank 0 alternates offers 7939/0, resetting the told-keyed sgl-project#944 streak
every pass: 1 termination in 506 laps. sgl-project#984 was measurably neutral on
these counts (its docstring premise falsified, recorded).

Part B (the serving fix): PPAdmissionEntry gains optional fill_len +
fill_tail (cap 8); build publishes on both entry branches; the follower
adopts at the ONE junction directly before the clause reads
local_fill_len, materialising the tail into a SHADOW pair honored only
by _refresh_fill_ids -- NEVER into output_ids: consumer sweep found two
unguarded readers (output_streamer slices output_ids into the CLIENT
payload on pp_rank 0; the per-rank max_new_tokens check would let one
rank finish a step early = the recorded cross-rank hang class). The
clause itself is untouched -- the fix ends the disagreement instead of
weakening the guard. Adopt is idempotent, self-cancelling once the rank
catches up, refuses loudly (log, no new raise) on cap/behind. Wire
codec rewritten index-based: a legacy 8-wide row decodes to
fill_len=None (byte-identical); the old fixed-arity unpack would have
crashed on any width skew between ends one commit apart.

Part A (abort net): the sgl-project#944 streak keys on refusal persistence --
_refused_since_offer written by PP0 as it absorbs its own void
(strictly local; a schedule refusal laps home with ZERO entries, which
is why UNRESOLVABLE read 0 beside a 506-refusal census), consumed in
note_offer; the told<=0 exemption no longer applies to a refused rid.
The alternation livelock now escalates in ~4 laps to the existing
terminator. sgl-project#955's healthy end preserved: an unrefused moving offer
never escalates.

Part C: sgl-project#986 -- pp_queue_orphaned_chunked_req returns the admission
tree-lock ref (adjudicated NEW-with-#968b leak). Plus the test-agent's
zero-case instrument at the rehome reset-shape return.

Instruments: sgl-project#987 FILL-ADOPT (both fills + appended count, gated
first+64th), sgl-project#987 FILL-REFUSE (cap/behind), refusal-streak visible in
the existing UNRESOLVABLE line. Bonus recorded: last_chunk is now
computed from the same number on both ranks.

Execution proof: import smoke on all five modules (CVD=""); call-graph
chains close at scheduler.py:9472/:8982, schedule_policy.py:1442/:1804,
event loop :9515; the 1390-clause ran 338x in boot 7 -- the adopt site
is on a proven hot path. 31/31 cold checks on the exact boot-7 numbers
(appended=1, id=25, veto-before/pass-after, 4-lap abort vs 506); one
fixture self-catch recorded. Suites live in the test-agent lane.
efschu pushed a commit to efschu/htsglang that referenced this pull request Aug 28, 2026
…eeps its one ref

Boot 9 died in 24s on the lock_ref>0 underflow (full_component.py:320,
via cache_unfinished_req's dec at the FIRST chunk stash). R12 closed the
pair on metal: ONE inc (schedule_policy.py:2021, transferred
stash-to-stash by unified_radix_cache.py:1354-1355) met TWO decs -- the
give-back at the void (slice line 270, lock_ref_returned=True) and the
stash's own dec. Both give-back sites (sgl-project#984 park, sgl-project#986 orphan-queue)
assert 're-admission takes a FRESH ref', which is FALSE for a carried
chunk: the chunked continuation holds ONE admission ref across all its
chunk passes (schedule_policy.py:1441/:1747/:2008, scheduler.py:9059).
The sgl-project#988 guard removed the accidental compensating inc of the second
waiting-queue visit, exposing the imbalance (that causal half is
desk-inferred; the pair itself is measured).

Fix, two lines at the ONE shared function: pp_give_back_admission_lock_ref
returns False for the request currently held as self.chunked_req --
ownership discriminated by identity; the sgl-project#986 orphan route keeps giving
back (its orphan has by definition LEFT the field). The assert is never
weakened.

Context (R12's design evaluation, register 20:xx): the user's re-entry
design is ALREADY shipped as sgl-project#856/W30-FIX-B but scoped by _live_reqs to
batch residents; extending it is measurement-gated on sgl-project#972/sgl-project#975 + a GDN
host anchor (ABSENT today) -- registered as posten, not smuggled in
here. Import smoke green; suites in the test-agent lane.
efschu pushed a commit to efschu/htsglang that referenced this pull request Aug 28, 2026
…d acquired

Boot 10 (b27d7c2) died on PP0 at 19:58:17 with
`AttributeError: 'NoneType' object has no attribute 'unsqueeze'` at
mem_cache/unified_cache_components/mamba_component.py:1056, reached via
_pp_process_batch_result -> process_batch_result_prefill ->
maybe_cache_unfinished_req -> cache_unfinished_req, rid
e3d6a9ff11c5460cb115f266f2b187a3.

ROOT (class, not effect). The admission loop's revert site in
managers/scheduler.py frees `req.mamba_pool_idx` whenever a request is not
added to `can_run_list`, guarded by `not req.session`. That predicate is
upstream's PROXY for "nobody else already owned this slot" -- correct
upstream, where a session was the only carrier of a pre-existing slot. This
fork invalidated it: sgl-project#984 (void-park), #968b (rehome-on-displace) and sgl-project#971
(rehome-on-refusal) re-queue requests UNRESET, holding the live mamba slot
of a prefill batch that is still IN FLIGHT. The revert then hands a live
owner's slot back to the allocator and nulls the field; the in-flight
batch's result arrives and dereferences it.

The silent direction is the same defect: when the free lands outside the
in-flight window, HybridReqToTokenPool.alloc re-acquires a fresh slot with
mamba_needs_clear=True -- a ZEROED recurrent state in place of the
request's own (sgl-project#767's shape), with no crash and no log line. Both close at
the provenance test.

FIX, at the one junction. Stamp provenance where the slot is acquired
speculatively by an admission round's prefix match --
unified_cache_components/mamba_component.py (COW resume + host load-back),
mamba_radix_cache.py, hi_mamba_radix_cache.py (both siblings) -- and free
at the revert site only when that stamp is set. The stamp is cleared the
moment HybridReqToTokenPool.alloc takes the slot into a batch, and on every
release/reset path (free_mamba_cache, _rollback_alloc, release_kv_cache's
early branch, reset_for_retract, Req.__init__ default). This is sgl-project#984's own
"pages versus claim" doctrine applied to the third increment: the lock ref
(sgl-project#990) and the chunk (sgl-project#971) already carry provenance; the mamba slot was
the one that did not. It opens neither sgl-project#929 (deficit) nor sgl-project#924 (surplus) --
the stamp is one-shot and dies with the slot.

AFFIRMATIVE REPORTING. None of the five mamba free sites named a rid, which
is why the specimen could only be reached by elimination. The refused
give-back now logs rid + rejection result, so the next boot is the metal
proof of the root rather than a yes/no on the crash.

BACKSTOP, named as a defect. mamba_component.prepare_for_caching_req now
declines retention with a counted, rid-bearing warning instead of
dereferencing a None slot. This is NOT the fix and is stated as such in
place: it costs the step's retention and re-prefills the chunk, which the
standing no-double-prefill order forbids as a routine. It exists because a
crash here takes the whole instance while a decline takes one step.

SIBLING SWEEP: all five writers of `mamba_pool_idx = None` audited and the
stamp cleared at each. gdn_slot_runtime.py:136 is not wired on this build
(the gdn_resident_state_slots gate is off, no "armed" line in any boot log)
and is left untouched. streaming_session.py's clears stay covered by the
retained `not req.session` term.
FUTURE CHECK: a new producer of a pre-existing slot cannot reopen this
silently -- it either stamps at its acquire (and is given back correctly)
or it does not (and is kept), and the backstop's counter names it with a
rid if it manages to drop one anyway.

Evidence: desk. py_compile + import smoke on all seven touched files;
lifecycle probe (Req default False, reset_for_retract clears). Belegstufe:
DESK-BEWIESEN, not metal -- boot 11 is the test.
efschu pushed a commit to efschu/htsglang that referenced this pull request Aug 28, 2026
The order was to check the RESET CLASS, not to find the next single root.
The census answers it, and the answer moves the target.

THE RESET IS CORRECT FOR ITS SCOPE, and its scope is `_live_reqs`
(phase_flip_runtime.py:1280): running_mbs, last_mbs, running_batch,
last_batch, chunked_req. `reset_for_retract` clears 33 fields
(schedule_batch.py:1846-1959) including last_node, prefix_indices,
mamba_pool_idx. The object surgery the ~15-instance corpus is made of does
not sit on the residents. It sits on the HOLDERS THE RESET DOES NOT
ENUMERATE.

THE RATCHET IS A TAUTOLOGY, and that is the finding that outranks the rest.
test_cutover_discovery_diff_859.py:54-58 builds `undeclared` by filtering
to what IS declared, then asserts those are declared; `known` (:50-53) is
a dead store. Run hermetically with two new attribute writes planted into
the module source: GREEN. It has been green since it was written and would
be green if the cutover grew fifty participants. The one mechanism built to
turn an undeclared survivor into a testfail measures nothing -- the
indicator law, live, in this tree.

Second defect, and it explains the first: `_cutover_source()` (:33-36)
reads the WHOLE 11k-line module instead of the cutover. Module-wide there
are 163 undeclared writes; in the cutover's own call-closure there are 17.
Fixing the predicate alone yields permanent red at 163 posts, which is why
inverting it was tempting. Both halves have to move together.

Cut 1 holds: 44 passed / 11.82 s hermetically, no symbol drift.

FOUR BRIEFING PREMISES THE CODE REFUTES, recorded because the campaign kept
paying for them: cutover_participants.py has NO production importer
(schedule_batch.py:1904 is a comment); the R12 eval is wrong about
chunked_req, which IS enumerated (:1361-1364) and IS cleared (:1855); sgl-project#955's
_escalated is carried by a written invariant (pp_admission_congruence.py
:560-575), not un-cleared; and sgl-project#984's justification for keeping last_node
cites a line that no longer exists and a claim -- "called unconditionally
for every waiting-queue member" -- that five guards and a break contradict.

The corpus splits in two. Family A (seam) is declared and unenforced.
Family B (the void/park loop, the larger half) has no registry at all, and
its three void paths are two duplicated disposal loops split by rank.

Evidence tier DESK-PROVEN throughout. No metal was touched, so nothing here
says fixed. Doc only, no code, no test.
efschu pushed a commit to efschu/htsglang that referenced this pull request Sep 1, 2026
…pping form

`_1058_note_told_vs_local` was called from exactly one place: the `else` arm
of the three-way clamp selection in `_get_new_batch_prefill_raw` -- the arm
taken when NEITHER clamp is armed. Row authority is the shipping default
(`SGLANG_PP_ROW_AUTHORITY` unset -> "1", `pp_size > 1`), so on every boot this
campaign actually runs the FIRST arm is taken and the observation point was
unreachable.

Measured, boot_855_1071cut (169f53c): `sgl-project#1058 TOLD-VS-LOCAL CENSUS
evaluated=0 absent=0 agree=0 told_over=0 told_under=0` on ALL THREE ranks.
`told_over=0` reads as "PP0 never overshot" and means "nobody ever looked" --
on the ONE axis where rank divergence is proven on metal (sgl-project#944 told=8192
local=0; sgl-project#984 told=8192 local=4096, 514x UNEXECUTABLE -> 512 voids). The
census text already guards the reader ("evaluated=0 means the observation
point was never reached on this rank"), so the instrument never lied; it was
simply never fed on the configuration that ships.

Same defect class as #995e's WIDTH-AGREEMENT, measured earlier today: a
counter that cannot compare on the ranks and the configuration that matter
(skipped=54/54, 52/52, 3/3, 3/3, reason `no_local_input_ids`). An instrument
is a finding only once it is checked THAT it measures what it claims.

THE FIX IS THE ORDERING, not just the call. `truncate_prefix_to` sets
`prefix_indices` to `told`, so an observation taken after it compares told
against itself and reports agree=100% by construction -- the category error
that killed boot 1071cut in its own way (operands from two different
pipeline positions). Both operands are therefore read at the SAME point:
this rank, this admission, PRE-clamp. `told` = PP0's published prefix for
the rid; local = what this rank actually holds.

Measure-only, exactly as before: no gate, no collective, no behaviour change.
The callee wraps its whole body in try/except and the caller ignores its
return. Desk-proven: py_compile, import smoke, and an ast check that the
note call precedes the truncate in BOTH clamped branches.

Effect: the acceptance boot now measures the ADMISSION/told axis as well as
the HiCache-read axis, instead of reporting a structural zero on the axis
that has actually killed boots.
efschu pushed a commit to efschu/htsglang that referenced this pull request Sep 24, 2026
…H42)

Task sgl-project#118 (Nutzer 24.09. 07:00Z: bei Agentenlast kommen viele kleine
Prefills), Folge von H37.

Wurzel: Gruppe P haelt mit dem END-ANCHOR (sgl-project#1233) das letzte Token jedes
Prompts zurueck; der Koerper [0, N-1') wurde DIE chunked-Fortsetzung
(Scheduler.chunked_req, ein Feld, eine je Durchlauf sgl-project#959/sgl-project#996). Der zweite
ganz passende Prompt eines Durchlaufs bekam OTHER ([sgl-project#967] SECOND
CONTINUATION REFUSED add_one_req/end-anchor), die Zulassungsschleife brach
ab: je Forward erreichte hoechstens EIN Request sein Ende, jeder Schwanz
(1-4 Token) lief als eigener Forward.

Schalter SGLANG_WEG2_ENABLE_P_MULTI_ANCHOR_TAILS (Default aus; wirkt nur
mit SGLANG_WEG2_END_ANCHOR, also nur auf P, und nur ohne Draft auf P):
jeder END-ANCHOR-Koerper wird ein ANKER-SCHWANZ seines eigenen Requests
(managers/anchor_tails.py, Kollaborateur mit schmalen Argumenten).
PrefillAdder: new_anchor_tails, _mint_anchor_tail ohne sgl-project#996-Assert und
ohne sgl-project#959/sgl-project#967-Verweigerung; dieselbe Geometrie-Regel (Ausdehnung endet am
Anker-Schnitt der Fuellung) an allen drei Praegestellen (add_one_req
ganz passend, add_one_req gekuerzt, _add_scheduled_req der Folgeraenge mit
getragenem last_chunk-Urteil). Scheduler.anchor_tails: aus dem Merge
ausgenommen und gestasht wie chunked_req (je Request eigener Ankerknoten,
MAMBA-ARENA-Claim, Tail-Capture), im naechsten Durchlauf VOR der Schleife
alle ueber add_chunked_req wieder zugelassen (Budget, geplante Ausdehnung
sgl-project#791/sgl-project#996, carried_chunk-Locks wie beim einen Feld; nicht benannte Schwaenze
bleiben sgl-project#992, abweichender told-Prefix sgl-project#994), danach fuellt die Schleife das
Chunk-Budget mit neuen Koerpern. ScheduleBatch.weg2_anchor_tail_bodies
nimmt die Koerper beim spaeteren last_batch-Merge aus (PP-Slot).
contains_last_prefill_chunk ohne Schwanz byte-gleich. Void (#797d):
_pp_void_own_batch stellt die Schwaenze des Slots wieder her, parkt jeden
wieder zugelassenen Schwanz (Inflight-Rueckgabe nur wenn genommen) und
laesst die Praegungen des verworfenen Durchlaufs als Mitglieder parken
(sgl-project#984). Abbruch: Schwanz laeuft seinen letzten Forward auf allen Raengen und
endet mit FINISH_ABORT. Leerlauf/Quiesce/Zensus/Flip-Autoritaet zaehlen
die Schwaenze; process_batch_result_prefill haelt alle Mittel-Chunks vom
Stream fern; tail_handoff haelt bei Schalter Captures und Teildateien fuer
--max-running-requests Requests (statt 2).

Tests test_weg2_p_multi_anchor_tails_h42.py: echter PrefillAdder in einer
PP3-Form (PP0 entscheidet, PP1/PP2 fuehren je rid aus): 4 x 4k in einem
16k-Forward, Schwaenze gemeinsam im naechsten, Ausdehnungen/Anker/Ausgabe je
Request auf allen Raengen, 8 Requests gemischt mit einer chunked-Fortsetzung,
Abbruch mitten im Burst, Void je Schwanz, Schalter aus = aufgezeichnete
Spur des Baums vor H42 (4 und 8 Requests), Startzeile, Capture-Grenze.
18 Tests, 15 Mutanten fallen. Gate (gate_tests.sh-Kopie + H37 + END-ANCHOR/
Void-Kette) unter incg.sh: keine neue Rotmeldung gegen den Baum vor H42.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant