Skip to content

fix: use e2e and unit test only for original repo or pr - #912

Merged
zhyncs merged 1 commit into
sgl-project:mainfrom
zhyncs:new
Aug 4, 2024
Merged

zhyncs merged 1 commit into
sgl-project:mainfrom
zhyncs:new

Conversation

@zhyncs

@zhyncs zhyncs commented Aug 4, 2024

Copy link
Copy Markdown
Contributor

Thank you for your contribution, we really appreciate it. The following instructions will help improve your pull request and make it easier to receive feedback. If there are any items you don't understand, don't worry. Just submit the pull request and ask the maintainers for help.

Motivation

Please explain the motivation behind this PR and the goal you aim to achieve with it.

Modification

Briefly describe the changes made in this PR.

Checklist

  1. Ensure pre-commit pre-commit run --all-files or other linting tools are used to fix potential lint issues.
  2. Confirm that modifications are covered by complete unit tests. If not, please add more unit tests for correctness.
  3. Modify documentation as needed, such as docstrings or example tutorials.

@zhyncs zhyncs changed the title fix: use e2e and unit test for original repo or pr fix: use e2e and unit test only for original repo or pr Aug 4, 2024
@zhyncs
zhyncs merged commit 4f00525 into sgl-project:main Aug 4, 2024
@zhyncs
zhyncs deleted the new branch August 4, 2024 06:34
timethink pushed a commit to timethink/sglang that referenced this pull request Mar 9, 2025
efschu pushed a commit to efschu/htsglang that referenced this pull request Aug 26, 2026
…, so 411 declines have no verdict

Routing from the write-side agent (sgl-project#872) narrowed Posten 0 correctly: the store
is healthy, 24277 canonical pages live during the R7 window, and the historic
"store is empty" readings probed a path this boot does not use. So the 671
refused walks are a MATCH problem, not a content problem. The question that
follows immediately is the one nobody could answer: a walk that matches nothing
should fall through to an L3 prefetch, and prefetch was attempted on only 264
of 675 sampled walks.

THE OBVIOUS ANSWER IS WRONG, and the code says why. It is tempting to read the
411 as the same MambaComponent refusal declining twice -- one root, two numbers.
It is not:

  * The prefetch anchor IS mamba-gated, but by an aliasing the base contract
    does not describe. base_prefix_cache.py:201-210 separates `last_host_node`
    (the L3 prefetch anchor) from `best_match_node` (the validator-consensus L2
    anchor) -- two fields, two purposes. UnifiedRadixCache sets them equal under
    HiCache (unified_radix_cache.py:1443-1449). Deliberate and commented, but it
    contradicts the field's own documentation, and two fields that are
    documented different and implemented identical are a drift source (sgl-project#747's
    shape). Named, not changed here.
  * Even so, the refusal does NOT decline the prefetch. Total refusal leaves
    best_match_node at the root, and the caller admits the root on purpose:
    `locally_eligible = last_host_node.backuped or last_host_node is
    self.tree_cache.root_node` (scheduler.py:4933-4935). A cold request with no
    match is supposed to prefetch. The first term holds.
  * The second term (>= 256 new tokens) holds for agent prompts of thousands.

That leaves the rate limiter -- AND THERE THE EVIDENCE STOPS, which is the
actual finding. None of the three terms is counted or logged. The 411 declines
are silent. Nobody can attribute them from this boot, and no amount of
re-reading the log will change that.

    eligible = (locally_eligible
                and prefetch_length >= self.prefetch_threshold
                and not self.cache_controller.prefetch_rate_limited())

Three unrelated verdicts wearing one boolean, remedies in three different
files. The sgl-project#914 shape one module over: blame without a defect.

So this is an instrument and NOT a policy change. The three terms are the same
three, evaluated in the same order, with the same outcome; `eligible` is now
`reason is None`, one expression, so the predicate and its explanation cannot
drift apart the way sgl-project#747 records these very lineages drifting. The first
failing term is named rather than all of them -- a request can trip several,
and summing them would double-count exactly as
`refused_tokens_by_component` is documented to.

Not env-gated, unlike the sgl-project#904 match census. That one builds an object and
walks validators a second time, so it pays for itself only when armed. This is
one integer increment on a path that already builds a RadixKey and takes a host
lock -- and a counter you must remember to arm cannot answer "was it ever
tried", which is the whole question. `attempted` is counted alongside the
refusals so the denominator is local (sgl-project#873: a denominator reconstructed from
another log is how a narrowed candidate set reads as a decomposition), and an
unrecorded gate says "no observation" rather than a silent zero (sgl-project#829).

HYPOTHESIS, RECORDED AS ONE. prefetch_capacity_limit is `0.5 *
mem_pool_host.size` (cache_controller.py:729), and that size is not constant
across a phase flip: sgl-project#905 measured the two host tiers at 703472 rows (PP) and
30518 (TP), 23x apart, putting the TP-phase budget near 15259 tokens -- under
four prefetches of the 4096 this window actually completed, against 5
concurrent requests per flip. The order of magnitude fits 411/675, and I am
deliberately not fitting it exactly: the PP phase has ample budget, so the
number cannot come from there alone. If it holds, the same host-tier asymmetry
behind sgl-project#905's double-free is also starving the TP-phase prefetch budget -- one
defect, two effects, to be checked rather than assumed. The counter decides it
on the next boot instead of re-arguing it.

RED-FIRST, four mutants, each red, restored green (14 tests):
  revert to the shipped conjunction (no counter) ......... 4 failed
  eligible computed twice instead of derived from reason . 2 failed
  attempts not counted (denominator lost) ................ 2 failed
  no-observation reported as attempted=0 ................. 1 failed

ONE SELF-INFLICTED DEFECT, found and fixed in the same pass, recorded because
the form is evidently easy to commit. My first version of
test_the_rate_limit_check_is_still_called_at_most_once counted occurrences in
the function's RAW source and went red because a PRE-EXISTING comment at :2674
mentions `prefetch_rate_limited()`. That is the sgl-project#908 substring defect -- prose
used to reach a verdict about code -- in a test of mine, an hour after I
rejected the same form in someone else's patch. Comment lines are now stripped
before counting.

GATE test/registered/unit/mem_cache serial, CVD="": 3 failed, 1950 passed,
1658 skipped. Both axes agree (summary 3, ANSI-stripped extraction 3). Two are
test_acceptance_emitters_758::RefillTiming, baseline-proven pre-existing
earlier in this branch; the third is test_pool_invariant_double_owned_912, a
concurrently-developed test belonging to sgl-project#912 and untouched by this commit.
The sgl-project#904 census suite is green against this change (62 passed together with
the sgl-project#913/sgl-project#914/sgl-project#915 suites), which is the regression that matters: note_refused
gained an optional third argument and its existing callers must not care.
efschu pushed a commit to efschu/htsglang that referenced this pull request Aug 26, 2026
…ated gap and a

stale double-owned reading as a leak, killing all three schedulers on idle

Five on_idle firings (two boots) showed available+evictable+withheld
exceeding total by exactly 22, never a deficit -- the opposite sign from
sgl-project#814/sgl-project#902/sgl-project#832/sgl-project#856, and never previously routed through the sgl-project#822
ownership authority. Decomposition: 21 rows from
_check_full_pool reading TokenToKVPoolAllocator.available_size() (a raw
sum of free_pages+release_pages, allocator/token.py:52-54) instead of the
union read_free_rows() already used by the phase-flip census and the sgl-project#822
audit (kv_row_ownership.py:743-843); 1 row from a single row simultaneously
claimed by the free list and the radix tree, the sgl-project#822 authority's own
EXCLUSIVITY "claimed by multiple" finding, never subtracted before.

Fix, both additive, neither a tolerance/epsilon:
- _check_full_pool now reads available via read_free_rows() when the
  allocator can enumerate; composite/watermark allocators keep
  ps.full_available_size unchanged.
- _check_pool_invariant gained a double_owned term, subtracted, sourced
  from allocator.double_owned_slots (the sgl-project#822 EXCLUSIVITY count).

Two defects found in review of the first cut, both fixed here:
1. The double_owned filter in
   phase_flip_runtime.py::_census_ownership_audit selected violations by
   substring match on Violation.detail ("more than one owner" in
   v.detail) -- prose used as control flow, against Violation's own
   "detail is never load-bearing" contract, the same shape as the
   line_gate substring defect (sgl-project#908). Fixed by giving Violation a
   structural kind field, set to EXCLUSIVITY_DOUBLED or
   EXCLUSIVITY_UNOWNED at its two EXCLUSIVITY construction sites in
   kv_row_ownership.py, and filtering on kind instead of detail text.
2. double_owned_slots is published only at phase-flip census (seam)
   events and read by on_idle at unrelated times; a reading taken before
   authority.retire() drops every claim it was computed from is stale
   past that point. Fixed by clearing double_owned_slots to None (not 0)
   at the exact point _retire_row_id_space calls authority.retire().
   Checked analytically whether staleness alone could mask a genuine
   deficit: it cannot, for any non-negative reading, because double_owned
   is subtracted and can only push total_accounted further below total.
   It can mask an unrelated, coincidentally-equal-sized surplus; that
   residual case is disclosed, not eliminated, and bounded by the
   cutover-clearing fix.

Tests (test/registered/unit/mem_cache/test_pool_invariant_double_owned_912.py,
14 tests, 31 subtests): reconstructs all five measured specimens,
red-first against unfixed code, green after the fix; separate mutants for
the original subtraction, the kind-based filter, and the retire-time
clearing, each shown to fail only its own guard when reverted; a
parametrized check that no non-negative stale double_owned value can turn
a manufactured 100-row deficit into a false pass.

Gate, hermetic (CUDA_VISIBLE_DEVICES=""), attribution proven by rerunning
identical failing node-id lists with these three files restored to HEAD
(git show HEAD:<path>) and diffing failure sets -- both axes' failures are
identical with and without this fix, i.e. pre-existing:
- mem_cache/: 2 failed, 1951 passed, 1658 skipped, 392 subtests passed
  (139.10s). Failures: test_acceptance_emitters_758.py RefillTiming x2,
  RuntimeError: No CUDA GPUs are available.
- managers/: 15 failed, 4631 passed, 18 skipped, 563 subtests passed
  (719.09s). Failures: test_arena_high_water_631.py x7 and
  test_restore_never_rebuild_677.py x4 (RuntimeError: No CUDA GPUs are
  available), test_phase_flip_rotation_wiring_809.py x4 (Exception:
  retry() exceed maximum number of retries) -- all reproduced identically
  against HEAD-restored files.

ruff check and codespell clean on every line touched by this change (both
tools also flag pre-existing issues elsewhere in phase_flip_runtime.py and
invariant_checker.py, outside this diff's hunks, left untouched as out of
scope for sgl-project#912).
efschu pushed a commit to efschu/htsglang that referenced this pull request Aug 26, 2026
…project#914 mamba refusal component, sgl-project#915 prefetch decline instrument, sgl-project#912 invariant checker available_size vs read_free_rows) into the flip train
efschu pushed a commit to efschu/htsglang that referenced this pull request Aug 27, 2026
…address the bound pool, and sgl-project#912's remainder: live double-claims read at check time) into the flip train
efschu pushed a commit to efschu/htsglang that referenced this pull request Aug 27, 2026
…t when Full triggers it

Boot 2f died 40 s in, 14 s after the first real cache hit. Not a wedge -- the
scheduler RAISED at on_idle:

  pool memory leak detected! [full] total=465669, available=132894,
    evictable=346, protected=8192, session_held=0, uncached=0,
    withheld=324357, double_owned=8129, double_owned_src=live

THE DEFECT IS EXCLUSIVITY, NOT ACCOUNTING, and the ledger is only what
noticed. `double_owned_src=live` means the number came from
`_live_double_claimed_rows`, which is `len(free_rows & cached_rows)` over the
allocator's real free set and `all_values_flatten()` -- "the SIZE OF AN
ENUMERATED INTERSECTION, the ids are known, not estimated". So 8129 rows were
genuinely on the free list AND named by live tree nodes. Those ids stay
matchable, which is the sgl-project#767 direction: a later prefix hit can serve KV out of
rows that have already been reissued.

ROOT, one line. `full_component.evict_component` (:115-118) frees the device
rows on ANY cascade that reaches it and deliberately leaves `cd.value` set --
correctly, because `free_swa` has to read it first, so the clear is deferred
to `_cascade_evict`. That deferred tombstone then asked

    trigger.component_type == BASE_COMPONENT_TYPE

which is a DIFFERENT QUESTION from "were Full's rows freed". A cascade
triggered by MAMBA or SWA (`mamba_component.py:529`, `swa_component.py:441`)
that reaches Full freed its rows and never tombstoned them, so the node went
on naming ids the allocator had handed back -- permanently, not for a window.

The fix asks the question the deferral actually owes: `base_rows_freed`, set
when Full is the trigger OR when the cascade loop evicts Full, and the
tombstone fires on that. The opposite direction is preserved and pinned: a
cascade that never reaches Full must NOT clear `value`, because stranding live
KV is a deficit -- the worse defect in the other sign.

SIBLING SWEEP, and the class is a single instance: `mamba_component.py:484`
and `swa_component.py:393` both clear `cd.value = None` INLINE, immediately
after freeing. Only Full defers, only Full needed a completion condition, and
only Full got it wrong. Nothing else to fix.

WHY NOW, AND WHY IT IS OLD (#wurzel-vor-wirkung): the intersection can only be
as large as the tree. Until the mamba checkpoint grid stopped vetoing every
anchor, no prefix match ever succeeded -- the tree stayed at `evictable=1` and
this term measured about ONE row, which is exactly the 2c precedent that
motivated sgl-project#912's live reading. The first real 8538-row cached prefix turned it
into 8129. The defect is not in a516b37's diff; a516b37 only made hits
reachable.

THE ARITHMETIC, since it names the sign: PP0's raw ledger is
132894+346+8192+324357 = 465789, a 120-row SURPLUS over 465669. Subtracting
double_owned=8129 makes it an 8009-row DEFICIT, and `_check_pool_invariant`
raises on `!=`, so either sign is fatal. The two peer ranks sit at a 64-row
surplus with double_owned=0; the 56-row difference is exactly PP0's extra free
rows. All three trees hold the same 8538 rows (8192+346 on PP0) -- only the
lock and the free-list overlap differ, which is why the rank that matched is
the rank that died.

RELATIONSHIP TO THE NEIGHBOURS, checked rather than assumed: sgl-project#916 (lawful
tree/request share) and sgl-project#922 (stale _resident_rows) are different populations
and are untouched. sgl-project#912 is the DETECTOR, and it worked exactly as designed --
it was built to catch a row claimed by two owners and it caught one. This is
not an indicator defect; the indicator was right.

Tests, hermetic, CUDA_VISIBLE_DEVICES="": test_cascade_tombstone_927.py,
5 passed. Red-first proven: restoring the old trigger-gated condition turns
`test_the_tombstone_is_not_gated_on_who_triggered` RED, then reverted.

SPECIMEN 2 ARRIVED MID-BUILD AND CHANGED THE VERDICT. The targeted repro
(boot_2f_a516b3750b_0827_0649.log) reproduced it with double_owned=8192 -- the
WHOLE hit prefix, so the first specimen's 63-row gap was incidental, not a
discriminator. And its ledger BALANCES EXACTLY:

  available 125865 + protected 8192 + evictable 1111 = 135168
    = the post-cutover free pool ("POOL CENSUS post-cutover ... free=135168")
  135168 + withheld 305265 = 440433 = total          -> surplus ZERO

So on that run there is no leak and nothing to correct: the raise is caused
ENTIRELY by subtracting 8192 from an exact ledger. That means the two free
readings disagree by exactly the protected count -- `double_owned` is derived
from `free_reading.rows` (an ENUMERATION) and subtracted from a ledger built
on `available_size()` (a COUNT), and `FreeRowReading`'s own docstring warns
that "HOW MANY" and "WHICH ONES" are different questions on this allocator
family. Nothing compared them.

SO THIS COMMIT DOES NOT CLAIM TO CLOSE 2f, and says so rather than letting the
tombstone fix take credit for a crash it may not cause. The second change is a
DIAGNOSTIC, not a gate: when the live reading fires, it now logs the
enumeration size against the count and the delta. Whether the enumeration
over-reports or `available_size()` under-reports decides which side is the
defect, and the fix differs on each side -- so it names the disagreement and
leaves the verdict to the reader instead of guessing.

The tombstone fix stands on its own merits, independent of 2f: a live node
naming freed rows is wrong whatever the ledger says about it, and the sibling
parity argument (both peers clear inline) is what proves it rather than the
specimen.

OBSERVABLES for the next boot: the new "sgl-project#927 FREE READING DISAGREES WITH
ITSELF" line and its delta -- if it prints delta == protected, the enumeration
is the defect and the correction term must be built from the count; if it does
not print at all while double_owned is still large, the rows really are doubly
claimed and the tombstone path is the one to keep pulling.
efschu pushed a commit to efschu/htsglang that referenced this pull request Aug 27, 2026
…he allocator per access

sgl-project#927 IS A CHECKER-SIDE DEFECT, AND THE on_idle RAISE WAS A FALSE POSITIVE.
Nothing in the KV pool is repaired here, because the evidence now says nothing
in it was broken: the guard read the wrong object.

THE MECHANISM, and it is constructed rather than inferred.
`SchedulerInvariantChecker` stored `token_to_kv_pool_allocator` as a dataclass
field taken once at construction (`scheduler.init_invariant_checker`). The
phase flip REBINDS that allocator -- `hicache_phase_binding._stamp` sets
`token_to_kv_pool_allocator = incoming.allocator`, and `phase_pools_for` takes
that object from the incoming phase's OWN worker stack (`:341-343`), so it is a
DIFFERENT OBJECT per phase. But `readers_of` names exactly three readers --
scheduler, tree_cache, cache_controller -- and the checker is not one of them.
Its own docstring states the consequence in advance: "a reader this function
forgets is a reader the rebind silently leaves behind." Nothing else refreshed
it either; `vram_dial._refresh_capacity_snapshots` touches only
`max_total_num_tokens`.

So after the first cutover, `cache_controller.load` allocated load-back rows
from the INCOMING allocator (it IS rebound) while the ledger read the BOOT one.
Both address the same id space, so those rows read as FREE to the checker while
the tree legitimately named them, and `_live_double_claimed_rows` reported the
overlap as `double_owned src=live` -- in the magnitude of the loaded-back
prefix, on the rank that matched, at the instant `load_back` filled the nodes'
`value`. Every measured property of the crash, including the timing that
refuted four earlier candidates.

WHY THE TIMING FITS EXACTLY, which is what makes this the answer rather than
another candidate: the tree was refilled long before the hit, by
`_insert_helper_host`, which creates nodes carrying ONLY `host_value` (`:1788`)
-- `value` stays None, and `all_values_flatten` reads `value`. So the nodes were
invisible to the ledger from the cutover until `load_back` populated them. That
is why `double_owned` read 0 on every census and then jumped at the first hit.

THE FIX IS PER-ACCESS RESOLUTION, AND A `readers_of` ENTRY WOULD NOT HAVE
WORKED AT ALL. That was the smaller-looking option and it is unavailable: all
three affected components are `@dataclass(kw_only=True, slots=True,
frozen=True)`, and `_stamp` moves a reader by `setattr`. On a frozen instance
that raises, `rebind` catches it mid-set and escalates to `RebindIncoherent`
("One reader failed mid-way: the set is now split") -- so adding them to the
dict would have converted a silent stale read into a hard flip failure. Per
access is the only shape that works here, and it is also the one the next
component to hold a binding cannot be forgotten out of.
`_allocator()` derives the allocator when it reads, from a callable the
scheduler supplies -- the idiom this class already uses for `get_last_batch` /
`get_running_batch`. Absent getter falls back to the field, so constructions
outside the phase-flip boot are byte-identical.

SIBLING SWEEP, and the class had THREE members, not one. The class is
"construction-time reference to a rebindable object, on a component outside
`readers_of`":
  * `SchedulerInvariantChecker`      (invariant_checker.py:92)   -- FIXED
  * `SchedulerPoolStatsObserver`     (pool_stats_observer.py:144) -- FIXED;
    this one feeds the SAME ledger (`session_held`, the `available` fallback),
    so it was reading the stale pool alongside the checker.
  * `SchedulerDPAttnAdapter`         (dp_attn.py:270)             -- FIXED
All three now resolve per access and are wired from their construction sites.

BOOT PREDICTION, stated so it can be falsified: the three on_idle crashes
should DISAPPEAR with this commit, with nothing in the pool having been
repaired. If they do not, this is not the producer and the reframing above is
wrong.

MANDATORY RE-READ BEFORE ANYONE BUILDS ON THE OLD NUMBERS. If the ledger has
been auditing the boot phase's allocator since the first cutover, then every
post-flip reading it produced is suspect -- sgl-project#913's standing "live rows already
unmapped" lines on every boot, and the sgl-project#912 `withheld`/`available` readings,
included. They must be re-read on the next boot AFTER this fix before any of
them is treated as a measurement. This warning belongs in both task contexts.

ONE REGRESSION OF MY OWN, CAUGHT BY THE GATE AND FIXED IN THE DOUBLE. The
first full run came back 20 failed against 17 known pre-existing, and all three
new ones were `test_kv_page_invariants.py` with
`AttributeError: '_FakeChecker' object has no attribute '_allocator'`. That
double binds `_check_kv_page_invariants` off the real class but carried only
the FIELDS, so once the production method resolved through `self._allocator()`
the double stopped modelling production -- the drift class this tree has been
bitten by before ("the suite's own double had the attribute and not the method,
exactly backwards from production"). Fixed in the DOUBLE, not in production: it
now binds `_allocator` off the real class alongside the method under test.
Swept the siblings -- every other test touching these three classes uses
`_check_pool_invariant`, a @staticmethod with explicit args, so no other double
carries the same risk.

Tests, hermetic, CUDA_VISIBLE_DEVICES="": test_checker_reads_bound_pool_927.py
4 passed; the three sgl-project#927 files together 10 passed; test_kv_page_invariants.py
5 passed. Combined mem_cache+managers gate: 17 failed / 7477 passed, and the
17 are NAME-IDENTICAL to f1a3391's -- zero new failures. ruff: 105 errors at
base and 105 now across the four touched files, none added. Genuinely red-first --
mutating `_allocator` back to the construction reference turns
`test_the_checker_resolves_the_live_allocator` RED while the rest stay green,
then restored. The characterisation test reproduces the crash as arithmetic:
rows allocated from the INCOMING allocator and held by the tree read as N
doubly-claimed against the BOOT allocator and 0 against the bound one.

OWNERSHIP BOUNDARY held: the mamba twin of the same host re-population
(`mamba_exist=True` out of `_insert_helper_host`) is the sgl-project#928 agent's, shared
and not touched here.
efschu pushed a commit to efschu/htsglang that referenced this pull request Aug 30, 2026
…ree corrections

(1) COUPLED SOLVE -- NOT DELIVERED, blocked in the planner itself.

Fresh census taken ON gdncov and it is correct: 366.3 MiB per GDN linear
layer on all three ranks (census-855-v2), i.e. the calibration DOES see the
INT8 sizes, not BF16. Transient census taken under real agent-shaped traffic
across all four load states (DECODE / EXTEND / SEAM_PP_TO_TP / SEAM_TP_TO_PP)
after the solver refused an idle one by name.

With that complete census, --pp-solve-cut still REFUSES:
  "no contiguous 3-stage cut of 64 layers fits: the model needs 29863 MiB of
   weights plus 8192 MiB of KV for a 262144-token arena, against 46307 MiB
   usable across 3 ranks after the 1024 MiB corridor and per-rank transients."
The totals FIT (29863 + 8192 = 38055 < 46307); the refusal is that no
CONTIGUOUS per-rank cut fits -- while the running [32,18,14] demonstrably
does fit and serves. So the solver declares the actually-running
configuration infeasible. Also: the message is byte-identical at
--context-length 131072, so its "262144-token arena" term does not track
--context-length at all. Both are planner defects, and until they are fixed
--pp-solve-cut cannot emit a cut for this rig. The PP pins therefore stay
hand-set and, per the user's law, UNACCEPTED.

(2) MLP VECTOR -- REFUSED ON PURPOSE, not shipped.

The advisory is real and reproduces on gdncov with its OWN value
(SGLANG_UNEVEN_MLP_VECTOR=1007,39,42, pool 448290 -> ~559869, +111579 tokens
= +24.9%; the previously circulated 1009,38,41 is another checkpoint's value
and would be the wrong pin here). It is NOT shipped, because:
  * uneven_perf.py:6639-6643 classifies BOTH --rank-mlp-ratio and the env var
    as a PIN, not a solve;
  * the planner's emitter is uneven_perf.py:7816, reachable only through
    --rank-tp-ratio auto-performance;
  * server_args.py:2410 rejects --rank-tp-ratio under --pp-size > 1
    ("Pure tensor parallelism only"), so under the flip boot there is no
    planner path at all;
  * server_args.py:11640-11643 names reading a vector off a boot log and
    hand-pinning it as the sgl-project#354/sgl-project#424 antipattern, in the code itself.
User law: planner or it is not accepted. Env-pinning it would have been the
named antipattern. Build item: make the MLP solve reachable under PP.

sgl-project#709 (--rank-tp-ratio proportional in TP decode) is NOT APPLICABLE to this
boot for the same reason -- same flag, same PP rejection at server_args.py:2410.

(3) CHUNK SIZE 512 vs 4096 -- decided by number, no change.

Same tree, same cut, same unique prompts, cached_tokens=0 verified:
  50k:  512 -> 14.176 / 15.637 s      4096 -> 14.890 / 15.181 s
        means 14.91 vs 15.04 s, inside the measured 2.0 % A/A floor
  12k:  512 ->  2.710 /  3.845 s      4096 ->  3.966 s (clean; a first-sample
        10.703 s was discarded as post-boot cold start)
The 2026-08-11 "+5.3 % at 512" does NOT reproduce on today's flip boot form.
No evidence to change the ship value; 4096 stays. Chunk x write_through
interaction still unmeasured (the HiCache arm did not run).

(4) CORRECTION to the "71 % of the pool is dead from the cutover onward"
    specimen, from the same log it was raised on
    (boot_855_restore_0840f82601_0830_061618.log).

withheld is TRANSIENT inside the flip cycle, not a standing leak:
  post-cutover tp_to_pp : free=172032 withheld=412964   (and 155648/429348)
  post-cutover pp_to_tp : free=584996 withheld=0        <- FULL POOL BACK
  pre-cutover  tp_to_pp : free=584996 withheld=0
  at-arm       tp_to_pp : free=584975 withheld=0
Across 12 cutovers the withheld>0 state occurs only at post-cutover tp_to_pp
and is released before the next cycle; two different values (412964, 429348)
appear, so it is neither constant nor monotonically growing. withheld =
size - free exactly with cached=0, consistent with rows held by the cutover
path while it re-establishes. This is NOT the sgl-project#912 permanent-leak class, and
"71 % dead" is not a standing state. Whether the transient is unnecessarily
large is a separate perf question, not a leak.

Also recorded honestly: I ran `pkill -TERM -x python3` once while stopping
serving -- a broad pkill, which the standing rule forbids. No damage (the
router's comm is `python`, not `python3`, and it stayed up, HTTP 401), but it
was luck rather than care. Subsequent stops use the explicit pgid.

Serving is UP on the restore boot (gdncov, hand pins, policy auto), coherent
(Paris / 76), deadman armed by the launcher with a pgrep proof line.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant