Skip to content

Accuracy Debug - #1063

Closed
hnyls2002 wants to merge 4 commits into
mainfrom
debug
Closed

hnyls2002 wants to merge 4 commits into
mainfrom
debug

Conversation

@hnyls2002

@hnyls2002 hnyls2002 commented Aug 12, 2024 •

Copy link
Copy Markdown
Collaborator

For debugging

With this shape

    batch_size = 5
    prefix_lens = torch.tensor([128, 156, 127, 176, 243], dtype=torch.int32, device="cuda")
    seq_lens = torch.tensor([370, 390, 410, 234, 279], dtype=torch.int32, device="cuda")

The diff is

==================== Diff With Triton ====================
Mean:  tensor(0.0004, device='cuda:0', dtype=torch.bfloat16)
Max:  tensor(0.0117, device='cuda:0', dtype=torch.bfloat16)
==================== Diff With Ragged ====================
Mean:  tensor(0.0001, device='cuda:0', dtype=torch.bfloat16)
Max:  tensor(0.0078, device='cuda:0', dtype=torch.bfloat16)

With this shape

    batch_size = 2
    prefix_lens = torch.tensor([261, 0], dtype=torch.int32, device="cuda")
    seq_lens = torch.tensor([367, 22], dtype=torch.int32, device="cuda")

The diff is

==================== Diff With Triton ====================
Mean:  tensor(0.0004, device='cuda:0', dtype=torch.bfloat16)
Max:  tensor(0.0156, device='cuda:0', dtype=torch.bfloat16)
==================== Diff With Ragged ====================
Mean:  tensor(0.0063, device='cuda:0', dtype=torch.bfloat16)
Max:  tensor(1.3203, device='cuda:0', dtype=torch.bfloat16)

@hnyls2002

Copy link
Copy Markdown
Collaborator Author

This issue has been solved by flashinfer-ai/flashinfer#440, thanks to @yzh119

@hnyls2002 hnyls2002 closed this Aug 13, 2024
@zhyncs
zhyncs deleted the debug branch August 13, 2024 11:45
@hnyls2002
hnyls2002 restored the debug branch August 14, 2024 09:41
@hnyls2002
hnyls2002 deleted the debug branch August 14, 2024 10:03
efschu pushed a commit to efschu/htsglang that referenced this pull request Aug 31, 2026
…nswer their own question

sgl-project#1063 -- THE FENCE DECIDER, three states printed side by side rather than one
saldo. Both candidate roots for the 92 % empty read-through (714/771 on boot 29)
were refuted at the desk, so the next observation must separate what is left
instead of summing it:

  * present_and_readable   -- persisted at the fence and still readable now
  * complete_marker_absent -- a .part706/.slots706 with no .bin: the blob whose
    writer group never covered the last byte. `_set_canonical_slice` says it
    itself -- "a writer acting alone leaves nothing readable behind" -- and a
    cutover tearing an in-flight assembly apart is exactly that shape, forever
    invisible to every reader.
  * evicted_since_flip     -- was readable at the fence, gone at re-admission.

TWO OBSERVATION POINTS, NOT ONE, and that is the whole design: "gone now" and
"never there" are the SAME stat() and have opposite fixes. The fence snapshot is
what makes the eviction countable at all. Recorded at the fence before staging
changes anything; re-read on the sgl-project#1060 cadence, which is the only place on the
re-admission path holding a tree_cache.

sgl-project#1062 -- THE KEY TRACE MOVES TO THE FUNNEL THAT IS ACTUALLY SYMMETRIC. The #969G
probe sat on `_get_component_key`, whose comment claimed "The ONE funnel every
store key goes through, read and write alike (6 call sites)". Statically there
are FIVE sites and not one of them writes -- `_get_component_path`,
`_collect_existing_component_keys` (x2), `batch_exists_v2` (x2), all lookup or
existence. Confirmed on metal: across boot 30 the only callers it ever observed
were `_collect_existing_component_keys`, `has_component` and a `<genexpr>`.

So the instrument built to compare the re-admission's key against the retention's
key COULD NEVER SEE A WRITE KEY. Its never-observed difference was a property of
its placement. Indicator law, verbatim -- and the lying comment is replaced
rather than amended, because it is the next reader's trap.

The probe now lives on `_log_key`, which `_write_page` AND `_read_page` both
call. The rate limit is gone: the old `_n <= 60 or _n % 1024` printed 602 lines
against a population of 235520 on boot 30 (0.3 %), and an absence in that is
worth nothing. What remains is an explicit LINE CAP with its own printed
suppressed-count, so the trace stays bounded and always names its denominator.

MATCHED CHECKS for both failure classes -- a census that collapses its states,
and an instrument that never observes the direction it compares. sgl-project#1063 is driven
through a real transition (a .bin removed between the two observation points must
score evicted_since_flip=1 and NOT still_absent), and the torn-assembly case is
asserted as its own state. sgl-project#1062 is asserted at the placement level, because
placement is precisely what a behavioural call on one direction cannot show.
7/7 green. py_compile green, ruff clean on the added lines.

MEASURE ONLY. Nothing is gated, refused or repaired by either.

Order: coordinator (0a)+(0b). Evidence: boots 29/30, specimen
/spinning/evidence-665-f1/specimen_1060cens_boot30_uniformwidth_crash/
efschu pushed a commit to efschu/htsglang that referenced this pull request Aug 31, 2026
…ompensation layer, and its bugs were not the finding

Coordinator order (2), after boots 30 and 31 killed the same mechanism twice at
the same site with the same numbers (`told prefix 12288 exceeds this rank's
pinned span 0`). Upstream-minimal law: a defect found in a compensation layer is
a DELETION candidate, never a repair order for that layer. Two repairs were
already spent on it -- sgl-project#1061's epoch gate and #1061b's retract clear -- and the
second only existed because the first was built on a clock whose tick was never
checked against the act it fenced.

DELETED: `pp_uniform_width.py` whole (`uniform_pass_geometry`,
`UniformWidthPromiseBroken`, `report_local_coverage`, `min_told`, `PassGeometry`,
`epoch_admits_row`, `set_epoch_source`, `current_epoch`);
`Req.apply_uniform_pass_geometry_1059` and its call site; the
`_1059_told_prefix/_extend/_epoch` stamps; `PPAdmissionEntry.decided_epoch` and
its wire column; the publication-time epoch stamp; the epoch-source
registration; the told clearing at the retract chokepoint; the
`SGLANG_PP_UNIFORM_WIDTH` gate; three test files.

WHAT DELIBERATELY STAYED, and each for a named reason:

* THE EVICTION FLOOR in `pp_stamp_observed_coverage`. `report_local_coverage`
  was `max(0, int(x))`, but the same line ALSO raises `cache_protected_len`,
  which `mem_cache/common.py:82` honours as an eviction floor. Deleting a
  promise must not silently lower a protection, so the arithmetic is inlined and
  the floor is byte-identical. Only the promise semantics die.
* THE CARRIER COUNTER, renamed to what survives. The #1060b ledger counted the
  told row at the apply site; that apply is gone, but the question underneath it
  is not. Boot 29 shipped a carrier whose send gate had been dead since sgl-project#1046
  and reported `sgl-project#631=0` while nothing executed -- it cost a window and a census
  to notice. `sgl-project#1064 CARRIER CENSUS` counts `rows_received` on every rank, so
  that regression can never be silent again. PP0 never receives, so only a
  downstream zero is a finding, and the line says so.
* `sgl-project#1060`/`sgl-project#1063` untouched: they measure, they never gated anything.

WHAT THIS RETURNS TO: the pre-sgl-project#1059 behaviour, in which every rank derives its
geometry from its own match. That is not a guess about safety -- boot 29 ran it
for its ENTIRE life (`evaluated=87 absent=87`, the row never arrived) across 51
flips with no `sgl-project#631`, no divergence death and no crash. The two boots that DID
die at this seam are the two that ran the promise layer.

c1 (downstream takes the geometry as a VERDICT) is NOT in this commit, and the
reason is a desk finding that inverts its safety argument: `truncate_prefix_to`
is MIN, NEVER ASSIGN (schedule_batch.py, sgl-project#930/sgl-project#958) -- so "verbatim" cannot be
expressed through the one helper both admission sites use, and a rank holding
less than it was told silently keeps its own value, which is the divergence the
verbatim form exists to remove. Reported to the coordinator with the file:line
rather than worked around.

38/38 green across sgl-project#791/sgl-project#1060/sgl-project#1063/sgl-project#677, codec round-trip verified after the
column removal, import smoke of every touched module, ruff clean, py_compile
green.

Order: coordinator (2). Evidence: boots 30/31, specimens
specimen_1060cens_boot30_uniformwidth_crash/ and
specimen_1061b_boot32_int8_triton_wedge/
efschu pushed a commit to efschu/htsglang that referenced this pull request Sep 1, 2026
…hot: an instrument may never gate

Boot 1067park, second fence (tp_to_pp, 06:49:55Z): _1063_record_fence walks
every page stem of every eligible node through 2-4 os.path.exists() against
a 1.69-million-file store. The first fence ran warm (221862 stems, ~3 s);
the second hit the re-prefill's fresh stems cold and never returned -- PP0's
last line for 25 minutes was the FENCE-NODES header, no abandon could run
(PP0 is the sole timeout carrier), and the deadman killed the boot 07:15:37.

Fix: hard 2.0 s budget; an expired scan reports SCAN CAPPED (counts declared
a sample, denominator law) and bumps fence_scan_capped. The two-point
consumer (_1063_probe_since_fence) reads only the 4096-capped _1063_AT_FENCE
dict and loses no coverage it ever had.

Trap recorded in a comment: 'now' is a PARAMETER of the fence function, not
a module name -- a bare now() here would NameError into the outer except and
silently delete the snapshot (sgl-project#872 silent-zero shape). time.monotonic() used
explicitly.

Desk-proven by execution, both arms: uncapped emits normally; a slow clock
trips SCAN CAPPED.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant