Skip to content

misc: update unit test config - #873

Merged
zhyncs merged 1 commit into
sgl-project:mainfrom
zhyncs:upd
Aug 1, 2024
Merged

zhyncs merged 1 commit into
sgl-project:mainfrom
zhyncs:upd

Conversation

@zhyncs

@zhyncs zhyncs commented Aug 1, 2024

Copy link
Copy Markdown
Contributor

Thank you for your contribution, we really appreciate it. The following instructions will help improve your pull request and make it easier to receive feedback. If there are any items you don't understand, don't worry. Just submit the pull request and ask the maintainers for help.

Motivation

Please explain the motivation behind this PR and the goal you aim to achieve with it.

Modification

Briefly describe the changes made in this PR.

Checklist

  1. Ensure pre-commit pre-commit run --all-files or other linting tools are used to fix potential lint issues.
  2. Confirm that modifications are covered by complete unit tests. If not, please add more unit tests for correctness.
  3. Modify documentation as needed, such as docstrings or example tutorials.

@zhyncs
zhyncs merged commit aba6f51 into sgl-project:main Aug 1, 2024
@zhyncs
zhyncs deleted the upd branch August 1, 2024 19:27
timethink pushed a commit to timethink/sglang that referenced this pull request Mar 9, 2025
McZyWu pushed a commit to McZyWu/sgl-sglang that referenced this pull request Jul 2, 2026
Update test_npu_matched_stop.py,test_npu_request_length_validation.py
efschu pushed a commit to efschu/htsglang that referenced this pull request Aug 26, 2026
…, so 411 declines have no verdict

Routing from the write-side agent (sgl-project#872) narrowed Posten 0 correctly: the store
is healthy, 24277 canonical pages live during the R7 window, and the historic
"store is empty" readings probed a path this boot does not use. So the 671
refused walks are a MATCH problem, not a content problem. The question that
follows immediately is the one nobody could answer: a walk that matches nothing
should fall through to an L3 prefetch, and prefetch was attempted on only 264
of 675 sampled walks.

THE OBVIOUS ANSWER IS WRONG, and the code says why. It is tempting to read the
411 as the same MambaComponent refusal declining twice -- one root, two numbers.
It is not:

  * The prefetch anchor IS mamba-gated, but by an aliasing the base contract
    does not describe. base_prefix_cache.py:201-210 separates `last_host_node`
    (the L3 prefetch anchor) from `best_match_node` (the validator-consensus L2
    anchor) -- two fields, two purposes. UnifiedRadixCache sets them equal under
    HiCache (unified_radix_cache.py:1443-1449). Deliberate and commented, but it
    contradicts the field's own documentation, and two fields that are
    documented different and implemented identical are a drift source (sgl-project#747's
    shape). Named, not changed here.
  * Even so, the refusal does NOT decline the prefetch. Total refusal leaves
    best_match_node at the root, and the caller admits the root on purpose:
    `locally_eligible = last_host_node.backuped or last_host_node is
    self.tree_cache.root_node` (scheduler.py:4933-4935). A cold request with no
    match is supposed to prefetch. The first term holds.
  * The second term (>= 256 new tokens) holds for agent prompts of thousands.

That leaves the rate limiter -- AND THERE THE EVIDENCE STOPS, which is the
actual finding. None of the three terms is counted or logged. The 411 declines
are silent. Nobody can attribute them from this boot, and no amount of
re-reading the log will change that.

    eligible = (locally_eligible
                and prefetch_length >= self.prefetch_threshold
                and not self.cache_controller.prefetch_rate_limited())

Three unrelated verdicts wearing one boolean, remedies in three different
files. The sgl-project#914 shape one module over: blame without a defect.

So this is an instrument and NOT a policy change. The three terms are the same
three, evaluated in the same order, with the same outcome; `eligible` is now
`reason is None`, one expression, so the predicate and its explanation cannot
drift apart the way sgl-project#747 records these very lineages drifting. The first
failing term is named rather than all of them -- a request can trip several,
and summing them would double-count exactly as
`refused_tokens_by_component` is documented to.

Not env-gated, unlike the sgl-project#904 match census. That one builds an object and
walks validators a second time, so it pays for itself only when armed. This is
one integer increment on a path that already builds a RadixKey and takes a host
lock -- and a counter you must remember to arm cannot answer "was it ever
tried", which is the whole question. `attempted` is counted alongside the
refusals so the denominator is local (sgl-project#873: a denominator reconstructed from
another log is how a narrowed candidate set reads as a decomposition), and an
unrecorded gate says "no observation" rather than a silent zero (sgl-project#829).

HYPOTHESIS, RECORDED AS ONE. prefetch_capacity_limit is `0.5 *
mem_pool_host.size` (cache_controller.py:729), and that size is not constant
across a phase flip: sgl-project#905 measured the two host tiers at 703472 rows (PP) and
30518 (TP), 23x apart, putting the TP-phase budget near 15259 tokens -- under
four prefetches of the 4096 this window actually completed, against 5
concurrent requests per flip. The order of magnitude fits 411/675, and I am
deliberately not fitting it exactly: the PP phase has ample budget, so the
number cannot come from there alone. If it holds, the same host-tier asymmetry
behind sgl-project#905's double-free is also starving the TP-phase prefetch budget -- one
defect, two effects, to be checked rather than assumed. The counter decides it
on the next boot instead of re-arguing it.

RED-FIRST, four mutants, each red, restored green (14 tests):
  revert to the shipped conjunction (no counter) ......... 4 failed
  eligible computed twice instead of derived from reason . 2 failed
  attempts not counted (denominator lost) ................ 2 failed
  no-observation reported as attempted=0 ................. 1 failed

ONE SELF-INFLICTED DEFECT, found and fixed in the same pass, recorded because
the form is evidently easy to commit. My first version of
test_the_rate_limit_check_is_still_called_at_most_once counted occurrences in
the function's RAW source and went red because a PRE-EXISTING comment at :2674
mentions `prefetch_rate_limited()`. That is the sgl-project#908 substring defect -- prose
used to reach a verdict about code -- in a test of mine, an hour after I
rejected the same form in someone else's patch. Comment lines are now stripped
before counting.

GATE test/registered/unit/mem_cache serial, CVD="": 3 failed, 1950 passed,
1658 skipped. Both axes agree (summary 3, ANSI-stripped extraction 3). Two are
test_acceptance_emitters_758::RefillTiming, baseline-proven pre-existing
earlier in this branch; the third is test_pool_invariant_double_owned_912, a
concurrently-developed test belonging to sgl-project#912 and untouched by this commit.
The sgl-project#904 census suite is green against this change (62 passed together with
the sgl-project#913/sgl-project#914/sgl-project#915 suites), which is the regression that matters: note_refused
gained an optional third argument and its existing callers must not care.
YAMY1234 added a commit to YAMY1234/sglang that referenced this pull request Sep 27, 2026
…n the stock fp32 dense state path, sgl-project#873/sgl-project#878)

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
YAMY1234 added a commit to YAMY1234/sglang that referenced this pull request Sep 27, 2026
…efault bf16): reference-order fp32 HC mix, QSA K / V / raw index key, GDN projections and torch gating; backend fp32 conv on a gathered fp32 copy of the conv states and fla 0.5.2 chunk update (the reference's kernel) with per-chunk states for tracking; emit_kv fp32 branch; CPU tests bitwise vs verbatim reference, GPU check vs fla top-level call

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
YAMY1234 added a commit to YAMY1234/sglang that referenced this pull request Sep 30, 2026
…efault bf16): reference-order fp32 HC mix, QSA K / V / raw index key, GDN projections and torch gating; backend fp32 conv on a gathered fp32 copy of the conv states and fla 0.5.2 chunk update (the reference's kernel) with per-chunk states for tracking; emit_kv fp32 branch; CPU tests bitwise vs verbatim reference, GPU check vs fla top-level call

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
(cherry picked from commit 3af0d41)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant