Skip to content

chore: update flashinfer to v0.1.3 - #850

Merged
zhyncs merged 1 commit into
sgl-project:mainfrom
zhyncs:upd
Jul 31, 2024
Merged

zhyncs merged 1 commit into
sgl-project:mainfrom
zhyncs:upd

Conversation

@zhyncs

@zhyncs zhyncs commented Jul 31, 2024

Copy link
Copy Markdown
Contributor

Motivation

ref https://github.com/flashinfer-ai/flashinfer/releases/tag/v0.1.3
dep https://github.com/flashinfer-ai/flashinfer/actions/runs/10178830015

fix #771 (comment)

We need to wait until the official release of the 0.1.3 whl, and after verification is complete, then merge.

Modification

as titled

Checklist

  1. Ensure pre-commit pre-commit run --all-files or other linting tools are used to fix potential lint issues.
  2. Confirm that modifications are covered by complete unit tests. If not, please add more unit tests for correctness.
  3. Modify documentation as needed, such as docstrings or example tutorials.

@zhyncs
zhyncs marked this pull request as draft July 31, 2024 11:48

@yzh119 yzh119 left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

v0.1.3 wheels are ready.

@zhyncs

zhyncs commented Jul 31, 2024

Copy link
Copy Markdown
Contributor Author

Ok

@zhyncs zhyncs changed the title [DO NOT MERGE] chore: update flashinfer to v0.1.3 chore: update flashinfer to v0.1.3 Jul 31, 2024
@zhyncs
zhyncs marked this pull request as ready for review July 31, 2024 18:30
@zhyncs

zhyncs commented Jul 31, 2024

Copy link
Copy Markdown
Contributor Author

The PR E2E Test was successful. cc @merrymercy @Ying1123 @hnyls2002

May you help verify the latest code regarding the bug fix for sampling on CUDA 11.8? Thanks. @ispobock

@zhyncs
zhyncs merged commit bc3eaac into sgl-project:main Jul 31, 2024
@zhyncs
zhyncs deleted the upd branch July 31, 2024 18:37
@ispobock

Copy link
Copy Markdown
Collaborator

I tested flashinfer 0.1.3 on cu118. It works fine!

@zhyncs

zhyncs commented Aug 1, 2024

Copy link
Copy Markdown
Contributor Author

I tested flashinfer 0.1.3 on cu118. It works fine!

Hi @ispobock Thank you for confirming!

timethink pushed a commit to timethink/sglang that referenced this pull request Mar 9, 2025
JinyanYi pushed a commit to JinyanYi/sglang that referenced this pull request Jun 23, 2026
efschu pushed a commit to efschu/htsglang that referenced this pull request Aug 24, 2026
CAUGHT BY A COLLECTED-TEST COUNT, one test short, and it is worth recording
how: the merged tip collected 3766 tests where the branch collected 3767. The
missing one was this file's own TestTheClockIsPerEpisode -- staged before the
fix it guards was written, so a `git commit` of the index carried neither. The
merge pin therefore shipped the very defect this test was added to kill. A
zero-failure suite proved nothing about a test that was never collected.

THE DEFECT. `_presence_futile_since` survived an abandon, so the NEXT arm
measured its bound from the PREVIOUS episode and expired in its first round --
before the armed service turn had a single chance to clear anything. sgl-project#800
states the rule for its own escape clock: reset when the key empties, "rather
than inheriting a stranger's age". The clock is now keyed per (epoch, round)
and rebased whenever that key changes.

Mutation-proven in both directions: reverting the rebase fails the new test
with "0.0 not greater than or equal to 2.0 -- the second arm inherited the
first arm's futility clock", and restoring it returns the file to green.
efschu pushed a commit to efschu/htsglang that referenced this pull request Aug 24, 2026
…g/round8

A presence withhold whose every clause is beyond the armed service turn's
reach stops running the 60 s clock out, and the detector alarms that were
logged but grepped by nobody (sgl-project#850 plus sgl-project#838's two) reach monitoring.

Desk evidence on the merged branch: 3749 passed / 0 failed on the full
managers suite with nothing concurrent, against 3726 passed / 1 failed
(pp_admission_wraparound, pre-existing) on pristine integ/round8 -- delta
+22 is exactly this branch's new tests. Metal UNPROVEN; booked as W22.
efschu pushed a commit to efschu/htsglang that referenced this pull request Aug 24, 2026
…eg/round8

Supersedes pin 3566ffd, which shipped the stale-clock defect: the fix and
its test were written after the index was staged, so the first commit carried
neither. Caught by a collected-test count one short of the branch's -- a
zero-failure suite proves nothing about a test that was never collected.
efschu pushed a commit to efschu/htsglang that referenced this pull request Aug 24, 2026
… bigger reserve

Re-scope of the F1+F2 acceptance instrument, per ruling, plus the defect that
re-scope immediately found in F2 itself.

THE RE-SCOPE. `test_a_self_declared_under_backed_rank_MUST_NOT_veto` injected
`floor=131073, cap=126976` into `collective_kv_target` and demanded no veto. At
that layer the veto is CORRECT: the only way to remove it is to drop the rank's
floor from the group MAX, and that rank still applies the resulting proportion
to its own cap -- below its own live set, which is cudaErrorIllegalAddress and
kills every rank rather than raising (sgl-project#796). The assertion demanded a defect and
could never flip. A permanently-xfail test measures nothing.

It is now `test_the_reduction_MUST_veto_rather_than_cap_below_a_live_set`,
green permanently, asserting the veto positively so a future "optimisation"
that drops a defective rank's floor fails loudly. F1+F2's real property --
REACHABILITY, that floor > cap is TRANSIENT rather than permanent because the
pool can grow to its lawful floor -- is pinned at the layer that can deliver it,
red-both-ways against the SHIPPED sizer:
  - the pre-F2 shape CANNOT reach the floor (125052 and even a size-sized
    126976 fall short of the lawful 131073); if that stops holding, the
    specimen changed and the green direction proves nothing
  - the shipped sizer DOES reach it, strictly above the old value, across a
    range of sizes so a constant cannot fake it

AND IT EARNED ITSELF IMMEDIATELY. Red-both-ways exposed that F2 as committed in
e62b1fa STILL UNDER-RESERVED: the pool sizes its reservation before a
scheduler exists, so `_admission_reserve_rows(None)` fell back to 512 while
W22's live value was 4096 (derived from chunked-prefill-size 4096). The shipped
sizer returned 127489 against a lawful floor of 131073 -- rebuilding the same
wall one layer down, in the fix for that wall. A test that could only pass would
never have found it.

Fixed: the boot assumption takes `max(derived, CONSERVATIVE_ADMISSION_RESERVE_
ROWS = 16384)`. The derived reserve is not knowable at reservation time, and the
two errors are not comparable -- over-reserving buys a slightly larger VA span,
under-reserving is a permanent wall no runtime actuator can lift.

RULE RECORDED in docs/dev/NOTE_851_build_caveats.md, with all three instrument
corrections on this build: an acceptance test asserts a property THE FIX LAYER
CAN DELIVER; a test that injects state into a deeper layer tests that layer's
contract, not the fix. Two of the three were mine.

Also recorded there: the metal criterion "0 over-cap floor vetoes under load"
is NOT substituted by any of this. The unit property proves the pool CAN reach
its floor; only metal proves it DOES under real funding dynamics.

Tests: 603 passed / 0 failed / 0 xfailed across the sgl-project#851, sgl-project#850, funding,
relief, corridor, backing and exposure suites -- the branch now carries NO
xfails, both former ones resolved by re-scoping rather than by weakening.
ruff clean. Hermetic, CVD="", no boots.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[Bug] generate wrong sequences with higher temperature

3 participants