Repository navigation
Conversation
BBuf
marked this pull request as ready for review
August 23, 2026 13:25
BBuf
requested review from
Alisehen,
AniZpZ,
DarkSharpness,
Edwardf0t1,
FlamingoPg,
HaiShaw,
HydraQYH,
OrangeRedeng,
b8zhong,
celve,
ch-wan,
mmangkad and
yuan-luo
as code owners
August 23, 2026 13:25
Collaborator
Author
|
/tag-and-rerun-ci extra |
BBuf
force-pushed
the
agent/qwen38-nvfp4-skinny-gemm
branch
from
August 23, 2026 13:36
fe71f76 to
25685b1
Compare
BBuf
marked this pull request as draft
August 23, 2026 15:12
BBuf
marked this pull request as ready for review
August 23, 2026 16:52
Collaborator
Author
|
/rerun-failed-ci |
BBuf
marked this pull request as draft
August 25, 2026 06:57
4 of 5 tasks
Collaborator
Author
|
Closing this draft in favor of the stronger KDA-Pilot implementation from BBuf/KDA-Pilot#195. I will re-port that optimized SM120 NVFP4 GEMM into a dedicated python/sglang/kernels/kda_kernels package with explicit Humanize2 / Kernel Design Agents provenance, then revalidate DSpark end-to-end throughput and acceptance length on RTX PRO 6000 before opening the replacement PR. |
detain
added a commit
to detain/sglang
that referenced
this pull request
Nov 11, 2026
…+ re-plant era CLOSED at b86b7c9 (125/125 markers; grep census ZERO verified at this tree): header STACK epoch 321-2026-10-30 -> 329-2026-11-10 (supersession note kept) + +merge6-era-closeout-2026-11-10 trailer line + entries-count 1-329; tip/HEAD re-pinned (era CODE TIP b86b7c9 NAMED with the full wave chain, superseded values dea2641/e9f0ae1838/065fdcdf33/b7c656f69f/5cde7a5ede kept, docs tip recorded as this wave) + a prominent ** LIVE/BOOT STATE ** line (main checkout detached @ 2d87757 = the ONLY boot-validated state, 9b 16:58:52 warm all-needles-hit per prod-health-2026-11-10.txt; era branch CPU-green but NOT BOOTED -- switching prod = operator-chosen checkout + the entry-326 boot-watch). ENTRY 324: imports+bootfix trunk wave -- sgl-project#48781 require_attn_tp_gather->dp_attention relocation invisible to markers, base_runner deletion-only fix unmasking the 145E/32E/31E collection cascade (1dcc2d6); THEN the operator-live crash: maybe_init_router_registration read tokenizer_ipc_name before init_ipc_channels at OUR fork's entry-201 init order, call moved back to upstream's position, RED(2F)/GREEN(2P) test_tokenizer_manager_init_order.py, family scan clean + report-only carry (10a1c08). ENTRY 325: RE-PLANT ERA TABLE b1 12 / b3 18 / b4 39 / b5 17 / b2 23 / b6 16 = 125/125 per-wave dispositions (compressed from the commit bodies, all shas cited) + the NINE ERA CATCHES with FULL CREDIT: _rope_cache_key 10/11-arg universal get_rope crash (15 spec F were its fingerprints), PLE file-backend env registration gap (AttributeError every boot), COW-catcher doubled-staging-arm cascade bug, stat_loggers DI test restored from its corrupted merge hybrid, w13 refined-grid mis-key, the budget-contract decision (fork accounting RESTORED, evidence 106.9GB/100GB = the entry-317 leak shape; INDEXER collective order verified), verify-gate SUBSUME pick (sgl-project#44047 eats sgl-project#42779, both intents preserved), mm_process_config ambient-widths root-cause (validators now state the full 8-way layout), TLFIX twin re-pinned 282/285. ENTRY 326: BATTERY RE-PIN TABLE -- era pins managers 1307P/0F, spec 406P/0F, server_args 496P/0F, sampling 252P/0F, distributed 213P/0F, mem_cache 2965P/24F (the ACCEPTED device-only 24-node list over 8 files), cargo 966P/0F/1-ignored, five registration/wheel gates rc=0 + the PATH-TO-BOOTABLE statement: CPU-proven vs GPU-window debt -- first-boot needles incl. the sampling.cuh-vs-0.7.2 JIT arbitration (if it JIT-fails, wheel-trio-then-pin per S-FI073, never a lone re-pin), autotune caches SHOULD hit warm (no retune owed on 0.7.2), entry-322 verify-at-boot quartet, and the still-riding 321-era watches (sgl-project#45472/sgl-project#46643 TTFT+starvation, first-delta tool-stream latency). ENTRY 327: WHEELS & VENV RECORD -- pairing matrix (FI 0.7.2 HELD, sgl-kernel 0.4.10/0.4.10 era-consistent with the RETIRED-rule cross-ref, torch/cutlass-dsl MATCH), blake3 1.0.11 INSTALLED with ZERO-DRIFT PROOF (0 battery lines vs ~380; matrix re-read unchanged; negative-shim prefix-sim causality) + no-pip/uv-note (operator lane), the two tree-side wheel reverts with reachability + residual-symbol evidence, check_dockerfile_wheel_pins re-green, FI-0.7.3 adoption contract restated with post-era facts (upstream code now expects 0.7.3 in one reported JIT path; tree HELD 0.7.2; first boot arbitrates), dist-name/module-name note sglang_kernel/sgl_kernel. ENTRY 328: INCIDENT RECORD 2026-11-10 ~16:4x-16:58, honest narrative -- re-plant waves landed into the EDITABLE-IMPORT working tree while :30000/:30001 ran; 9b died ~pre-16:50 (cause UNVERIFIABLE from the user acct, /root logs unreadable; worker-respawn-importing-changed-code is the LEADING MECHANISM, stated AS SUCH); the operator restart hit the init-order crash = the era's flagship find; recovery = merge6-era-backup @ b069e6a + detached checkout 2d87757, 9b re-booted WARM 16:58:52 all needles hit, 9d NEVER dropped (Nov 3, HTTP 200); NEW ERA LAW: working tree = prod import path, integration by branch ref-move ONLY; b2's emergency-freeze WIP 7ccf76a (author 16:55:02) kept in range and completed as C1 -- provenance note. ENTRY 329: BACKFILL AUDIT RECORD -- window 2026-08-10..2026-10-27, universe 2714 = 1830 in-tree / 255 already-documented / 628 NEW REJECTS / 1 NEW-INTERESTING; method one-liners; the qwen4-main-squashed side-branch class discovery (REJECT-MERGED-TO-SIDE-BRANCH x44 -- future sweeps MUST filter by base; orchestrator's "206" NOT FOUND in the artifacts, 44 recorded as verified); sgl-project#36043 NVFP4 skinny GEMM SM120 full paragraph (closed-unmerged, absent-tree-verified, W4A4 M<=16 = our exact decode cell, BENCHMARK-BEFORE-PORT -> S-36043-BENCH filed); artifact paths. EVALUATED: the 671-line DOCS-BLOCK.txt inserted VERBATIM as its own dated block (165 dropped list-commas tidied at insertion, noted once at the block head; nothing else reflowed). DATED CORRECTIONS where the waves superseded 322/323-era operational text: the header PROD-BOOT-FORBIDDEN bullet and entry 323's WHEEL DRIFT + STATE lines get [11-10 ERA-CLOSE CORRECTION] annotations (classes closed, wheel landed, ban now purely the operator gate); marker-zone math re-verified against the wave ledgers (121-at-merge vs 125 recount stands correct; planted == consumed per zone). WATCH LIST: era-close block appended (sgl-project#36043 bench-gated; sgl-project#42790/sgl-project#40916 arrived-via-merge6 noted; sgl-project#43703 revert-trap cross-ref from entry 323; sgl-project#45520 STILL-OPEN re-confirmed at era close -- zero in-window refs, b5 verified the fail-safe layer intact; bootfix's report-only get_tokenizer_worker_class/entrypoints wiring carry). Docs-only: NO .py touched; check_registered_tests rc=0 re-run against this tree. NOTHING PUSHED; the live tree was NOT touched by this wave (ref-move integration follows with HEAD/status equality proofs).
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Motivation
Qwen3.8-27B NVFP4 decode on high-SM-count SM120 GPUs spends a significant amount of time in FlashInfer's general block-scaled GEMM. Its decode and DSpark verification shapes have only 1-16 rows, so the general kernel pays unnecessary tile padding and scheduling overhead.
Modifications
SGLANG_ENABLE_QWEN38_FP4_SKINNY_GEMM(off by default), with strict shape, dtype, stride, device, SM-count, and architecture guards.Accuracy Tests
Tested on an NVIDIA RTX PRO 6000 Blackwell Server Edition:
The FlashInfer comparison covers all three shapes at M=1, 9, and 16.
DSpark E2E validation used three fixed seeds. The accept lengths are unchanged after applying the safe down-projection routing:
Speed Tests and Profiling
RTX PRO 6000 kernel microbenchmarks against
flashinfer.mm_fp4(backend="auto"):Down M=9/16 numbers show the raw kernel result; those two paths are disabled by the safe default because of the acceptance sensitivity described above.
Qwen3.8-27B end-to-end output throughput:
Checklist
CI States
Latest PR Test (Base): ❌ Run #32962658503
Latest PR Test (Extra): ❌ Run #32962658497
Latest PR Test (AMD ROCm 7.2): ❌ Run #32962658549