Repository navigation
Conversation
This was referenced Oct 17, 2026
This branch has not been deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Change
Enable the existing M-fast BF16 gate/up prefill specialization for
(M, N, K) = (4096, 12288, 1536), covering Gemma4's 15 narrow MLP layers. The shared binary-capability/dispatch predicate currently admits only the wide shape. Older callers and the M128 route retain their behavior.This is a two-file child of #558, tested at
891b15b5e143be53192523bc427594a5868ca088against parentc743cdf3d1b012296333da87701718f506eca9d7. The upstream base is main because the unmerged parent branch lives in the fork; the incremental change is the shape predicate and corresponding query/compiled-replay tests.No kernel body, tile, weight format, schema or API changes. The complete packaged BMG device section is byte-identical to the parent (28,317 bytes, SHA256
62b25e75dbd0d434fc63d4c177261e32049f38cef8095697540a7787bf86d7ae). All 22 shared MoE headers are unchanged. The host query and dispatcher use the same predicate.Validation
Performance
Intel Arc Pro B70, BF16, M4096/N12288/K1536, gradients disabled and inference mode disabled. Control is actual Torch BF16 F.linear plus the published standalone BF16 GELU lookup.
Each fresh process rotates 16 distinct physical 36 MiB weight allocations (576 MiB), retains 32 disjoint outputs across the two backends, validates poisoned/current-input replay for every bank, and measures six mirrored rounds. Both runs win all six rounds; source, package and validation hashes remain unchanged before/after.
These are operator microbenchmarks. SGLang integration/serving validation is separate; no serving or model-quality gain is claimed.
Evidence archives: numerical/source/dispatch
5192e75dbc5aae4ffa0ca2542b77e4bc364be80d93b2c3b4ac924bf03b3050a0(35 payloads), targeted testsbd4218217fa56336f9fd080c44339a480671cafa85f66537eaea86fee5a1588c(5 payloads), and both cold runs9cce3fa8993d5f1ccfbb25ffb63a63bacdf2e71f9b025ddbe62942ad49b49ff3(24 payloads). Payload hashes and raw-round means were independently verified.