Repository navigation
perf(minimax-m3): raise decode score memory throughput - #2982
Draft
XiaobingSuper wants to merge 5 commits into
Draft
XiaobingSuper wants to merge 5 commits into
XiaobingSuper wants to merge 5 commits into
Conversation
Co-authored-by: Cursor <cursoragent@cursor.com>
Contributor
🏷️ CI GuideRuns automatically on every eligible PR before approval:
Heavy model tests:
|
Contributor
Author
|
Corrected roofline follow-up using the supplied native HIP benchmark:
The PR keeps the robust bit-exact 14.8% score / 7.8% full-path gain, but does not claim roofline. |
Contributor
Author
|
Rejected low-N follow-ups (kept out of production):
gfx1250 BF16 WMMA requires a 16x16x32 tile, and manually replacing the exact reduction with low-N vector/native paths both lost performance and exactness. These prototypes remain benchmark-only and are not dispatched. The exact transposed-WMMA path in this PR remains the selected implementation. |
lirui927
pushed a commit
to lirui927/ATOM
that referenced
this pull request
Nov 9, 2026
…tion On gfx1250 the CP scorer takes Q as WMMA's M operand and the converted K as its N operand, with streaming K loads, as the gfx1250 TP decode scorer of ROCm#2982. The D reduction order and FP32 accumulation are unchanged, so the scores keep their bits. Other architectures keep the previous orientation. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-authored-by: Cursor <cursoragent@cursor.com>
XiaobingSuper
force-pushed
the
xiaobingsuper/m3-gfx1250-decode-score-warps
branch
from
November 10, 2026 08:22
0f84959 to
eaa36f8
Compare
lirui927
pushed a commit
to lirui927/ATOM
that referenced
this pull request
Nov 16, 2026
…tion On gfx1250 the CP scorer takes Q as WMMA's M operand and the converted K as its N operand, with streaming K loads, as the gfx1250 TP decode scorer of ROCm#2982. The D reduction order and FP32 accumulation are unchanged, so the scores keep their bits. Other architectures keep the previous orientation. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
lirui927
pushed a commit
to lirui927/ATOM
that referenced
this pull request
Nov 16, 2026
…tion On gfx1250 the CP scorer takes Q as WMMA's M operand and the converted K as its N operand, with streaming K loads, as the gfx1250 TP decode scorer of ROCm#2982. The D reduction order and FP32 accumulation are unchanged, so the scores keep their bits. Other architectures keep the previous orientation. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This branch has not been deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Depends on #2873 and remains draft until it lands; review commits
a1fbfaed1andeaa36f8in the meantime. Part of https://github.com/wuhuikx/LLM-Booster/issues/341.Summary
Two exact gfx1250 decode-score optimizations:
Dispatch depends only on arch and capture-time batch/context/query bounds, so it is graph-safe and model-name independent.
Roofline and performance
MI455X e15-05, graph replay, disjoint request pages and rotating working sets larger than the 192 MB L2.
At B4/Q4/193K, score-only same-run ABBA:
Final vs original: -20.5%. Final vs the first commit in this PR: -6.8%. Complete exact score + ordered top-k + sparse-table emit improves 26.676 -> 25.285 us (-5.2%) over the first commit.
Across the full 70-cell matrix (S=1,2,4,8,16,32,64,128; q1/q4; 32K/64K/128K/193K/256K), all 33 transposed-dispatch cells have no >2% regression, 30 improve by >2%, and their geometric-mean score reduction is 6.3% over the first commit. Fallback cells keep the first commit unchanged. Representative incremental score gains: 193K S32/q4 -8.8%, 256K S32/q4 -15.0%, 128K S16/q1 -11.1%.
The supplied native HIP non-temporal read benchmark reaches 20.0 TB/s at the exact 98.8 MB scorer traffic size (4.94 us), and 19.2-22.3 TB/s over 64 MB-3 GB. MI455X theoretical HBM peak is 23.3 TB/s. The final scorer is 38.7% of measured same-size read roofline, so meaningful low-N/LDS/pipeline headroom remains; this PR does not claim roofline.
Correctness
topk_idx,sparse_bt, andsparse_ctxbit-exactgit diff --checkclean