Repository navigation
[Triton/Gluon] [Config] [Perf] Tune gfx950 FP8 blockscale selections for DSv4 decode - #7614
XinyuJiangCMU wants to merge 2 commits into
Conversation
Add 57 tuned gfx950 FP8 blockscale entries for DeepSeek-V4 decode shapes to the bpreshuffle table. Retune the N=1536, K=4096 Triton buckets for M <= 128 to select faster small-M kernels. Signed-off-by: Xinyu Jiang <xinyuj2@andrew.cmu.edu> Co-authored-by: Zhiyao Jiang <jessicajiang324@gmail.com>
🏷️ CI GuideRuns automatically on every PR:
Extended tests (opt-in via labels):
One backend per PR: PR title tags & labels: |
WAITING ON AUTHORPR Self-ReviewRevision Waiting for the PR author. The readiness report will appear after the self-review answers are complete for this revision. Questions to answer in the PR description
Edit the Missing or invalid answers
Shadow mode reports what would block but keeps the status successful. Set |
Rescreen every candidate kernel for the 57 DeepSeek-V4 decode shapes on gfx950 with rotating weights. Nine N=8192, K=1024 shapes (M=72, 120 and 136-184) now select the ASM 32x128 or 48x128 blockscale kernels; the other 48 keep their kernels. The timing columns are refreshed from the new measurements. The Triton configuration is unchanged. Signed-off-by: Xinyu Jiang <xinyuj2@andrew.cmu.edu> Co-authored-by: Zhiyao Jiang <jessicajiang324@gmail.com>
Problem
DeepSeek-V4 FP8 decode runs more slowly on 256-CU gfx950 GPUs because small projection GEMMs fall back to slower default kernels.
Change
Add bpreshuffle selections for N1536/K4096 and N8192/K1024, and retune the N1536/K4096 Triton buckets through M128.
Validation
The measured kernel selections match the original outputs within a relative squared error of 1e-4 (FP64, over each whole output and each 512-element block). On MI350X, fixed-length batch-32 plain decode with the full 43-layer model and TP4/EP4 averaged 525.8→549.1 decode tok/s/GPU (+4.4%), with 13.97→13.33 ms GPU decode steps.
Co-author-with: @JessicaJiang-123