[karverma-amd]

[DSV4] E=49 shared-expert-fusion fp8/fp4 fused-MoE tuned configs (gfx950)

Draft — attaching GSM8K validation below before marking ready.

What

Adds 14 tuned FlyDSL rows to aiter/configs/model_configs/dsv4_fp8fp4_tuned_fmoe.csv
for the E=49 shape (48 routed + 1 shared expert per rank), which is what
DeepSeek-V4-Pro produces under EP8 with shared-expert fusion (w1.shape[0] == 49).

Why

The runtime fused-MoE config lookup keys on expert (the materialized expert-weight
count), so shared-fusion EP8 queries expert=49. No dsv4_fp8fp4 rows existed for
E=49, so the routed a8(fp8)×w4(fp4) GEMM fell back to the heuristic FlyDSL tile picker
([aiter][fused_moe] no tuned FlyDSL config).

Coverage

Full padded-M ladder — decode conc 4–512 and prefill to 32k:
M ∈ {4,8,16,32,64,128,256,512,1024,2048,4096,8192,16384,32768} at
model_dim=7168, inter_dim=3072, topk=6, per_1x32, a=fp8_e4m3fn, w=fp4_e2m1fn_x2.
All 14 shapes verified 0.0% err on both stages (flydsl_moe1/2_..._gui_fp8).

Tuner vs heuristic tile

M	speedup
512	+26.2%
1024	+36.5%
2048	+8.0%
16384	+8.5%
32768	+6.7%
others	~tie

Validation

GSM8K (1319 questions, 5-shot, temperature 0, parallel 1319) — results below.

[github-actions]

🏷️ CI Guide

Runs automatically on every PR:

• ✅ Pre-checks (submodule verification, code formatting)
• ✅ Aiter op tests (gfx942 + gfx950)
• ✅ Triton tests on MI35X (only when aiter/ops/triton/** or related paths are changed)

Extended tests (opt-in via labels):

Label	Tests
ci:gfx1250-ffm-triton	Run the five-shard gfx1250 FFM Triton test suite
ci:triton-300x	Run an additional Triton test job on MI300X in PRs; main branch always runs both MI35X and MI300X
multigpu	Aiter multi-GPU tests on the 8-GPU runner
ci:sglang	SGLang integration tests: DeepSeek-R1-MXFP4 accuracy, Qwen 3.5 accuracy
ci:atom	ATOM benchmark: DeepSeek-R1-0528, GPT-OSS-120B
ci:atom_full	ATOM accuracy suite for PR and main models from ATOM models_accuracy.json
ci:vllm	vLLM benchmark: GPT-OSS-120B, DeepSeek-R1-0528, Kimi-K2.5
ci:all	All standard extended tests (excludes ci:atom_full)

Only add ci:atom_full for FlyDSL or Triton upgrades.
Add labels via the sidebar or gh pr edit 1 --add-label <label>

PR title tags & labels:
Component tags ([Triton/Gluon], [HIP], [CK], [ASM], ...) are added to the PR title and as PR labels automatically from the changed files and re-synced on every push — change-type tags like [fix]/[Perf], op tags like [MLA], and human labels (ci:*) are left untouched. Add the no-auto-title label to opt this PR out.

[karverma-amd]

GSM8K validation (new image, tuned rows exercised)

Image: rocm/mlperf-inference:v0.5.19-rocm720-mi35x-20260907 (aiter ROCm#4994 + these E=49 rows, AITER_CONFIG_FMOE wired)
HW: 8× MI355X (gfx950), DeepSeek-V4-Pro, EP8 + enforced shared-expert fusion (w1.shape[0]=49), MoRI a2a, dp-attention.

Tuned rows confirmed active

Server boot + full eval produced 0 no tuned FlyDSL config fallbacks. The runtime
lookup key resolves to (gfx950, 256, M, 7168, 3072, 49, 6, Silu, bf16, fp8, fp4, per_1x32, …)
— i.e. expert=49, topk=6 after the EP fake-slot strip — which is exactly what these rows key on.

GSM8K — 1319 questions, parallel 1319

5-shot, temperature 0, max-new-tokens 512, SIMULATE_ACC=0 (real decode).

metric	value
Accuracy	0.952
Invalid	0.000
Latency (full set)	65.22 s
Output throughput	1803.75 tok/s
dispatch / combine dtype	fp4 (mxfp4_blockwise) / fp8 (float8_blockwise)

Reference: bf16-dispatch baseline on this workload is 0.945; MXFP8 arm 0.951. This run is
0.952 with no invalid outputs, so the E=49 tuned kernels are accuracy-neutral (all rows
err1=err2=0.0%) while removing the heuristic-fallback tile selection on the routed a8/w4 MoE.