[a-canadasruiz]

Summary

Add a tuner for the DeepSeek-V4 FHMoE rows in aiter/configs/tuned_fhmoe.csv on gfx950: csrc/ck_gemm_moe_2stages_codegen/fhmoe_tune.py.
Those rows already serve through fused_moe, but nothing in the repo could produce or re-check them.

Motivation

DSv4 FHMoE runs routed MXFP4 experts plus one FP8 shared expert in a single launch (E=385, topk 7, shared expert 384, GateMode.INTERLEAVE). There was no tuner for its rows in the repo: no way to retune a shape, add a new one, or check with --run_config that the shipped rows still match torch.

Since #6050, the same CSV also holds the HY4 rows, tuned by op_tests/tuners/tune_fhmoe.py. This tuner only touches rows with MXFP4 weights (DSv4) and skips the FP8-weight HY4 rows. The HY4 tuner can't cover DSv4 as it is: its shape, FP8 weights, kernel list and reference are fixed in the script, and it only retunes rows that already exist, on one GPU. This tuner reads the shape from untuned_fhmoe.csv, can add new shapes, runs on several GPUs through mp_tuner and has --run_config. One FHMoE tuner for both models would be a follow-up.

Changes

The commits follow this list.

1. Tuner. FhmoeTuner in its own module, fhmoe_tune.py, next to gemm_moe_tune.py, plus the catalogue untuned_fhmoe.csv (12 serving buckets, 1 to 2048 tokens).
   • It subclasses TunerCommon and imports a few helpers from gemm_moe_tune.py (weight quant, the torch stage references, cosine_diff_compare). gemm_moe_tune.py is not changed.
   • Each candidate is one FlyDSL stage1 x stage2 pair with a block_m. Stage 1 comes in fused and non-fused quant (_fp8 suffix). That is about 5632 pairs per token. The kernel list follows the row's activation and weight dtypes, as in FmoeTuner.
   • Each pair is timed through _fused_moe_impl with a one-row temp CSV. The public fused_moe() cannot take that file, because DSv4 serving needs every padded-M tier in the config.
   • Pairs that miss the torch reference are dropped. The fastest of the rest is written under get_padded_M(token).
   • Only DSv4 rows are rewritten. Tokens that are not serving buckets are rejected. Rows pinned to another gfx/cu_num are skipped.
   • Weights are quantized on the CPU, so the GPU worker only runs fused_moe.
2. CPU tests for FhmoeTuner (test_fhmoe_dsv4_tuner.py), run in the level 0+1 job.
3. Tuning Tests coverage as family fhmoe_dsv4: CSV validation, --run_config, a small tune pipeline, and a manual replay test.
4. op_tune.sh: a new fhmoe job, skipped on anything but gfx950. It does not pass --all, so it is a no-op until a new shape is added to the catalogue.
5. Reference that quantizes like the kernel. Rows with MXFP8 activations (DSv4, INTERLEAVE) quantize them before stage 1 and between the stages. The reference now does the same, in two variants (stage-1 result rounded to bf16 first, or quantized from fp32 like a fused-quant stage 1), and the candidate is judged against the closer one. --run_config tries the fp32 variant only when the first one misses. Serving code (fused_moe, aiter/fhmoe.py) is not changed. The shipped tuned_fhmoe.csv is not changed.

Testing

• Unit tests added or updated
• Performance benchmarks run
• Tested on MI250X
• Tested on MI300X
• Tested on MI355X (gfx950)

PR Self-Review

Reviewed revision

8b80abe

Behavior and intent

Adds a tuner path only: a new script, fhmoe_tune.py. Before this PR the DSv4 rows of tuned_fhmoe.csv could not be retuned or checked with --run_config. gemm_moe_tune.py is not changed. Serving and the shipped CSV are unchanged.

Validation and unverified scope

On MI355X (gfx950), at this revision, all passed:

• test_fhmoe_dsv4_tuner, test_csv_validation (CPU, 51 tests)
• test_fhmoe_replay (3 tests)
• test_tune_pipeline ... test_fhmoe_dsv4_mp1
• test_run_config ... test_fhmoe_dsv4: 12 shipped DSv4 rows, no ERROR or MISMATCH

Not verified:

• Rows with fp4 activations still use an unquantized reference. The tuner warns. No such rows ship today.
• Full sweeps only for token 1.
• tuning-tests.yaml runs on schedule or workflow_dispatch only, so this PR's CI does not run it.

PR-specific question

No new permissions, secrets or triggers. op_tune.sh gets one job entry and a gfx950 check. tuning-tests.yaml adds one module to the level 0+1 command. Both workflows run only on schedule or workflow_dispatch, so fork PR code or text cannot reach them.

Changed op tests

• test_fhmoe_dsv4_tuner.py: CPU tests for FhmoeTuner, including that a rewrite keeps the HY4 rows. Nothing else covers it.
• test_fhmoe_replay.py: replays the tuner output through the public fused_moe. Manual, gfx950.
• test_csv_validation.py: no duplicate shapes, catalogue tokens are serving buckets.
• test_run_config.py: adds the fhmoe_dsv4 family.
• test_tune_pipeline.py: small --mp 1 tune for fhmoe_dsv4.
• README.md: documents the new tests and family.
• test_fhmoe.py: __main__ now runs pytest.main. Before, it exited 0 without running any test.

[github-actions]

🏷️ CI Guide

Runs automatically on every PR:

• ✅ Pre-checks (submodule verification, code formatting)
• ✅ Aiter op tests (gfx942 + gfx950)
• ✅ Triton tests on MI35X (only when aiter/ops/triton/** or related paths are changed)

Extended tests (opt-in via labels):

Label	Tests
ci:gfx1250-ffm-triton	Run the five-shard gfx1250 FFM Triton test suite
ci:triton-300x	Run an additional Triton test job on MI300X in PRs; main branch always runs both MI35X and MI300X
multigpu	Aiter multi-GPU tests on the 8-GPU runner
ci:sglang	SGLang integration tests: DeepSeek-R1-MXFP4 accuracy, Qwen 3.5 accuracy
ci:atom	ATOM benchmark: DeepSeek-R1-0528, GPT-OSS-120B
ci:atom_full	ATOM accuracy suite for PR and main models from ATOM models_accuracy.json
ci:vllm	vLLM benchmark: GPT-OSS-120B, DeepSeek-R1-0528, Kimi-K2.5
ci:all	All standard extended tests (excludes ci:atom_full)

Only add ci:atom_full for FlyDSL or Triton upgrades.
Add labels via the sidebar or gh pr edit 5751 --add-label <label>

PR title tags & labels:
Component tags ([Triton/Gluon], [HIP], [CK], [ASM], ...) are added to the PR title and as PR labels automatically from the changed files and re-synced on every push — change-type tags like [fix]/[Perf], op tags like [MLA], and human labels (ci:*) are left untouched. Add the no-auto-title label to opt this PR out.

[aiter-bot]

NOT READY FOR HUMAN REVIEW

PR Self-Review

Revision 8b80abecd1dd · mode shadow

Author self-review accepted. Readiness report:

Author answers

• PASS — current-SHA answers are present and structurally specific

Show the author's self-review answers

Behavior and intent

Adds a tuner path only: a new script, `fhmoe_tune.py`. Before this PR the DSv4 rows of `tuned_fhmoe.csv` could not be retuned or checked with `--run_config`. `gemm_moe_tune.py` is not changed. Serving and the shipped CSV are unchanged.

Validation and unverified scope

On MI355X (gfx950), at this revision, all passed:

- `test_fhmoe_dsv4_tuner`, `test_csv_validation` (CPU, 51 tests)
- `test_fhmoe_replay` (3 tests)
- `test_tune_pipeline ... test_fhmoe_dsv4_mp1`
- `test_run_config ... test_fhmoe_dsv4`: 12 shipped DSv4 rows, no ERROR or MISMATCH

Not verified:

- Rows with fp4 activations still use an unquantized reference. The tuner warns. No such rows ship today.
- Full sweeps only for token 1.
- `tuning-tests.yaml` runs on schedule or `workflow_dispatch` only, so this PR's CI does not run it.

PR-specific question

No new permissions, secrets or triggers. `op_tune.sh` gets one job entry and a gfx950 check. `tuning-tests.yaml` adds one module to the level 0+1 command. Both workflows run only on `schedule` or `workflow_dispatch`, so fork PR code or text cannot reach them.

Changed op tests

- `test_fhmoe_dsv4_tuner.py`: CPU tests for `FhmoeTuner`, including that a rewrite keeps the HY4 rows. Nothing else covers it.
- `test_fhmoe_replay.py`: replays the tuner output through the public `fused_moe`. Manual, gfx950.
- `test_csv_validation.py`: no duplicate shapes, catalogue tokens are serving buckets.
- `test_run_config.py`: adds the `fhmoe_dsv4` family.
- `test_tune_pipeline.py`: small `--mp 1` tune for `fhmoe_dsv4`.
- `README.md`: documents the new tests and family.
- `test_fhmoe.py`: `__main__` now runs `pytest.main`. Before, it exited 0 without running any test.

PR state

• WAIT — pull request is still a draft

Copilot findings

• PASS — no active Copilot finding lacks a substantive author reply
• ADVISORY — Copilot did not review the current head SHA

Code and op-test hygiene

• PASS — no committed artifact or conflict marker detected

Required CI

• PASS — Black: passed
• PASS — Ruff: passed
• PASS — Repository dependency: passed
• PASS — Aiter Test Gate: passed

_{Shadow mode reports what would block but keeps the status successful. Set AITER_PR_SELF_REVIEW_MODE=enforce only after reviewing real false positives.}

[fxmarty-amd]

RElated: https://github.com/ROCm/aiter/blob/main/op_tests/tuners/tune_fhmoe.py ?