Skip to content

[fix][fmoe] propagate flat_mode through fmoe_g1u1 dispatch - #4675

Merged
valarLip merged 2 commits into
mainfrom
alyr/fix-fmoe-g1u1-flat-mode
Aug 11, 2026
Merged

valarLip merged 2 commits into
mainfrom
alyr/fix-fmoe-g1u1-flat-mode

Conversation

@alexioslyrakis-amd

@alexioslyrakis-amd alexioslyrakis-amd commented Aug 11, 2026 •

Copy link
Copy Markdown
Contributor

Summary

  • Both Python and C++ read cfg.flat directly from the config, so the flat_mode parameter added in PR [PERF] gfx942 FMoE: a16 kernel dispatch for GLM-5 decode (T=16-256) #4558 was redundant
  • The AITER_CHECK(cfg.flat == flat_mode) caused a crash on gfx950 because fmoe_g1u1 was never wired up to pass flat_mode, defaulting to 0 while the config had flat=1
  • Removes flat_mode from get_heuristic_kernel, fmoe_g1u1, and fmoe_fp8_blockscale_g1u1 entirely
  • Hardcodes cfg.flat == 0 in the heuristic filter to exclude flat kernels (they are always dispatched by explicit kernel name from the tuned config CSV)

Test plan

  • Verify Qwen3.5-397B-A17B-MXFP4 accuracy test passes on gfx950 (reproduces the ATOM CI failure from ROCm/ATOM#1857)
  • Verify no regression on gfx942 FMoE dispatch (flat and non-flat paths)

PR #4558 added flat_mode to get_heuristic_kernel() and
fmoe_fp8_blockscale_g1u1 but missed fmoe_g1u1. On gfx950, when the
tuned config selects a flat kernel for the xbf16 MXFP4 path (e.g.
Qwen3.5-397B), fmoe_g1u1 calls get_heuristic_kernel with the default
flat_mode=0, triggering the AITER_CHECK(cfg.flat == flat_mode) assertion.

Add int flat_mode to the fmoe_g1u1 C++ entrypoint and Python stub, and
forward it from fused_moe_1stage.
@alexioslyrakis-amd
alexioslyrakis-amd requested a review from a team August 11, 2026 07:47
@github-actions

Copy link
Copy Markdown
Contributor

🏷️ CI Guide

Runs automatically on every PR:

  • ✅ Pre-checks (submodule verification, code formatting)
  • ✅ Aiter op tests (gfx942 + gfx950)
  • ✅ Triton tests on MI35X (only when aiter/ops/triton/** or related paths are changed)

Extended tests (opt-in via labels):

Label Tests
ci:triton-300x Run an additional Triton test job on MI300X in PRs; main branch always runs both MI35X and MI300X
ci:sglang SGLang integration tests: DeepSeek-R1-MXFP4 accuracy, Qwen 3.5 accuracy
ci:atom ATOM benchmark: DeepSeek-R1-0528, GPT-OSS-120B
ci:atom_full ATOM accuracy suite for PR and main models from ATOM models_accuracy.json
ci:vllm vLLM benchmark: GPT-OSS-120B, DeepSeek-R1-0528, Kimi-K2.5
ci:all All standard extended tests (excludes ci:atom_full)

Only add ci:atom_full for FlyDSL or Triton upgrades.
Add labels via the sidebar or gh pr edit 4675 --add-label <label>

Both Python and C++ read cfg.flat directly from the config, so passing
flat_mode as a parameter was redundant. The AITER_CHECK(cfg.flat ==
flat_mode) added in PR #4558 caused a crash on gfx950 because fmoe_g1u1
was never wired up to pass it.

Remove flat_mode from get_heuristic_kernel, fmoe_g1u1, and
fmoe_fp8_blockscale_g1u1. Hardcode cfg.flat == 0 in the heuristic
filter to exclude flat kernels (they are always dispatched by explicit
kernel name from the tuned config CSV).
@valarLip
valarLip merged commit 2e09ebf into main Aug 11, 2026
41 of 42 checks passed
@valarLip
valarLip deleted the alyr/fix-fmoe-g1u1-flat-mode branch August 11, 2026 13:37
@alexioslyrakis-amd alexioslyrakis-amd self-assigned this Oct 15, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants