Skip to content

[ROCm][AMD] MiniMax-M3 MXFP8 MI355x recipe update - #581

Merged
xianbaoqian merged 3 commits into
vllm-project:mainfrom
hongxiayang:update-mm3-mxfp8-amd
Jul 6, 2026
Merged

xianbaoqian merged 3 commits into
vllm-project:mainfrom
hongxiayang:update-mm3-mxfp8-amd

Conversation

@hongxiayang

@hongxiayang hongxiayang commented Jun 26, 2026 •

Copy link
Copy Markdown
Contributor

Update recipe for MiniMax-M3 MXFP8 on MI355x only

  • Added several environment variables

    • VLLM_ROCM_USE_AITER=1 — aiter master: grouped-topk shared-expert fusion (this image's fusion needs it) + fused RMSNorm/quant. Set 0 for --moe-backend triton (native mxfp8) — it conflicts there.
    • VLLM_ROCM_USE_AITER_FUSION_SHARED_EXPERTS=1 — explicit shared-expert fusion enable.
    • VLLM_ROCM_QUICK_REDUCE_QUANTIZATION=INT6 — INT6 quantized all-reduce (+6–14% at batch). gsm8k 0.9143 (= baseline); NONE to disable.

VLLM_USE_BREAKABLE_CUDAGRAPH=0 — still needed.

  • Depending on the following vLLM PRs to be merged.
  1. [ROCm] [MoE] [Perf] Shared-expert fusion for bias-routed MoE; enable on MiniMax-M3 mxfp8 model vllm#46545 fused experts for mxfp8 model
  2. [ROCm][Perf] Use flydsl moe with Minimax-M3 mxfp8 weights on gfx950 and implemented moe-backend selection vllm#46184 aiter flydsl moe backend enablement
  • updated to use nightly image after the above PRs are merged ..

Only Merge this PR after that.

image

Signed-off-by: Hongxia Yang <hongxia.yang@amd.com>
@vercel

vercel Bot commented Jun 26, 2026 •

Copy link
Copy Markdown
Contributor

The latest updates on your projects. Learn more about Vercel for GitHub.

Project Deployment Actions Updated (UTC)
vllm-recipes Ready Ready Preview, Comment Jun 29, 2026 5:15pm

Request Review

@gemini-code-assist gemini-code-assist Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code Review

This pull request introduces hardware overrides for the AMD MI355X GPU under the mxfp8 variant of the MiniMax-M3 model configuration, specifying the nightly ROCm Docker image and enabling AITER kernels, fused shared-experts MoE, and INT6 QuickReduce via environment variables. The documentation guide is also updated with manual launch instructions and an example command. There are no review comments, so I have no feedback to provide.

@functionstackx functionstackx left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

i am happy to help verify this recipe once there is an upstream image that contains the AITER M3 enablement PRs

Comment thread models/MiniMaxAI/MiniMax-M3.yaml Outdated
# comfortably, ~4 GPUs for weights alone on Blackwell (B200/B300) or AMD MI350X/MI355X (gfx950)).
vram_minimum_gb: 513
description: "NVIDIA-quantized MXFP8 weights — Blackwell (B200/B300) for native MX tensor cores, and AMD CDNA4 (MI350X/MI355X, gfx950) for native MXFP8 Matrix Cores."
# MI355X-only acceleration for the MXFP8 path: AITER kernels, the fused

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Remove all these comments. They are all described by the attributes you defined in the yaml already.

Signed-off-by: Hongxia Yang <hongxia.yang@amd.com>
@hongxiayang

Copy link
Copy Markdown
Contributor Author

As of 6/28/2026, the PRs mentioned above have been merged, and the nightly image is available to support the recipe. Sanity checked on mi350x.

cc functionstackx, @chunfangamd @andyluo7

Signed-off-by: Hongxia Yang <hongxia.yang@amd.com>
@hongxiayang

Copy link
Copy Markdown
Contributor Author

cc @functionstackx @chunfangamd
Added an additional parameter

--linear-backend emulation

This will help on dense gemm.

functionstackx pushed a commit to SemiAnalysisAI/InferenceX that referenced this pull request Jun 30, 2026
Pin minimaxm3-fp8-mi355x-vllm{,-mtp} to nightly-4559c43a, which bakes in
fused shared-experts MoE (vllm-project/vllm#46545) and the AITER flydsl
MoE backend (#46184).
Align both bench scripts with vllm-project/recipes#581 by exporting
VLLM_ROCM_USE_AITER=1 and VLLM_ROCM_USE_AITER_FUSION_SHARED_EXPERTS=1
alongside the existing INT6 quick-reduce; no --moe-backend override, so
AITER is auto-selected.
cquil11 pushed a commit to SemiAnalysisAI/InferenceX that referenced this pull request Jun 30, 2026
…educe (#1946)

* [AMD] Update MiniMax-M3 FP8 MI355X vLLM image and enable INT6 quick-reduce

* [AMD] Update changelog

* [AMD] Retune MiniMax-M3 FP8 MI355X vLLM search space

* [AMD] Bump MiniMax-M3 FP8 MI355X image and enable AITER fused experts

Pin minimaxm3-fp8-mi355x-vllm{,-mtp} to nightly-4559c43a, which bakes in
fused shared-experts MoE (vllm-project/vllm#46545) and the AITER flydsl
MoE backend (#46184).
Align both bench scripts with vllm-project/recipes#581 by exporting
VLLM_ROCM_USE_AITER=1 and VLLM_ROCM_USE_AITER_FUSION_SHARED_EXPERTS=1
alongside the existing INT6 quick-reduce; no --moe-backend override, so
AITER is auto-selected.

* aiter master flag and ep

Signed-off-by: Hongxia Yang <hongxia.yang@amd.com>

* [AMD] Gate AITER master switch on EP for MiniMax-M3 MXFP8 recipes

Set VLLM_ROCM_USE_AITER on only for expert-parallel (EP/DP-attention)
runs, where the AITER fused MoE is the auto-selected backend. TP-only
runs leave it off and use the native MXFP8 path (the master switch
otherwise produces degenerate MiniMax-M3 output).

Keep VLLM_ROCM_USE_AITER_FUSION_SHARED_EXPERTS=1 unconditional: the
router-append shared-experts fusion checks the env directly (independent
of the master switch) and self-disables under EP inside the model.

Co-authored-by: Claude <noreply@anthropic.com>

* [AMD] Drop conc=512 from FP8 MI355X vLLM MTP tp4/ep4 1k1k sweep

The minimaxm3-fp8-mi355x-vllm-mtp tp=4 ep=4 (dp-attn=false) 1k1k point was
failing at concurrency 512; lower conc-end 512 -> 256 so it sweeps 128/256.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

---------

Signed-off-by: Hongxia Yang <hongxia.yang@amd.com>
Co-authored-by: Hongxia Yang <hongxia.yang@amd.com>
Co-authored-by: Claude <noreply@anthropic.com>
Co-authored-by: functionstackx <47992694+functionstackx@users.noreply.github.com>
@hongxiayang

Copy link
Copy Markdown
Contributor Author

@xianbaoqian Can you help review and merge this PR?

@xianbaoqian

Copy link
Copy Markdown
Collaborator

LGTM. Thanks!

@xianbaoqian
xianbaoqian merged commit 1ad1537 into vllm-project:main Jul 6, 2026
4 checks passed

This branch was successfully deployed

1 active deployment
Preview — f6ff840f Deployed Jun 29, 2026 by vercel[bot]
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants