Repository navigation
[ROCm][AMD] MiniMax-M3 MXFP8 MI355x recipe update - #581
Conversation
Signed-off-by: Hongxia Yang <hongxia.yang@amd.com>
|
The latest updates on your projects. Learn more about Vercel for GitHub.
|
There was a problem hiding this comment.
Code Review
This pull request introduces hardware overrides for the AMD MI355X GPU under the mxfp8 variant of the MiniMax-M3 model configuration, specifying the nightly ROCm Docker image and enabling AITER kernels, fused shared-experts MoE, and INT6 QuickReduce via environment variables. The documentation guide is also updated with manual launch instructions and an example command. There are no review comments, so I have no feedback to provide.
functionstackx
left a comment
There was a problem hiding this comment.
i am happy to help verify this recipe once there is an upstream image that contains the AITER M3 enablement PRs
| # comfortably, ~4 GPUs for weights alone on Blackwell (B200/B300) or AMD MI350X/MI355X (gfx950)). | ||
| vram_minimum_gb: 513 | ||
| description: "NVIDIA-quantized MXFP8 weights — Blackwell (B200/B300) for native MX tensor cores, and AMD CDNA4 (MI350X/MI355X, gfx950) for native MXFP8 Matrix Cores." | ||
| # MI355X-only acceleration for the MXFP8 path: AITER kernels, the fused |
There was a problem hiding this comment.
Remove all these comments. They are all described by the attributes you defined in the yaml already.
Signed-off-by: Hongxia Yang <hongxia.yang@amd.com>
|
As of 6/28/2026, the PRs mentioned above have been merged, and the nightly image is available to support the recipe. Sanity checked on mi350x. |
Signed-off-by: Hongxia Yang <hongxia.yang@amd.com>
|
cc @functionstackx @chunfangamd This will help on dense gemm. |
Pin minimaxm3-fp8-mi355x-vllm{,-mtp} to nightly-4559c43a, which bakes in
fused shared-experts MoE (vllm-project/vllm#46545) and the AITER flydsl
MoE backend (#46184).
Align both bench scripts with vllm-project/recipes#581 by exporting
VLLM_ROCM_USE_AITER=1 and VLLM_ROCM_USE_AITER_FUSION_SHARED_EXPERTS=1
alongside the existing INT6 quick-reduce; no --moe-backend override, so
AITER is auto-selected.
…educe (#1946) * [AMD] Update MiniMax-M3 FP8 MI355X vLLM image and enable INT6 quick-reduce * [AMD] Update changelog * [AMD] Retune MiniMax-M3 FP8 MI355X vLLM search space * [AMD] Bump MiniMax-M3 FP8 MI355X image and enable AITER fused experts Pin minimaxm3-fp8-mi355x-vllm{,-mtp} to nightly-4559c43a, which bakes in fused shared-experts MoE (vllm-project/vllm#46545) and the AITER flydsl MoE backend (#46184). Align both bench scripts with vllm-project/recipes#581 by exporting VLLM_ROCM_USE_AITER=1 and VLLM_ROCM_USE_AITER_FUSION_SHARED_EXPERTS=1 alongside the existing INT6 quick-reduce; no --moe-backend override, so AITER is auto-selected. * aiter master flag and ep Signed-off-by: Hongxia Yang <hongxia.yang@amd.com> * [AMD] Gate AITER master switch on EP for MiniMax-M3 MXFP8 recipes Set VLLM_ROCM_USE_AITER on only for expert-parallel (EP/DP-attention) runs, where the AITER fused MoE is the auto-selected backend. TP-only runs leave it off and use the native MXFP8 path (the master switch otherwise produces degenerate MiniMax-M3 output). Keep VLLM_ROCM_USE_AITER_FUSION_SHARED_EXPERTS=1 unconditional: the router-append shared-experts fusion checks the env directly (independent of the master switch) and self-disables under EP inside the model. Co-authored-by: Claude <noreply@anthropic.com> * [AMD] Drop conc=512 from FP8 MI355X vLLM MTP tp4/ep4 1k1k sweep The minimaxm3-fp8-mi355x-vllm-mtp tp=4 ep=4 (dp-attn=false) 1k1k point was failing at concurrency 512; lower conc-end 512 -> 256 so it sweeps 128/256. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> --------- Signed-off-by: Hongxia Yang <hongxia.yang@amd.com> Co-authored-by: Hongxia Yang <hongxia.yang@amd.com> Co-authored-by: Claude <noreply@anthropic.com> Co-authored-by: functionstackx <47992694+functionstackx@users.noreply.github.com>
|
@xianbaoqian Can you help review and merge this PR? |
|
LGTM. Thanks! |
Update recipe for MiniMax-M3 MXFP8 on MI355x only
Added several environment variables
VLLM_USE_BREAKABLE_CUDAGRAPH=0 — still needed.
Only Merge this PR after that.