Repository navigation
[Klaud Cold] minimaxm31flash-fp4-mi355x-vllm-agentic-dspark: day-zero MiniMax-M3.1-Flash MI355X recipe - #4420
Conversation
…MI355X AgentX recipe
|
Thanks for the contribution!
中文感谢你的贡献!
|
| - "Add day-zero MiniMax-M3.1-Flash (MiniMaxAI/MiniMax-M3.1-Flash, NVFP4 experts + NVFP4 KV) AgentX on MI355X with vLLM image vllm/vllm-openai-rocm:minimax-m31-flash (https://vllm.ai/blog/2026-10-23-minimax-m31-flash)." | ||
| - "Serve flags follow the recipes.vllm.ai MI355X TP4 recipe: block size 128, nvfp4 KV and indexer cache with indexer decode CP, MoE requantized to MXFP8, hybrid KV cache manager disabled, AITER + fused shared experts, minimax_m3 reasoning and tool parsers, 1M context." | ||
| - "Search space: TP4 at conc 1-128, no KV offloading. DSpark is off until a golden AL curve exists." | ||
| pr-link: TBD |
There was a problem hiding this comment.
🔴 This PR's changelog validation will fail, and no result collection is triggered for the new config key. The appended entry uses pr-link: TBD. validate_added_pr_link accepts only the canonical PR URL or an XXX placeholder. Fix: set pr-link to https://github.com/SemiAnalysisAI/InferenceX/pull/<number>, or to XXX until the number is known.
Why this was flagged
The new block at perf-changelog.yaml:10881 sets pr-link: TBD. For appended entries, infx/workflows/validate_perf_changelog.py:193 calls validate_added_pr_link. On a PR run (lines 146-150), any link not in PR_LINK_PLACEHOLDERS ("XXX" and "https://github.com/SemiAnalysisAI/InferenceX/pull/XXX") and not equal to the PR's own URL raises "new PR entry must use ... or an XXX placeholder; found 'TBD'". On main (line 142), it must match CANONICAL_PR_LINK, so it fails there too. The changelog check therefore rejects this PR. Without a valid changelog entry, results for minimaxm31flash-fp4-mi355x-vllm-agentic are never collected. The base branch has no such entry and passes.
Verification: The trigger is merging this PR through the supported infx.workflows.merge_with_reuse path (AGENTS.md:140). At that step the appended pr-link: TBD is rejected and the merge aborts. merge_with_reuse.py:631 calls canonicalize_changelog, which calls compare_entries and raises on 'TBD'. The PR's CI changelog check does not reach this code.
| # MiniMax-M3.1-Flash AgentX on MI355X with vLLM, following the MI355X TP4 | ||
| # recipe at https://recipes.vllm.ai/MiniMaxAI/MiniMax-M3.1-Flash and | ||
| # https://vllm.ai/blog/2026-10-23-minimax-m31-flash. DSpark is off until a | ||
| # MiniMax-M3.1-Flash golden AL curve is committed. |
There was a problem hiding this comment.
🟡 nit (optional): The new recipe and master-config entry add header and provenance comments. AGENTS.md "Code comments (mandatory)" forbids both. Recipe lines 1-4 describe the file and cite upstream recipe and blog URLs. amd-master.yaml:1535-1539 adds a block header with "day-zero ... image from , verified 2026-10-23". The "Native zip expansion..." comment above zip_override_tp4 (agentic.yaml:65-66) is a section header. Fix: delete these headers and the provenance-only comments "# Upstream AMD recipe environment." (agentic.yaml:56) and "(upstream spells this ...)" (agentic.yaml:48). Keep the short technical per-line comments.
Why this was flagged
AGENTS.md "Code comments (mandatory)" applies to recipes and configs. It says "No header comments", meaning nothing above a recipe, override or section. It also says "No history", meaning no comments on where something was derived from. agentic.yaml:1-4 is a file header with upstream URLs. agentic.yaml:65-66 is a header above zip_override_tp4. agentic.yaml:48 and :56 only record upstream provenance. amd-master.yaml:1535-1539 is a header above minimaxm31flash-fp4-mi355x-vllm-agentic that cites the source blog and a verification date. Each of these is new in this diff. The PR adds comments the repository's mandatory policy forbids, and the base has none of them. Older recipes such as kimik3/vllm/mi355x-fp4-mtp/agentic.yaml break the same rule. They are precedent only, not an exemption.
Verification: AGENTS.md:60-65 ("Code comments (mandatory)") covers "every file, including recipes, configs". It forbids "header comments ... at the top of a file or above a recipe, override, or section" and comments describing "where it was derived from". agentic.yaml:1-4 is a file header that cites https://vllm.ai/blog/.... Nothing breaks at runtime: these are comment-only changes.
|
View unofficial run (performance): https://inferencex.semianalysis.com/inference?unofficialRun=39929767608 View unofficial run (accuracy): https://inferencex.semianalysis.com/evaluation?unofficialRun=39929767608 |
…(7 drafts c1-c4, 3 from c8)
|
May I made some changes to the PR? Conc128 & 64 can't run I think. bad cache hit rate. @Oseltamivir |
…point, conc up to 64 - Model amd/MiniMax-M3.1-Flash-Quark-MXFP4 (routed experts stored in OCP MXFP4; every other layer and the dspark/ draft as in the base checkpoint). - MoE as the export recommends: experts loaded as stored on the AITER W4A4 MoE (moe-backend aiter, VLLM_ROCM_USE_AITER_MOE=1), no shared-expert fusion; no MXFP8 requantization. - Drop conc 128 (search space and the DSpark-3 zip). Signed-off-by: Taoyu Zhu <zhutaoyu97@gmail.com>
…ill steps Signed-off-by: Taoyu Zhu <zhutaoyu97@gmail.com>
…P_ALIVE=900 Signed-off-by: Taoyu Zhu <zhutaoyu97@gmail.com>
…rk 7 at conc 1-8 Conc 1-8 run DSpark 7 with the recipe's mono decode (VLLM_ROCM_MINIMAX_M3_MONO=1, FP8 KV and indexer caches, shuffled KV layout, FULL_DECODE_ONLY graphs, no DBO); AgentX tool-call gaps keep c4 and c8 decode steps within mono's 16 tokens. Conc 16-64 keep DSpark 3, NVFP4 KV and DBO. The image moves to the current minimax-m31-flash tag (sha256:ae868121), which carries the sparse PA K-view fix mono needs (vllm #63270). Signed-off-by: Taoyu Zhu <zhutaoyu97@gmail.com>
…UES=0 on the DBO lanes The image's HIP moves an active stream to the least loaded hardware queue, which can put DBO's all-reduce stream on the compute stream's queue: one serialized TP rank took a 32K-token prefill from 581 to 938 ms. With static queues, 4 server launches kept all 16 ranks overlapped (91-95% of the all-reduce hidden, 563-579 ms). Signed-off-by: Taoyu Zhu <zhutaoyu97@gmail.com>
Signed-off-by: Taoyu Zhu <zhutaoyu97@gmail.com> # Conflicts: # inferencex-e2e/perf-changelog.yaml
|
As a PR reviewer and CODEOWNER review code + recipe link |
❌❌❌ REJECTED ❌❌❌@Oseltamivir This sign-off cannot be accepted. You are not a CODEOWNER of ❌ Check 0 (CODEOWNER): FAIL — The signer does not own a specific changed path: ❌ Check 3 (Recipe): FAIL — The sign-off has no link to a merged vLLM recipe PR or a published recipes.vllm.ai page. Its whole body is "As a PR reviewer and CODEOWNER review code + recipe link". The only recipe link in the thread is vllm-project/vllm#63257, which is an engine PR and not ❌ Check 5 (Checklist template): FAIL — The sign-off includes none of the 18 items in the current PR_REVIEW_CHECKLIST.md, and it has no additional detail section.
Passed and not applicable checks✅ Check 1 (Sweep on in-PR commit): PASS — Head commit 3fb089b has green, executed ✅ Check 2 (Eval accuracy): PASS — GSM8K em_strict is 0.967 (±0.005, n=1319) at TP4 c64, on the same image ✅ Check 6 (Upstream image): PASS — ✅ Check 7 (Deprecated models): PASS — ✅ Check 8 (No architecture hacks): PASS — There are no ✅ Check 9 (Spec-decode chat template): PASS — The DSpark configs are benchmarked through AgentX trace replay on the chat-completions endpoint with the ✅ Check 10 (No engine patches): PASS — The PR adds no patches, inline source rewrites, monkey-patches or engine wheel installs, only serving flags and environment variables. ✅ Check 11 (Golden AL): PASS — The harness injects ➖ Check 12 (Append-only): N/A — The new perf-changelog entry is not marked ➖ Check 16 (NVIDIA overview): N/A — The only affected hardware is AMD MI355X. ➖ Check 17 (Metrics publication): N/A — The config uses plain vLLM with a vLLM frontend and no Dynamo, so these Dynamo metrics settings do not apply. Assessed commit: |
|
/use 39929767608 |
|
@Oseltamivir staged run 39929767608. This run remains available across future |
Day-zero MiniMax-M3.1-Flash (
MiniMaxAI/MiniMax-M3.1-Flash) AgentX on MI355X with vLLM, following the vLLM blog and the recipes.vllm.ai MI355X TP4 recipe.vllm/vllm-openai-rocm:minimax-m31-flash, pinned to digestsha256:dbe558ed…(verified on Docker Hub, pushed 2026-10-23).minimaxm31flash/vllm/mi355x-fp4/agentic.yamlnvfp4KV cache.nvfp4withindexer_decode_cp.quantization-config {"moe":"mxfp8"}converts the NVFP4 experts to MXFP8 for the AITER MoE.--disable-hybrid-kv-cache-manager.VLLM_ROCM_USE_AITER=1,VLLM_ROCM_USE_AITER_FUSION_SHARED_EXPERTS=1.minimax_m3for reasoning and tool calls.minimaxm31flash-fp4-mi355x-vllm-agentic-dspark): TP4 at conc 1, 2, 4, 8, 16, 32, 64 and 128, with no KV offload.TRITON_ATTNdrafter, fp8 draft KV, probabilistic draft sampling. 7 draft tokens at conc 1-4 and 3 from conc 8. Throughput runs use golden AL curveinfx/golden_al_distribution/minimaxm31flash_dspark.yaml, merged in [Klaud Cold] Add MiniMax-M3.1-Flash DSpark golden AL curve and SPEED-Bench collector / [Klaud Cold] 新增 MiniMax-M3.1-Flash DSpark 黄金 AL 曲线与 SPEED-Bench 采集脚本 #4425: AL 4.04 at 7 drafts and 2.88 at 3 drafts, thinking_on.minimaxm31flashto thedocs/MODELS.mdsupport matrix.