Skip to content

[Klaud Cold] minimaxm31flash-fp4-mi355x-vllm-agentic-dspark: day-zero MiniMax-M3.1-Flash MI355X recipe - #4420

Merged
Oseltamivir merged 12 commits into
mainfrom
feat/minimaxm31flash-mi355x-dayzero
Oct 23, 2026
Merged

Oseltamivir merged 12 commits into
mainfrom
feat/minimaxm31flash-mi355x-dayzero

Conversation

@Oseltamivir

@Oseltamivir Oseltamivir commented Oct 23, 2026 •

Copy link
Copy Markdown
Collaborator

Day-zero MiniMax-M3.1-Flash (MiniMaxAI/MiniMax-M3.1-Flash) AgentX on MI355X with vLLM, following the vLLM blog and the recipes.vllm.ai MI355X TP4 recipe.

  • Image: vllm/vllm-openai-rocm:minimax-m31-flash, pinned to digest sha256:dbe558ed… (verified on Docker Hub, pushed 2026-10-23).
  • Recipe: minimaxm31flash/vllm/mi355x-fp4/agentic.yaml
    • TP4, block size 128 (MSA), and an nvfp4 KV cache.
    • Indexer KV is nvfp4 with indexer_decode_cp.
    • quantization-config {"moe":"mxfp8"} converts the NVFP4 experts to MXFP8 for the AITER MoE.
    • --disable-hybrid-kv-cache-manager.
    • Env: VLLM_ROCM_USE_AITER=1, VLLM_ROCM_USE_AITER_FUSION_SHARED_EXPERTS=1.
    • Parsers: minimax_m3 for reasoning and tool calls.
    • Context: 1M.
  • Search space (minimaxm31flash-fp4-mi355x-vllm-agentic-dspark): TP4 at conc 1, 2, 4, 8, 16, 32, 64 and 128, with no KV offload.
  • DSpark: native DSpark matching the blog's MI355X runs: TRITON_ATTN drafter, fp8 draft KV, probabilistic draft sampling. 7 draft tokens at conc 1-4 and 3 from conc 8. Throughput runs use golden AL curve infx/golden_al_distribution/minimaxm31flash_dspark.yaml, merged in [Klaud Cold] Add MiniMax-M3.1-Flash DSpark golden AL curve and SPEED-Bench collector / [Klaud Cold] 新增 MiniMax-M3.1-Flash DSpark 黄金 AL 曲线与 SPEED-Bench 采集脚本 #4425: AL 4.04 at 7 drafts and 2.88 at 3 drafts, thinking_on.
  • Docs: adds minimaxm31flash to the docs/MODELS.md support matrix.

@Oseltamivir Oseltamivir added the full-sweep-fail-fast Full sweep with canary gate; first failure cancels the rest of that matrix (recommended) label Oct 23, 2026
@github-actions

Copy link
Copy Markdown
Contributor

Thanks for the contribution!

  • Review: If this PR changes files owned by someone other than a repository admin or @SemiAnalysisAI/core, ask one eligible CODEOWNER to complete the latest PR_REVIEW_CHECKLIST.md before contacting a core maintainer on Slack. Follow the template exactly, including As a PR reviewer and CODEOWNER, I have reviewed this and have, so sign-off verification triggers.
  • PR verification: Sweeps run only when the PR appends an inferencex-e2e/perf-changelog.yaml entry and carries exactly one primary label: full-sweep-fail-fast (strongly recommended; canary plus per-matrix fail-fast), full-sweep-enabled (canary; matrix jobs continue after a failure), or non-canary-full-sweep-enabled (no canary or fail-fast). The modifiers all-evals, evals-only, and agentx-fast require a primary label. On fork PRs, a maintainer applies the label. See sweep labels and reuse.
  • After merging: PR authors must ensure all GitHub Actions jobs pass. Transient failures often pass on rerun; see how to rerun failed jobs.
中文

感谢你的贡献!

  • **审阅:**如果 PR 修改的文件归属于仓库管理员及 @SemiAnalysisAI/core 之外的 CODEOWNER,请先联系一位有资格的 CODEOWNER 填写最新的 PR_REVIEW_CHECKLIST.md,再通过 Slack 联系核心维护者。必须严格遵循模板,并保留 As a PR reviewer and CODEOWNER, I have reviewed this and have,才能触发签核验证。
  • **PR 验证:**只有当 PR 在 inferencex-e2e/perf-changelog.yaml 末尾追加了条目,并且恰好带有一个主标签时,才会运行扫描:full-sweep-fail-fast(强烈推荐;canary 加逐矩阵 fail-fast)、full-sweep-enabled(有 canary;矩阵任务在失败后继续运行)或 non-canary-full-sweep-enabled(无 canary,也无 fail-fast)。修饰标签 all-evals、evals-only 和 agentx-fast 必须与主标签一起使用。fork PR 的标签由维护者添加。参见扫描标签与复用。
  • **合并后:**PR 作者必须确保所有 GitHub Actions 任务通过。临时性失败通常可以通过重新运行恢复;参见重新运行失败任务的说明。

@claude claude Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Findings marked 🟡 are optional suggestions and need no follow-up push.

Comment thread inferencex-e2e/perf-changelog.yaml Outdated
- "Add day-zero MiniMax-M3.1-Flash (MiniMaxAI/MiniMax-M3.1-Flash, NVFP4 experts + NVFP4 KV) AgentX on MI355X with vLLM image vllm/vllm-openai-rocm:minimax-m31-flash (https://vllm.ai/blog/2026-10-23-minimax-m31-flash)."
- "Serve flags follow the recipes.vllm.ai MI355X TP4 recipe: block size 128, nvfp4 KV and indexer cache with indexer decode CP, MoE requantized to MXFP8, hybrid KV cache manager disabled, AITER + fused shared experts, minimax_m3 reasoning and tool parsers, 1M context."
- "Search space: TP4 at conc 1-128, no KV offloading. DSpark is off until a golden AL curve exists."
pr-link: TBD

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🔴 This PR's changelog validation will fail, and no result collection is triggered for the new config key. The appended entry uses pr-link: TBD. validate_added_pr_link accepts only the canonical PR URL or an XXX placeholder. Fix: set pr-link to https://github.com/SemiAnalysisAI/InferenceX/pull/<number>, or to XXX until the number is known.

Why this was flagged

The new block at perf-changelog.yaml:10881 sets pr-link: TBD. For appended entries, infx/workflows/validate_perf_changelog.py:193 calls validate_added_pr_link. On a PR run (lines 146-150), any link not in PR_LINK_PLACEHOLDERS ("XXX" and "https://github.com/SemiAnalysisAI/InferenceX/pull/XXX") and not equal to the PR's own URL raises "new PR entry must use ... or an XXX placeholder; found 'TBD'". On main (line 142), it must match CANONICAL_PR_LINK, so it fails there too. The changelog check therefore rejects this PR. Without a valid changelog entry, results for minimaxm31flash-fp4-mi355x-vllm-agentic are never collected. The base branch has no such entry and passes.

Verification: The trigger is merging this PR through the supported infx.workflows.merge_with_reuse path (AGENTS.md:140). At that step the appended pr-link: TBD is rejected and the merge aborts. merge_with_reuse.py:631 calls canonicalize_changelog, which calls compare_entries and raises on 'TBD'. The PR's CI changelog check does not reach this code.

Comment on lines +1 to +4
# MiniMax-M3.1-Flash AgentX on MI355X with vLLM, following the MI355X TP4
# recipe at https://recipes.vllm.ai/MiniMaxAI/MiniMax-M3.1-Flash and
# https://vllm.ai/blog/2026-10-23-minimax-m31-flash. DSpark is off until a
# MiniMax-M3.1-Flash golden AL curve is committed.

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟡 nit (optional): The new recipe and master-config entry add header and provenance comments. AGENTS.md "Code comments (mandatory)" forbids both. Recipe lines 1-4 describe the file and cite upstream recipe and blog URLs. amd-master.yaml:1535-1539 adds a block header with "day-zero ... image from , verified 2026-10-23". The "Native zip expansion..." comment above zip_override_tp4 (agentic.yaml:65-66) is a section header. Fix: delete these headers and the provenance-only comments "# Upstream AMD recipe environment." (agentic.yaml:56) and "(upstream spells this ...)" (agentic.yaml:48). Keep the short technical per-line comments.

Why this was flagged

AGENTS.md "Code comments (mandatory)" applies to recipes and configs. It says "No header comments", meaning nothing above a recipe, override or section. It also says "No history", meaning no comments on where something was derived from. agentic.yaml:1-4 is a file header with upstream URLs. agentic.yaml:65-66 is a header above zip_override_tp4. agentic.yaml:48 and :56 only record upstream provenance. amd-master.yaml:1535-1539 is a header above minimaxm31flash-fp4-mi355x-vllm-agentic that cites the source blog and a verification date. Each of these is new in this diff. The PR adds comments the repository's mandatory policy forbids, and the base has none of them. Older recipes such as kimik3/vllm/mi355x-fp4-mtp/agentic.yaml break the same rule. They are precedent only, not an exemption.

Verification: AGENTS.md:60-65 ("Code comments (mandatory)") covers "every file, including recipes, configs". It forbids "header comments ... at the top of a file or above a recipe, override, or section" and comments describing "where it was derived from". agentic.yaml:1-4 is a file header that cites https://vllm.ai/blog/.... Nothing breaks at runtime: these are comment-only changes.

@github-actions

github-actions Bot commented Oct 23, 2026 •

Copy link
Copy Markdown
Contributor

@Oseltamivir Oseltamivir removed the full-sweep-fail-fast Full sweep with canary gate; first failure cancels the rest of that matrix (recommended) label Oct 23, 2026
@benenzhu

Copy link
Copy Markdown
Collaborator

May I made some changes to the PR? Conc128 & 64 can't run I think. bad cache hit rate. @Oseltamivir

@Oseltamivir Oseltamivir changed the title [Klaud Cold] minimaxm31flash-fp4-mi355x-vllm-agentic: day-zero MiniMax-M3.1-Flash MI355X recipe [Klaud Cold] minimaxm31flash-fp4-mi355x-vllm-agentic-dspark: day-zero MiniMax-M3.1-Flash MI355X recipe Oct 23, 2026
@Oseltamivir Oseltamivir added full-sweep-fail-fast Full sweep with canary gate; first failure cancels the rest of that matrix (recommended) and removed full-sweep-fail-fast Full sweep with canary gate; first failure cancels the rest of that matrix (recommended) labels Oct 23, 2026
…point, conc up to 64

- Model amd/MiniMax-M3.1-Flash-Quark-MXFP4 (routed experts stored in OCP
  MXFP4; every other layer and the dspark/ draft as in the base checkpoint).
- MoE as the export recommends: experts loaded as stored on the AITER W4A4
  MoE (moe-backend aiter, VLLM_ROCM_USE_AITER_MOE=1), no shared-expert fusion;
  no MXFP8 requantization.
- Drop conc 128 (search space and the DSpark-3 zip).

Signed-off-by: Taoyu Zhu <zhutaoyu97@gmail.com>
…ill steps

Signed-off-by: Taoyu Zhu <zhutaoyu97@gmail.com>
@benenzhu benenzhu added full-sweep-fail-fast Full sweep with canary gate; first failure cancels the rest of that matrix (recommended) and removed full-sweep-fail-fast Full sweep with canary gate; first failure cancels the rest of that matrix (recommended) labels Oct 23, 2026
…P_ALIVE=900

Signed-off-by: Taoyu Zhu <zhutaoyu97@gmail.com>
…rk 7 at conc 1-8

Conc 1-8 run DSpark 7 with the recipe's mono decode (VLLM_ROCM_MINIMAX_M3_MONO=1, FP8 KV and indexer caches, shuffled KV layout, FULL_DECODE_ONLY graphs, no DBO); AgentX tool-call gaps keep c4 and c8 decode steps within mono's 16 tokens. Conc 16-64 keep DSpark 3, NVFP4 KV and DBO. The image moves to the current minimax-m31-flash tag (sha256:ae868121), which carries the sparse PA K-view fix mono needs (vllm #63270).

Signed-off-by: Taoyu Zhu <zhutaoyu97@gmail.com>
…UES=0 on the DBO lanes

The image's HIP moves an active stream to the least loaded hardware queue, which can put DBO's all-reduce stream on the compute stream's queue: one serialized TP rank took a 32K-token prefill from 581 to 938 ms. With static queues, 4 server launches kept all 16 ranks overlapped (91-95% of the all-reduce hidden, 563-579 ms).

Signed-off-by: Taoyu Zhu <zhutaoyu97@gmail.com>
Signed-off-by: Taoyu Zhu <zhutaoyu97@gmail.com>

# Conflicts:
#	inferencex-e2e/perf-changelog.yaml
@benenzhu benenzhu added non-canary-full-sweep-enabled Run the full sweep without the canary gate (full search space, no trim) and removed full-sweep-fail-fast Full sweep with canary gate; first failure cancels the rest of that matrix (recommended) non-canary-full-sweep-enabled Run the full sweep without the canary gate (full search space, no trim) labels Oct 23, 2026
@benenzhu

Copy link
Copy Markdown
Collaborator

@Oseltamivir

Copy link
Copy Markdown
Collaborator Author

As a PR reviewer and CODEOWNER review code + recipe link

@github-actions

Copy link
Copy Markdown
Contributor

❌❌❌ REJECTED ❌❌❌

⚠️ Pareto coverage needs additional review: @functionstackx @cquil11 @Oseltamivir @adibarra. At least 5 points per affected throughput-versus-E2EL frontier are highly recommended. Below 5, or when coverage cannot be verified, merge only with an explicit, recorded admin bypass for the assessed commit; this advisory comment does not grant or enforce a bypass.

@Oseltamivir This sign-off cannot be accepted. You are not a CODEOWNER of amd-master.yaml. The sign-off contains only one line, with no checklist and no merged upstream recipe link.

❌ Check 0 (CODEOWNER): FAIL — The signer does not own a specific changed path: inferencex-e2e/configs/amd-master.yaml is owned only by @billishyahao @chunfangamd @seungrokj @1am9trash @zijiecode, and the core team is not listed on that line.

❌ Check 3 (Recipe): FAIL — The sign-off has no link to a merged vLLM recipe PR or a published recipes.vllm.ai page. Its whole body is "As a PR reviewer and CODEOWNER review code + recipe link". The only recipe link in the thread is vllm-project/vllm#63257, which is an engine PR and not vllm-project/recipes. The upstream coverage of the Quark MXFP4 checkpoint, the mono-decode/FP8-KV lane at c1-c8 and DBO at c16-c64 is undocumented.

❌ Check 5 (Checklist template): FAIL — The sign-off includes none of the 18 items in the current PR_REVIEW_CHECKLIST.md, and it has no additional detail section.

⚠️ Check 4 (Reuse command): WARN — No authorized reuse command has been posted on this PR. Have an authorized maintainer comment /use 39929767608 on its own line before merging. Without it, the push-to-main merge-ingest.yml "Require sweep reuse" step fails for this perf-changelog entry and the results never publish.

⚠️ Check 13 (Draft precision): WARN — Draft precision could not be verified for minimaxm31flash-fp4-mi355x-vllm-agentic-dspark, so human review is needed. The sign-off gives no evidence for the stored precision of the dspark/ draft in amd/MiniMax-M3.1-Flash-Quark-MXFP4, which the recipe says matches MiniMaxAI/MiniMax-M3.1-Flash. It also gives no evidence that the image's default handling and the explicit "kv_cache_dtype":"fp8" draft KV leave the draft at its shipped precision.

⚠️ Check 14 (Pareto coverage): WARN — The curve minimaxm31flash / agentic-coding / mi355x_vllm / fp4 / p90 E2EL has 4/5 frontier points (c8, c16, c32, c64). c1, c2 and c4 are dominated. Evidence: bmk_agentic_* artifacts from run 39929767608 attempt 1, SHA 3fb089b, image minimax-m31-flash@sha256:ae868121. No admin exception has been posted.

⚠️ Check 15 (PR description): WARN — The body contradicts the pinned config. It says "conc 1, 2, 4, 8, 16, 32, 64 and 128", but the search space is c1-c64. It says "7 draft tokens at conc 1-4 and 3 from conc 8", but the recipe uses 7 drafts at c1-c8 and 3 from c16. It cites quantization-config {"moe":"mxfp8"}, which is absent; the MXFP4 experts load as stored via moe-backend: aiter. It says VLLM_ROCM_USE_AITER_FUSION_SHARED_EXPERTS=1, but the recipe sets '0'. It names digest sha256:dbe558ed…, but the config pins sha256:ae868121…. It names model MiniMaxAI/MiniMax-M3.1-Flash, but the config serves amd/MiniMax-M3.1-Flash-Quark-MXFP4. It claims an nvfp4 KV cache throughout, but c1-c8 use fp8. Please correct the PR body.

Passed and not applicable checks

✅ Check 1 (Sweep on in-PR commit): PASS — Head commit 3fb089b has green, executed agentic / jobs for c1-c64 and a green agentic eval / job (run 39929767608).

✅ Check 2 (Eval accuracy): PASS — GSM8K em_strict is 0.967 (±0.005, n=1319) at TP4 c64, on the same image vllm/vllm-openai-rocm:minimax-m31-flash@sha256:ae868121… (eval_results_all, run 39929767608).

✅ Check 6 (Upstream image): PASS — vllm/vllm-openai-rocm:minimax-m31-flash@sha256:ae868121… comes from the upstream vLLM org, and framework: vllm is an open-source engine, so the ordering rule does not apply.

✅ Check 7 (Deprecated models): PASS — minimaxm31flash is a new model whose agentic-coding scenario is active. MODELS.md has no deprecation that applies on 2026-10-23.

✅ Check 8 (No architecture hacks): PASS — There are no --hf-overrides, model-override arguments or config edits. The Quark MXFP4 routed experts quantize the target only and do not remove FLOPs.

✅ Check 9 (Spec-decode chat template): PASS — The DSpark configs are benchmarked through AgentX trace replay on the chat-completions endpoint with the minimax_m3 reasoning and tool parsers.

✅ Check 10 (No engine patches): PASS — The PR adds no patches, inline source rewrites, monkey-patches or engine wheel installs, only serving flags and environment variables.

✅ Check 11 (Golden AL): PASS — The harness injects rejection_sample_method: synthetic with AL 4.04 at 7 drafts and 2.88 at 3 drafts. Both values match minimaxm31flash_dspark.yaml thinking_on and were confirmed in the c1 and c16 launch logs of run 39929767608.

➖ Check 12 (Append-only): N/A — The new perf-changelog entry is not marked append-only: true.

➖ Check 16 (NVIDIA overview): N/A — The only affected hardware is AMD MI355X.

➖ Check 17 (Metrics publication): N/A — The config uses plain vLLM with a vLLM frontend and no Dynamo, so these Dynamo metrics settings do not apply.

Assessed commit: 3fb089b35e4fef80f542b7c5583db2795ee024b5.

@Oseltamivir

Copy link
Copy Markdown
Collaborator Author

/use 39929767608

@github-actions

github-actions Bot commented Oct 23, 2026 •

Copy link
Copy Markdown
Contributor

@Oseltamivir staged run 39929767608.

This run remains available across future /stage-results requests. Staging the same run ID again updates its staged data. Staging workflow

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

non-canary-full-sweep-enabled Run the full sweep without the canary gate (full search space, no trim)

Projects

Status: Done

Development

Successfully merging this pull request may close these issues.

2 participants