Skip to content

[Klaud Cold] minimaxm31flash-fp4-h200-vllm-agentic-dspark: day-zero MiniMax-M3.1-Flash H200 DSpark recipe - #4423

Merged
Oseltamivir merged 5 commits into
mainfrom
feat/minimaxm31flash-h200-dayzero
Oct 23, 2026
Merged

Oseltamivir merged 5 commits into
mainfrom
feat/minimaxm31flash-h200-dayzero

Conversation

@Oseltamivir

Copy link
Copy Markdown
Collaborator

Summary

Day-zero MiniMax-M3.1-Flash DSpark (MiniMaxAI/MiniMax-M3.1-Flash) AgentX on H200 with vLLM, from https://vllm.ai/blog/2026-10-23-minimax-m31-flash. This follows the Hopper section of https://recipes.vllm.ai/MiniMaxAI/MiniMax-M3.1-Flash.

  • Image: vllm/vllm-openai:minimax-m31-flash, pinned to the multiarch digest sha256:86f582da… (amd64 = minimax-m31-flash-x86_64-cu134).
  • Serve flags: TP8, --block-size 128, --kv-cache-dtype fp8, indexer_kv_dtype: fp8, --moe-backend marlin (NVFP4 experts; Hopper has no FP4 tensor cores), FULL_AND_PIECEWISE graphs, --disable-hybrid-kv-cache-manager, the minimax_m3 parsers, and safetensors-load-strategy: prefetch for the h200-dgxc VIRTIOFS cache.
  • Speculative decoding: three-token DSpark, using the GB200 recipe's config (FLASH_ATTN drafter, fp8 draft KV, block rejection). Upstream doesn't enable DSpark on Hopper, so whether the drafter runs on Hopper is still unverified. Throughput runs use synthetic acceptance from minimaxm31flash_dspark.yaml, which isn't committed yet (an AL curve run is in progress). The sweep is on hold until it lands.
  • Deviation from upstream: max-model-len is 1M rather than the 256K in the upstream Hopper example, because the AgentX corpus for this prefix is uncapped.
  • Search space: TP8, conc 1–24 (h200-dgxc), no KV offload.

Siblings: #4422 (H100), #4419 (GB200/GB300), #4420 (MI355X), #4421 (B200/B300).

🤖 Generated with Claude Code

…lash AgentX on H200

Hopper TP8 recipe from recipes.vllm.ai: FP8 KV and indexer caches, NVFP4
experts on Marlin MoE, block size 128, FULL_AND_PIECEWISE graphs, plus the
GB200 recipe's three-token DSpark. Context kept at 1M for the uncapped
AgentX corpus.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@github-actions

Copy link
Copy Markdown
Contributor

Thanks for the contribution!

  • Review: If this PR changes files owned by someone other than a repository admin or @SemiAnalysisAI/core, ask one eligible CODEOWNER to complete the latest PR_REVIEW_CHECKLIST.md before contacting a core maintainer on Slack. Follow the template exactly, including As a PR reviewer and CODEOWNER, I have reviewed this and have, so sign-off verification triggers.
  • PR verification: Sweeps run only when the PR appends an inferencex-e2e/perf-changelog.yaml entry and carries exactly one primary label: full-sweep-fail-fast (strongly recommended; canary plus per-matrix fail-fast), full-sweep-enabled (canary; matrix jobs continue after a failure), or non-canary-full-sweep-enabled (no canary or fail-fast). The modifiers all-evals, evals-only, and agentx-fast require a primary label. On fork PRs, a maintainer applies the label. See sweep labels and reuse.
  • After merging: PR authors must ensure all GitHub Actions jobs pass. Transient failures often pass on rerun; see how to rerun failed jobs.
中文

感谢你的贡献!

  • **审阅:**如果 PR 修改的文件归属于仓库管理员及 @SemiAnalysisAI/core 之外的 CODEOWNER,请先联系一位有资格的 CODEOWNER 填写最新的 PR_REVIEW_CHECKLIST.md,再通过 Slack 联系核心维护者。必须严格遵循模板,并保留 As a PR reviewer and CODEOWNER, I have reviewed this and have,才能触发签核验证。
  • **PR 验证:**只有当 PR 在 inferencex-e2e/perf-changelog.yaml 末尾追加了条目,并且恰好带有一个主标签时,才会运行扫描:full-sweep-fail-fast(强烈推荐;canary 加逐矩阵 fail-fast)、full-sweep-enabled(有 canary;矩阵任务在失败后继续运行)或 non-canary-full-sweep-enabled(无 canary,也无 fail-fast)。修饰标签 all-evals、evals-only 和 agentx-fast 必须与主标签一起使用。fork PR 的标签由维护者添加。参见扫描标签与复用。
  • **合并后:**PR 作者必须确保所有 GitHub Actions 任务通过。临时性失败通常可以通过重新运行恢复;参见重新运行失败任务的说明。

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

@claude claude Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Findings marked 🟡 are optional suggestions and need no follow-up push.

Comment on lines +5280 to +5295
minimaxm31flash-fp4-h200-vllm-agentic-dspark:
image: vllm/vllm-openai:minimax-m31-flash@sha256:86f582dae25ea92802353727164ae4eb03b37b18baf9119b805ffaec2a7ef07c
model: MiniMaxAI/MiniMax-M3.1-Flash
model-prefix: minimaxm31flash
runner: cluster:h200-dgxc
precision: fp4
framework: vllm
multinode: false
srt-recipe-dir: minimaxm31flash/vllm/h200-fp4-mtp
scenarios:
agentic-coding:
- dram-utilization: 0.80
search-space:
# TP8 is the upstream Hopper layout. The 250 GB checkpoint leaves ~88 GiB of
# KV per 141 GB GPU, about 1.3x MiniMax-M3 H200 (run to c14).
- { tp: 8, kv-offloading: none, spec-decoding: mtp, conc-list: [1, 2, 4, 8, 12, 16, 20, 24], srt-recipe: agentic.yaml }

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🔴 Every throughput run of this new config will fail before benchmarking, because its golden acceptance curve is not in the repo. The vLLM speculative-config is dspark, so the srt driver looks up minimaxm31flash_dspark.yaml in infx/golden_al_distribution/. That lookup raises "No committed golden curve" (curves.py:68), and the PR description confirms the file is not committed. Fix: commit minimaxm31flash_dspark.yaml in this PR, or hold back the master entry and its perf-changelog entry until the curve lands, so no run can be scheduled without it.

Why this was flagged

Any non-eval agentic run of minimaxm31flash-fp4-h200-vllm-agentic-dspark has SPEC_DECODING=mtp (nvidia-master.yaml:5295). It also has a vLLM speculative-config with method dspark (agentic.yaml:44). So acceptance_length in infx/srt_slurm/synthetic_acceptance.py calls golden_length("minimaxm31flash", spec, ...). load_curve raises ValueError "No committed golden curve minimaxm31flash_dspark.yaml in ..." at infx/golden_al_distribution/curves.py:68. The only files under golden_al_distribution/ for this family are minimaxm3_eagle3.yaml and minimaxm3_eagle3_gqa.yaml. Nothing checks this before merge: only the golden_al_distribution unit tests use GOLDEN_DIR, and they do not cross-check master configs. The perf-changelog entry adds this key to the benchmark set, so once merged every scheduled sweep point (8 concurrencies) fails at override construction and produces no results. MODELS.md and the changelog also cite a curve that does not exist. The PR description says the sweep is "on hold", but nothing in the code enforces that hold.

Verification: Trigger: any non-eval agentic throughput run of this config key. There is no minimaxm31flash_dspark.yaml in infx/golden_al_distribution/. load_curve raises at curves.py:68, and main returns 1 before any srtctl apply is submitted. No test checks master entries against the curve files, so CI will not catch this before merge.

Comment thread inferencex-e2e/perf-changelog.yaml Outdated
- "Add day-zero MiniMax-M3.1-Flash (MiniMaxAI/MiniMax-M3.1-Flash, NVFP4 experts + MXFP8 dense) AgentX on H200 with vLLM image minimax-m31-flash (https://vllm.ai/blog/2026-10-23-minimax-m31-flash)."
- "Serve flags follow the recipes.vllm.ai Hopper TP8 recipe: block size 128, FP8 KV and indexer cache, Marlin MoE backend for the NVFP4 experts, FULL_AND_PIECEWISE graphs, hybrid KV cache manager disabled, minimax_m3 reasoning and tool parsers, three-token DSpark (FLASH_ATTN drafter, fp8 draft KV, block rejection, from the GB200 recipe; upstream does not enable DSpark on Hopper). Context stays at 1M (upstream Hopper example uses 256K) because the AgentX corpus for this prefix is uncapped."
- "Search space: TP8 only, conc 1-24, no KV offloading. Throughput runs use synthetic acceptance from the minimaxm31flash_dspark golden curve."
pr-link: TBD

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🔴 The changelog validator rejects this new entry, so the PR's changelog check fails. pr-link: TBD is neither the canonical PR URL nor an accepted placeholder. Fix: use pr-link: https://github.com/SemiAnalysisAI/InferenceX/pull/XXX, or XXX, or the real PR URL. Those are the only values validate_added_pr_link accepts.

Why this was flagged

validate_added_pr_link in inferencex-e2e/infx/workflows/validate_perf_changelog.py:139-150 checks new PR entries. On PR runs it allows only values in PR_LINK_PLACEHOLDERS ("XXX" and "https://github.com/SemiAnalysisAI/InferenceX/pull/XXX", lines 22-25) or the expected PR URL. Anything else raises "new PR entry must use ... or an XXX placeholder; found 'TBD'". On main, where pr_number is None, it requires CANONICAL_PR_LINK, so TBD fails there too. The appended block at perf-changelog.yaml:10881 sets pr-link: TBD, so changelog validation fails for this PR. The merge-time canonicalisation in merge_with_reuse also rewrites only recognised placeholders, so it will not fix this value.

Verification: The appended block ends with pr-link: TBD (perf-changelog.yaml diff, new entry). ChangelogEntry.pr_link is a plain str (infx/matrix/validation.py:1062), so the PR's own changelog check does not reject TBD. The rejection actually happens at merge. Result: the supported merge flow refuses this PR until pr-link is changed to XXX or the real URL.

Comment on lines +30 to +35
served-model-name: MiniMaxAI/MiniMax-M3.1-Flash
language-model-only: true
trust-remote-code: true
reasoning-parser: minimax_m3
tool-call-parser: minimax_m3
enable-auto-tool-choice: true

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟡 (optional) Throughput results may pair a thinking_on golden acceptance length with replay traffic served in the template's default thinking mode. That mode may not be thinking-on. Every MiniMax-M3 vLLM AgentX recipe pins default-chat-template-kwargs: '{"thinking_mode":"enabled"}', but this new recipe's args do not. The AgentX client sends no thinking toggle. Fix: add default-chat-template-kwargs: '{"thinking_mode":"enabled"}' to roles.agg.args, or confirm the M3.1-Flash template defaults to thinking-on. Either way the served mode matches the thinking_on curve the driver picks.

Why this was flagged

The condition is the MiniMax-M3.1-Flash chat template not defaulting to thinking-on. I could not check this from the checkout, and the M3 template's toggle is the thinking_mode kwarg (speedbench/minimaxm3_fp4_b300_vllm.sh:43-44). The srt planner hard-codes THINKING_MODE = "thinking_on" (infx/srt_slurm/generate.py:32). That value selects the golden curve in acceptance_length (infx/srt_slurm/synthetic_acceptance.py:131). The recipe args at agentic.yaml:30-58 set no default-chat-template-kwargs. A grep of infx/bench finds no chat_template_kwargs or thinking field in the AgentX client. So the server falls back to the template's default mode. All seven MiniMax-M3 vLLM/ATOM agentic recipes (for example minimaxm3/vllm/h200-fp8-mtp/agentic.yaml:40) pin thinking_mode enabled. perf-changelog.yaml:6361 explains the pin as "Pin thinking-on so the run matches the golden ... thinking_on" acceptance target. If the default is off, the new config publishes numbers whose output-length profile does not match the thinking_on AL it simulates. No validator compares the two.

Verification: Trigger: the MiniMax-M3.1-Flash chat template does not default to thinking-on. The new recipe's roles.agg.args (agentic.yaml lines 30-58) set no default-chat-template-kwargs. infx/srt_slurm/generate.py:32 hard-codes THINKING_MODE = "thinking_on". In that case the throughput numbers model thinking-on acceptance on traffic that is not in thinking mode.

Comment on lines +5274 to +5279
# MiniMax-M3.1-Flash AgentX on H200: day-zero vLLM image from
# https://vllm.ai/blog/2026-10-23-minimax-m31-flash, multiarch manifest verified
# 2026-10-23 (amd64 is minimax-m31-flash-x86_64-cu134). Serve flags follow the
# Hopper TP8 recipe: FP8 KV and indexer caches, NVFP4 experts on Marlin MoE.
# Upstream does not enable DSpark on Hopper; this adds the GB200 recipe's
# three-token DSpark, with synthetic acceptance from minimaxm31flash_dspark.yaml.

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟡 (optional) These new comments break the mandatory AGENTS.md comment rules, which forbid header comments and history or provenance comments. The block above the master entry describes the config and records where it came from ("day-zero vLLM image from ", "verified 2026-10-23"). Fix: delete the header and provenance text, keep only short per-line technical reasons, and leave the history to perf-changelog, which covers the 4 sites listed. Same pattern at 4 sites (configs/nvidia-master.yaml:5274-5279, configs/nvidia-master.yaml:5293-5294, h200-fp4-mtp/agentic.yaml:26-29, h200-fp4-mtp/agentic.yaml:65-67).

Why this was flagged

AGENTS.md:64 says never add broad comments above a recipe, override or section that say what the config is. AGENTS.md:65 forbids comments about where something was derived from or about past results. nvidia-master.yaml:5274-5279 is a header above the new entry that cites the blog URL, a verification date and upstream provenance ("Upstream does not enable DSpark on Hopper; this adds the GB200 recipe's..."). nvidia-master.yaml:5293-5294 compares against a past result ("about 1.3x MiniMax-M3 H200 (run to c14)"). agentic.yaml:26-29 says "Serve flags follow the Hopper TP8 recipe at https://recipes.vllm.ai/..." and "DSpark follows the GB200 recipe", which is derivation. agentic.yaml:65-67 is a section comment above the zip_override_tp8 block. The base branch has none of these comments. Nothing enforces the rule automatically, so the comments merge as written.

Verification: nit. Nothing breaks at runtime. The comments break the mandatory comment rules in AGENTS.md. Line 64 says "No header comments... above a recipe, override, or section." Line 65 says "No history... where it was derived from, what it replaced, or past results." The header at nvidia-master.yaml:5274-5279 sits directly above minimaxm31flash-fp4-h200-vllm-agentic-dspark:.

Oseltamivir and others added 2 commits October 23, 2026 13:02
@Oseltamivir Oseltamivir added the full-sweep-fail-fast Full sweep with canary gate; first failure cancels the rest of that matrix (recommended) label Oct 23, 2026
@github-actions

Copy link
Copy Markdown
Contributor

@Oseltamivir

Oseltamivir commented Oct 23, 2026 •

Copy link
Copy Markdown
Collaborator Author

/use 39903574233

1 similar comment
@Oseltamivir

Copy link
Copy Markdown
Collaborator Author

/use 39903574233

@github-actions

github-actions Bot commented Oct 23, 2026 •

Copy link
Copy Markdown
Contributor

@Oseltamivir staged run 39903574233.

This run remains available across future /stage-results requests. Staging the same run ID again updates its staged data. Staging workflow

…p-sweep]

Keep main's H100 entries and place the H200 master-config block after
them; append the H200 changelog entry at the end.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

full-sweep-fail-fast Full sweep with canary gate; first failure cancels the rest of that matrix (recommended)

Projects

Status: Done

Development

Successfully merging this pull request may close these issues.

1 participant