Skip to content

[Klaud Cold] minimaxm31flash-fp4-gb300-vllm-agentic-dspark: day-zero MiniMax-M3.1-Flash GB300 DSpark recipe - #4419

Merged
Oseltamivir merged 9 commits into
mainfrom
feat/minimaxm31flash-gb200-gb300-dayzero
Oct 23, 2026
Merged

Oseltamivir merged 9 commits into
mainfrom
feat/minimaxm31flash-gb200-gb300-dayzero

Conversation

@Oseltamivir

@Oseltamivir Oseltamivir commented Oct 23, 2026 •

Copy link
Copy Markdown
Collaborator

Summary

Day-zero MiniMax-M3.1-Flash AgentX on GB300 with vLLM and DSpark. It follows the merged B200 recipe (#4426) with the vLLM blog's GB300 drafter and one extra DEP4 point. GB200 is #4433.

  • Config key: minimaxm31flash-fp4-gb300-vllm-agentic-dspark.
  • Recipe: minimaxm31flash/vllm/b200-fp4-mtp/agentic.yaml with these changes:
    • gpu_type: gb300 and gpus_per_node: 4.
    • The DSpark drafter runs on FLASHINFER_TS (FlashInfer PrimiTS), as in the vLLM blog's GB300 AgentX runs. B200 uses FLASH_ATTN.
    • VLLM_CHAT_TEMPLATE_RENDER_TIMEOUT: '600'. Under concurrent load, rendering the chat template for long AgentX conversations exceeded vLLM's 30 s default, and vLLM aborted those requests during warmup.
    • An extra DEP4 c96 point (max-num-seqs 48).
  • Image: vllm/vllm-openai:minimax-m31-flash-arm64-cu134@sha256:5d76347…, the arm64 build of B200's image.
  • Search space:
    • TP4 at conc 1/4/8/16/32 with 7-token DSpark (golden AL 4.04).
    • DEP4 (TP1 × DP4 + EP4, vLLM Router) at conc 64/96/128 with 3-token DSpark (golden AL 2.88).

What changed from the first version of this PR

  • TP4 c64 is dropped. It overloaded on GB300: 66–77 requests waiting, 1–2 running, then a connection reset during warmup. On GB200 it was saturated too (median TTFT 131 s).
  • DEP4 c256 is dropped to match B200.
  • DEP4 now drafts 3 tokens instead of 7, as on B200.

Validation

infx generate binds all 8 points to exactly one variant each, with the expected speculative config, golden AL and render timeout.

@github-actions

Copy link
Copy Markdown
Contributor

Thanks for the contribution!

  • Review: If this PR changes files owned by someone other than a repository admin or @SemiAnalysisAI/core, ask one eligible CODEOWNER to complete the latest PR_REVIEW_CHECKLIST.md before contacting a core maintainer on Slack. Follow the template exactly, including As a PR reviewer and CODEOWNER, I have reviewed this and have, so sign-off verification triggers.
  • PR verification: Sweeps run only when the PR appends an inferencex-e2e/perf-changelog.yaml entry and carries exactly one primary label: full-sweep-fail-fast (strongly recommended; canary plus per-matrix fail-fast), full-sweep-enabled (canary; matrix jobs continue after a failure), or non-canary-full-sweep-enabled (no canary or fail-fast). The modifiers all-evals, evals-only, and agentx-fast require a primary label. On fork PRs, a maintainer applies the label. See sweep labels and reuse.
  • After merging: PR authors must ensure all GitHub Actions jobs pass. Transient failures often pass on rerun; see how to rerun failed jobs.
中文

感谢你的贡献!

  • **审阅:**如果 PR 修改的文件归属于仓库管理员及 @SemiAnalysisAI/core 之外的 CODEOWNER,请先联系一位有资格的 CODEOWNER 填写最新的 PR_REVIEW_CHECKLIST.md,再通过 Slack 联系核心维护者。必须严格遵循模板,并保留 As a PR reviewer and CODEOWNER, I have reviewed this and have,才能触发签核验证。
  • **PR 验证:**只有当 PR 在 inferencex-e2e/perf-changelog.yaml 末尾追加了条目,并且恰好带有一个主标签时,才会运行扫描:full-sweep-fail-fast(强烈推荐;canary 加逐矩阵 fail-fast)、full-sweep-enabled(有 canary;矩阵任务在失败后继续运行)或 non-canary-full-sweep-enabled(无 canary,也无 fail-fast)。修饰标签 all-evals、evals-only 和 agentx-fast 必须与主标签一起使用。fork PR 的标签由维护者添加。参见扫描标签与复用。
  • **合并后:**PR 作者必须确保所有 GitHub Actions 任务通过。临时性失败通常可以通过重新运行恢复;参见重新运行失败任务的说明。

@Oseltamivir Oseltamivir added the full-sweep-fail-fast Full sweep with canary gate; first failure cancels the rest of that matrix (recommended) label Oct 23, 2026

@claude claude Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Findings marked 🟡 are optional suggestions and need no follow-up push.

Comment thread inferencex-e2e/perf-changelog.yaml Outdated
- "Add day-zero MiniMax-M3.1-Flash (MiniMaxAI/MiniMax-M3.1-Flash, NVFP4 experts + NVFP4 KV) AgentX on GB200 and GB300 with vLLM image minimax-m31-flash-arm64-cu134 (https://vllm.ai/blog/2026-10-23-minimax-m31-flash)."
- "Serve flags follow the recipes.vllm.ai GB200 TP4 recipe: block size 128, nvfp4 KV and indexer cache, prefill query quantization, FULL_AND_PIECEWISE graphs, hybrid KV cache manager disabled, minimax_m3 reasoning and tool parsers, 1M context."
- "Search space: TP4 at conc 1-64 and DEP4 (TP1 x DP4 + EP4, vLLM Router) at conc 64-256, no KV offloading. DSpark is off until a golden AL curve exists."
pr-link: TBD

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🔴 The changelog check will fail on this PR and on main, so neither MiniMax-M3.1-Flash config key gets results collected. The new block uses pr-link: TBD. validate_added_pr_link accepts only the canonical PR URL or an XXX placeholder, and TBD is neither. Fix: set pr-link to https://github.com/SemiAnalysisAI/InferenceX/pull/<this PR number> or to the XXX placeholder, which merge_with_reuse rewrites to the canonical link.

Why this was flagged

The appended block at inferencex-e2e/perf-changelog.yaml:10882 sets pr-link: TBD. infx/workflows/validate_perf_changelog.py:139-150 (validate_added_pr_link) raises "new PR entry must use '…/pull/N' or an XXX placeholder; found 'TBD'" on PR runs. On main runs it raises "new main-branch entry has invalid pr-link" because TBD does not match CANONICAL_PR_LINK (line 21). PR_LINK_PLACEHOLDERS (lines 22-25) holds only XXX and https://github.com/SemiAnalysisAI/InferenceX/pull/XXX. So the changelog validation step fails, and the entry that is meant to trigger result collection for minimaxm31flash-fp4-gb200-vllm-agentic and minimaxm31flash-fp4-gb300-vllm-agentic never passes. The base branch has no such entry. The placeholder canonicalization in merge_with_reuse.py:478 only rewrites recognised placeholders, so it does not fix TBD either.

Verification: Trigger: the PR is merged through the documented merge_with_reuse tool. The new entry ends with pr-link: TBD (perf-changelog.yaml:10882), and validate_added_pr_link (validate_perf_changelog.py:139-150) rejects that value. merge_with_reuse.merge_pr calls canonicalize_changelog(pr, repo) (merge_with_reuse.py:631). As a result the standard merge-with-reuse flow aborts for this PR.

Comment on lines +8757 to +8761
# MiniMax-M3.1-Flash AgentX on GB200: day-zero vLLM image from
# https://vllm.ai/blog/2026-10-23-minimax-m31-flash, arm64 CUDA 13.4 build,
# verified 2026-10-23 (manifest sha256:86f582dae25ea92802353727164ae4eb03b37b18baf9119b805ffaec2a7ef07c).
# NVFP4 routed experts with MXFP8 dense layers and an NVFP4 KV cache. DSpark is
# off until a MiniMax-M3.1-Flash golden AL curve is committed.

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟡 nit (optional): New provenance/header comments break the AGENTS.md "No header comments" and "No history" rules, sweep:day-zero vLLM image from|verified 2026-10-23|Serve flags follow the GB200 TP4 recipe. The cited manifest sha256:86f582… also differs from the pinned digest sha256:5d76347…, which misleads readers. Fix: delete the blog/verified-date/manifest header above each master entry, and delete the "Serve flags follow…" lines in both recipes (gb200-fp4/agentic.yaml:26-28). Keep only per-line technical why-comments.

Why this was flagged

AGENTS.md:64-66 says comments may not sit above a recipe or section describing the config, and may not say where something was derived from. nvidia-master.yaml:8757-8761 and the matching gb300 header at about 8780-8784 describe the image source blog, the verification date and a manifest sha. The cited manifest sha (86f582…) does not match the image digest pinned at :8763 (5d76347…). Both recipes, at gb200-fp4/agentic.yaml:26-28 and gb300-fp4/agentic.yaml:26-28, say "Serve flags follow the GB200 TP4 recipe at https://recipes.vllm.ai/…", which is a provenance comment. There is no runtime impact. This is a convention breach that maintainers have to clean up.

Verification: nit: the rule is broken on every merge, but nothing breaks at runtime. The repo's CLAUDE.md says "All guidance ... lives in AGENTS.md", and AGENTS.md:60-66 "Code comments (mandatory)" applies to "every file, including recipes, configs". It says "No header comments ... above a recipe, override, or section. Do not describe what the file, topology, or config is" and "No history. Do not write comments that describe ... where it was derived from".

Comment on lines +8764 to +8770
model: MiniMaxAI/MiniMax-M3.1-Flash
model-prefix: minimaxm31flash
runner: cluster:gb200-nv
precision: fp4
framework: vllm
multinode: false
srt-recipe-dir: minimaxm31flash/vllm/gb200-fp4

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟡 (optional) MiniMax-M3 sweeps and eval campaigns will also pick up the two new MiniMax-M3.1-Flash configs. The full-sweep --model-prefix filter at infx/matrix/generate.py:1169 uses key.startswith(p), and the key minimaxm31flash-fp4-gb200-vllm-agentic starts with minimaxm3. Fix: have the filter compare against each config's own model-prefix field (exact match, or key.startswith(f"{p}-")), so minimaxm3, dsv4 and the other family names select only their own configs. The canonical full-sweep examples (--model-prefix dsr1) keep working either way.

Why this was flagged

An operator runs the documented MiniMax-M3 deployment-smoke command in infx/evals/EVALS.md:100-105 (full-sweep ... --model-prefix minimaxm3 --scenario-type agentic-coding --evals-only --all-evals), or any full-sweep --model-prefix minimaxm3. infx/matrix/generate.py:1169 keeps configs where any(key.startswith(p) for p in options.model_prefix). The new keys minimaxm31flash-fp4-gb200-vllm-agentic and minimaxm31flash-fp4-gb300-vllm-agentic (nvidia-master.yaml:8762, 8785) pass that test. The MiniMax-M3 matrix therefore gains GB200/GB300 MiniMax-M3.1-Flash rows. Those rows do not get the minimax_m3_full suite, because AUTOMATIC_AGENTIC_VENDOR_EVALS (generate.py:35-38) is keyed on the exact minimaxm3. On base the filter returns only minimaxm3 configs. docs/MODELS.md:133 calls the Prefix column the value the --model-prefix filter selects by. Nothing checks the model-prefix field itself. The existing dsv4/dsv41flash overlap shows this has gone unchecked; it is not a safeguard.

Verification: The trigger is any full-sweep --model-prefix minimaxm3 run. infx/matrix/generate.py:1169 keeps a config when any(key.startswith(p) for p in options.model_prefix). The two new keys at nvidia-master.yaml:8762 and 8785 both start with minimaxm3. After it, the MiniMax-M3 smoke matrix and any MiniMax-M3 full-sweep also schedule GB200 and GB300 MiniMax-M3.1-Flash jobs.

@Oseltamivir Oseltamivir changed the title [Klaud Cold] minimaxm31flash-fp4-gb{200,300}-vllm-agentic: day-zero MiniMax-M3.1-Flash GB200/GB300 recipe [Klaud Cold] minimaxm31flash-fp4-gb{200,300}-vllm-agentic-dspark: day-zero MiniMax-M3.1-Flash GB200/GB300 DSpark recipe Oct 23, 2026
@github-actions

github-actions Bot commented Oct 23, 2026 •

Copy link
Copy Markdown
Contributor

@Oseltamivir Oseltamivir removed the full-sweep-fail-fast Full sweep with canary gate; first failure cancels the rest of that matrix (recommended) label Oct 23, 2026
@Oseltamivir Oseltamivir changed the title [Klaud Cold] minimaxm31flash-fp4-gb{200,300}-vllm-agentic-dspark: day-zero MiniMax-M3.1-Flash GB200/GB300 DSpark recipe [Klaud Cold] minimaxm31flash-fp4-gb300-vllm-agentic-dspark: day-zero MiniMax-M3.1-Flash GB300 DSpark recipe Oct 23, 2026
@Oseltamivir Oseltamivir added full-sweep-fail-fast Full sweep with canary gate; first failure cancels the rest of that matrix (recommended) and removed full-sweep-fail-fast Full sweep with canary gate; first failure cancels the rest of that matrix (recommended) labels Oct 23, 2026
@Oseltamivir

Copy link
Copy Markdown
Collaborator Author

/use 39949684795

@github-actions

github-actions Bot commented Oct 23, 2026 •

Copy link
Copy Markdown
Contributor

@Oseltamivir staged run 39949684795.

This run remains available across future /stage-results requests. Staging the same run ID again updates its staged data. Staging workflow

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

full-sweep-fail-fast Full sweep with canary gate; first failure cancels the rest of that matrix (recommended)

Projects

Status: Done

Development

Successfully merging this pull request may close these issues.

1 participant