Skip to content

[Klaud Cold] minimaxm31flash-fp4-gb200-vllm-agentic-dspark: day-zero MiniMax-M3.1-Flash GB200 DSpark recipe - #4424

Closed
Oseltamivir wants to merge 3 commits into
mainfrom
feat/minimaxm31flash-gb200-dayzero
Closed

Oseltamivir wants to merge 3 commits into
mainfrom
feat/minimaxm31flash-gb200-dayzero

Conversation

@Oseltamivir

Copy link
Copy Markdown
Collaborator

Summary

Day-zero MiniMax-M3.1-Flash AgentX on GB200 with vLLM and DSpark, per the recipes.vllm.ai GB200 command and the vLLM launch blog. This was split out of #4419, which now covers GB300 only.

  • New model prefix minimaxm31flash. It uses the uncapped 1M-context trace corpus.
  • Config key: minimaxm31flash-fp4-gb200-vllm-agentic-dspark.
  • Image: vllm/vllm-openai:minimax-m31-flash-arm64-cu134, pinned to digest sha256:5d76347…. The tag was published 2026-10-23 and verified on Docker Hub.
  • Serve flags:
    • --block-size 128
    • --kv-cache-dtype nvfp4
    • attention-config {"indexer_kv_dtype":"nvfp4","use_prefill_query_quantization":true}
    • FULL_AND_PIECEWISE CUDA graphs
    • --disable-hybrid-kv-cache-manager
    • minimax_m3 reasoning and tool parsers
    • VLLM_FLOAT32_MATMUL_PRECISION=high
    • --language-model-only
    • 1M context
  • DSpark: 3 draft tokens from the checkpoint's own dspark/ head, with the FLASH_ATTN drafter, probabilistic drafting and block verification.
  • Search space:
    • TP4 at conc 1/4/8/16/32/64.
    • DEP4 (TP1 × DP4 + EP4, vLLM Router) at conc 64/128/256.
    • No KV offloading.

Depends on the golden AL curve

Throughput runs need infx/golden_al_distribution/minimaxm31flash_dspark.yaml with a thinking_on cell for 3 draft tokens. The curve is being collected separately. Until it lands, recipe generation fails with No committed golden curve.

Validation

  • infx generate with a temporary placeholder curve binds all 9 points to exactly one variant each.
  • srtctl accepts every bound recipe.

@github-actions

Copy link
Copy Markdown
Contributor

Thanks for the contribution!

  • Review: If this PR changes files owned by someone other than a repository admin or @SemiAnalysisAI/core, ask one eligible CODEOWNER to complete the latest PR_REVIEW_CHECKLIST.md before contacting a core maintainer on Slack. Follow the template exactly, including As a PR reviewer and CODEOWNER, I have reviewed this and have, so sign-off verification triggers.
  • PR verification: Sweeps run only when the PR appends an inferencex-e2e/perf-changelog.yaml entry and carries exactly one primary label: full-sweep-fail-fast (strongly recommended; canary plus per-matrix fail-fast), full-sweep-enabled (canary; matrix jobs continue after a failure), or non-canary-full-sweep-enabled (no canary or fail-fast). The modifiers all-evals, evals-only, and agentx-fast require a primary label. On fork PRs, a maintainer applies the label. See sweep labels and reuse.
  • After merging: PR authors must ensure all GitHub Actions jobs pass. Transient failures often pass on rerun; see how to rerun failed jobs.
中文

感谢你的贡献!

  • **审阅:**如果 PR 修改的文件归属于仓库管理员及 @SemiAnalysisAI/core 之外的 CODEOWNER,请先联系一位有资格的 CODEOWNER 填写最新的 PR_REVIEW_CHECKLIST.md,再通过 Slack 联系核心维护者。必须严格遵循模板,并保留 As a PR reviewer and CODEOWNER, I have reviewed this and have,才能触发签核验证。
  • **PR 验证:**只有当 PR 在 inferencex-e2e/perf-changelog.yaml 末尾追加了条目,并且恰好带有一个主标签时,才会运行扫描:full-sweep-fail-fast(强烈推荐;canary 加逐矩阵 fail-fast)、full-sweep-enabled(有 canary;矩阵任务在失败后继续运行)或 non-canary-full-sweep-enabled(无 canary,也无 fail-fast)。修饰标签 all-evals、evals-only 和 agentx-fast 必须与主标签一起使用。fork PR 的标签由维护者添加。参见扫描标签与复用。
  • **合并后:**PR 作者必须确保所有 GitHub Actions 任务通过。临时性失败通常可以通过重新运行恢复;参见重新运行失败任务的说明。

@claude claude Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Findings marked 🟡 are optional suggestions and need no follow-up push.

- { tp: 4, kv-offloading: none, spec-decoding: mtp, conc-list: [1, 4, 8, 16, 32, 64], srt-recipe: agentic.yaml }
# DEP4 is native TP1 x DP4 + EP4 behind a consistent-hash vLLM Router.
- { tp: 4, ep: 4, dp-attn: true, kv-offloading: none, spec-decoding: mtp, conc-list: [64, 128, 256], router: { name: vllm-router, version: "0.1.14" }, srt-recipe: agentic.yaml }

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🔴 Once merged, every throughput point of the new config fails before benchmarking with No committed golden curve minimaxm31flash_dspark.yaml. The PR description says the curve is collected separately. The new changelog entry still triggers result collection on merge. Fix: add infx/golden_al_distribution/minimaxm31flash_dspark.yaml with a thinking_on value for 3 draft tokens in this PR, or hold the config and changelog entry until that curve lands. [also at: inferencex-e2e/benchmarks/single_node/srt-slurm-recipes/minimaxm31flash/vllm/gb200-fp4-mtp/agentic.yaml:43 - The changelog entry starts a sweep whose recipe generation fails at every point, because the golden curve this config needs is not in the repo.]

Why this was flagged

The new entry sets model-prefix: minimaxm31flash, and the recipe's speculative-config has "method":"dspark","num_speculative_tokens":3. For throughput runs, infx/srt_slurm/synthetic_acceptance.py:131 calls golden_length(MODEL_PREFIX, spec, ...). curve_name (infx/golden_al_distribution/curves.py:47) resolves this to minimaxm31flash_dspark. load_curve (curves.py:67-68) then raises No committed golden curve minimaxm31flash_dspark.yaml because no such file exists in infx/golden_al_distribution/. The perf-changelog block appended by this PR triggers collection for minimaxm31flash-fp4-gb200-vllm-agentic-dspark, so all 9 search-space points (TP4 conc 1-64, DEP4 conc 64-256) error out instead of producing results. The PR description admits generation fails until the curve lands, but nothing in the change gates the config or the entry on it.

Verification: After merge, any agentic throughput run of the new config key hits the fault. The recipe (agentic.yaml:43) uses "method":"dspark", so curve_name (curves.py:41-62) returns minimaxm31flash_dspark. load_curve (curves.py:66-68) then raises No committed golden curve minimaxm31flash_dspark.yaml, because the diff adds no such file. All 9 search-space points fail at recipe generation.

Comment thread inferencex-e2e/perf-changelog.yaml Outdated
- "Serve flags follow the recipes.vllm.ai GB200 TP4 recipe: block size 128, nvfp4 KV and indexer cache, prefill query quantization, FULL_AND_PIECEWISE graphs, hybrid KV cache manager disabled, minimax_m3 reasoning and tool parsers, 1M context."
- "Search space: TP4 at conc 1-64 and DEP4 (TP1 x DP4 + EP4, vLLM Router) at conc 64-256, no KV offloading."
- "Three-token DSpark from the checkpoint's dspark/ head with the FLASH_ATTN drafter, probabilistic drafting and block verification (recipes.vllm.ai GB200 command). Throughput runs use the minimaxm31flash_dspark golden AL curve."
pr-link: TBD

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🔴 The PR's perf-changelog check will reject this entry, and the main-branch check will reject it after merge. pr-link: TBD is not an accepted placeholder. validate_added_pr_link only accepts XXX, https://github.com/SemiAnalysisAI/InferenceX/pull/XXX or the PR's own canonical URL. Fix: set pr-link to https://github.com/SemiAnalysisAI/InferenceX/pull/XXX, or to the real PR URL, so the merge tooling can canonicalize it.

Why this was flagged

The new block ends with pr-link: TBD at inferencex-e2e/perf-changelog.yaml:10882. On PR runs, validate_added_pr_link (inferencex-e2e/infx/workflows/validate_perf_changelog.py:146-150) raises "new PR entry must use ... or an XXX placeholder; found 'TBD'". That is because PR_LINK_PLACEHOLDERS (line 22) contains only "XXX" and the XXX pull URL. On main (pr_number None), line 142 requires CANONICAL_PR_LINK and raises "new main-branch entry has invalid pr-link". No other entry in the file uses TBD, so this is the first one, and changelog validation fails where the base passes.

Verification: The trigger is that this PR goes through the supported reuse merge path, python -m infx.workflows.merge_with_reuse <pr>, while the new entry still has pr-link: TBD (inferencex-e2e/perf-changelog.yaml:10882). validate_added_pr_link (validate_perf_changelog.py:139-150) only accepts PR_LINK_PLACEHOLDERS or the PR's own URL, so "TBD" raises. The real call path is merge_with_reuse.py:631.

Comment on lines +8600 to +8604
# MiniMax-M3.1-Flash AgentX on GB200: day-zero vLLM image from
# https://vllm.ai/blog/2026-10-23-minimax-m31-flash, arm64 CUDA 13.4 build,
# verified 2026-10-23 (manifest sha256:86f582dae25ea92802353727164ae4eb03b37b18baf9119b805ffaec2a7ef07c).
# NVFP4 routed experts with MXFP8 dense layers and an NVFP4 KV cache, with
# DSpark from the checkpoint's dspark/ head (golden AL minimaxm31flash_dspark).

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟡 nit (optional): Maintainers get provenance and header comments that AGENTS.md "Code comments (mandatory)" forbids. One of them gives a digest that does not match the pinned image. The header above the new master entry cites manifest sha256:86f582da..., while image: pins sha256:5d763476.... Fix: remove the provenance and header comments, keeping only short technical line comments. This covers the 4 sites listed. Same pattern at 4 sites (nvidia-master.yaml:8600-8604, agentic.yaml:25-26, agentic.yaml:40-42, agentic.yaml:59).

Why this was flagged

AGENTS.md says there must be no header comments above a recipe, override or section. It also bans comments saying where something was derived from. The added block at nvidia-master.yaml:8600-8604 describes the entry and records its source blog, verification date and a manifest digest. That digest (86f5...) differs from the digest pinned at line 8606 (5d76...), so a reader cannot tell which image was verified. agentic.yaml:25-26 ("Serve flags follow the GB200 TP4 recipe at ...") and agentic.yaml:40-42 ("per the recipes.vllm.ai GB200 TP4 command") are provenance comments. agentic.yaml:59 ("TP4 is the upstream Blackwell layout...") is a header above an override. None of this changes runtime behaviour.

Verification: AGENTS.md:60-66, "Code comments (mandatory)", applies to "every file, including recipes, configs" and bans header comments above a recipe, override, or section and comments on where config was derived from. The nvidia-master.yaml header and the agentic.yaml comments break it. The comment cites sha256:86f582da..., while image: pins sha256:5d76347628973fb8....

@Oseltamivir Oseltamivir added the full-sweep-fail-fast Full sweep with canary gate; first failure cancels the rest of that matrix (recommended) label Oct 23, 2026
@github-actions

Copy link
Copy Markdown
Contributor

@Oseltamivir

Copy link
Copy Markdown
Collaborator Author

/use 39903544156

@github-actions

Copy link
Copy Markdown
Contributor

@Oseltamivir staging run 39903544156. Existing staged runs will be preserved; a completion link will be posted here.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

full-sweep-fail-fast Full sweep with canary gate; first failure cancels the rest of that matrix (recommended)

Projects

Status: Done

Development

Successfully merging this pull request may close these issues.

1 participant