Skip to content

[Klaud Cold] Add day-0 MiniMax-M3.1-Flash B300 vLLM AgentX DSpark recipe / [Klaud Cold] 新增 MiniMax-M3.1-Flash B300 vLLM AgentX DSpark 首日配方 - #4421

Merged
Oseltamivir merged 8 commits into
mainfrom
feat/minimaxm31flash-b200-b300-dayzero
Oct 25, 2026
Merged

Oseltamivir merged 8 commits into
mainfrom
feat/minimaxm31flash-b200-b300-dayzero

Conversation

@Oseltamivir

@Oseltamivir Oseltamivir commented Oct 23, 2026 •

Copy link
Copy Markdown
Collaborator

Summary

Day-0 MiniMax-M3.1-Flash (MiniMaxAI/MiniMax-M3.1-Flash, ~428B/23B active, MXFP8 dense + NVFP4 experts, QAT for an NVFP4 KV cache) AgentX recipe on B300 with DSpark, following the vLLM day-0 blog and recipes.vllm.ai. B200 is in a separate PR.

  • Key: minimaxm31flash-fp4-b300-vllm-agentic-dspark, recipe minimaxm31flash/vllm/b300-fp4-mtp/agentic.yaml.
  • Image: vllm/vllm-openai:minimax-m31-flash@sha256:86f582da… (day-0 multi-arch build, verified on Docker Hub).
  • Serve flags (Blackwell TP4 recipe): block-size 128, kv-cache-dtype nvfp4, attention-config {"indexer_kv_dtype":"nvfp4","use_prefill_query_quantization":true}, FULL_AND_PIECEWISE cudagraphs, disable-hybrid-kv-cache-manager, minimax_m3 reasoning/tool parsers, language-model-only, VLLM_FLOAT32_MATMUL_PRECISION=high.
  • DSpark with the checkpoint's built-in drafter: {"method":"dspark","attention_backend":"FLASH_ATTN","kv_cache_dtype":"fp8","draft_sample_method":"probabilistic","rejection_sample_method":"block"}. Throughput runs use synthetic acceptance from the golden curve minimaxm31flash_dspark.yaml ([Klaud Cold] Add MiniMax-M3.1-Flash DSpark golden AL curve and SPEED-Bench collector / [Klaud Cold] 新增 MiniMax-M3.1-Flash DSpark 黄金 AL 曲线与 SPEED-Bench 采集脚本 #4425).
Arm Concurrency DSpark tokens Golden AL
TP4 1, 4, 8, 16, 32 7 (vLLM blog GB300 AgentX frontier) 4.04
DEP4 (TP1 × DP4 + EP4, vllm-router consistent-hash) 64, 128, 256 3 (recipes.vllm.ai Blackwell) 2.88
  • Adds minimaxm31flash to docs/MODELS.md.

… vLLM AgentX recipes

TP4 + DEP4 aggregate recipes on the vllm/vllm-openai:minimax-m31-flash
day-0 image, serve flags from recipes.vllm.ai Blackwell TP4 (NVFP4 KV and
indexer cache, block-size 128). DSpark stays off until a golden AL curve
is committed for this model.
@Oseltamivir Oseltamivir added the full-sweep-fail-fast Full sweep with canary gate; first failure cancels the rest of that matrix (recommended) label Oct 23, 2026
@github-actions

Copy link
Copy Markdown
Contributor

Thanks for the contribution!

  • Review: If this PR changes files owned by someone other than a repository admin or @SemiAnalysisAI/core, ask one eligible CODEOWNER to complete the latest PR_REVIEW_CHECKLIST.md before contacting a core maintainer on Slack. Follow the template exactly, including As a PR reviewer and CODEOWNER, I have reviewed this and have, so sign-off verification triggers.
  • PR verification: Sweeps run only when the PR appends an inferencex-e2e/perf-changelog.yaml entry and carries exactly one primary label: full-sweep-fail-fast (strongly recommended; canary plus per-matrix fail-fast), full-sweep-enabled (canary; matrix jobs continue after a failure), or non-canary-full-sweep-enabled (no canary or fail-fast). The modifiers all-evals, evals-only, and agentx-fast require a primary label. On fork PRs, a maintainer applies the label. See sweep labels and reuse.
  • After merging: PR authors must ensure all GitHub Actions jobs pass. Transient failures often pass on rerun; see how to rerun failed jobs.
中文

感谢你的贡献!

  • **审阅:**如果 PR 修改的文件归属于仓库管理员及 @SemiAnalysisAI/core 之外的 CODEOWNER,请先联系一位有资格的 CODEOWNER 填写最新的 PR_REVIEW_CHECKLIST.md,再通过 Slack 联系核心维护者。必须严格遵循模板,并保留 As a PR reviewer and CODEOWNER, I have reviewed this and have,才能触发签核验证。
  • **PR 验证:**只有当 PR 在 inferencex-e2e/perf-changelog.yaml 末尾追加了条目,并且恰好带有一个主标签时,才会运行扫描:full-sweep-fail-fast(强烈推荐;canary 加逐矩阵 fail-fast)、full-sweep-enabled(有 canary;矩阵任务在失败后继续运行)或 non-canary-full-sweep-enabled(无 canary,也无 fail-fast)。修饰标签 all-evals、evals-only 和 agentx-fast 必须与主标签一起使用。fork PR 的标签由维护者添加。参见扫描标签与复用。
  • **合并后:**PR 作者必须确保所有 GitHub Actions 任务通过。临时性失败通常可以通过重新运行恢复;参见重新运行失败任务的说明。

@Oseltamivir Oseltamivir removed the full-sweep-fail-fast Full sweep with canary gate; first failure cancels the rest of that matrix (recommended) label Oct 23, 2026
@github-actions

github-actions Bot commented Oct 23, 2026 •

Copy link
Copy Markdown
Contributor

Built-in DSpark drafter, levels 1-7, thinking_on (adaptive template),
recipes.vllm.ai Blackwell TP4 serve flags.

@claude claude Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Findings marked 🟡 are optional suggestions and need no follow-up push.

Comment thread inferencex-e2e/perf-changelog.yaml Outdated
- "Add day-0 MiniMax-M3.1-Flash (MiniMaxAI/MiniMax-M3.1-Flash, NVFP4 experts + NVFP4 KV cache) AgentX recipes on B300 and B200 with the vLLM day-0 image vllm/vllm-openai:minimax-m31-flash."
- "Serve flags follow recipes.vllm.ai Blackwell TP4: block-size 128, nvfp4 KV and indexer cache, prefill query quantization, FULL_AND_PIECEWISE graphs, minimax_m3 parsers, text-only."
- "TP4 c1-c32 and DEP4 (TP1 x DP4 + EP4 behind vllm-router) c64-c256 on B300, c64-c128 on B200. DSpark is off until a golden AL curve exists for this model."
pr-link: TBD

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🔴 The changelog check will fail this PR, and merge tooling will refuse it, because the new block's pr-link: TBD is not an accepted placeholder. validate_added_pr_link accepts only XXX, https://github.com/SemiAnalysisAI/InferenceX/pull/XXX, or the real PR URL. prepare_perf_changelog_merge raises "appended entry ... has unexpected pr-link" for any other value. Fix: set the line to pr-link: https://github.com/SemiAnalysisAI/InferenceX/pull/XXX or to this PR's canonical URL.

Why this was flagged

The appended block at inferencex-e2e/perf-changelog.yaml:10883 sets pr-link: TBD. In inferencex-e2e/infx/workflows/validate_perf_changelog.py:22-25, PR_LINK_PLACEHOLDERS holds only "XXX" and the /pull/XXX URL. On PR runs, validate_added_pr_link (lines 146-150) raises ChangelogValidationError "new PR entry must use ... or an XXX placeholder; found 'TBD'". On main runs it needs CANONICAL_PR_LINK instead. In inferencex-e2e/infx/workflows/prepare_perf_changelog_merge.py:105-111, a link not in PR_LINK_PLACEHOLDERS also raises "appended entry N has unexpected pr-link 'TBD'" instead of being rewritten to the PR URL. The changelog entry is what triggers result collection, so the two new config keys never get swept or merged through the normal path. The base branch has no such entry.

Verification: The appended block ends with pr-link: TBD (perf-changelog.yaml:10883). In inferencex-e2e/infx/workflows/validate_perf_changelog.py, on a PR run, validate_added_pr_link raises new PR entry must use ... or an XXX placeholder; found 'TBD'. Result: the changelog check fails and merge tooling refuses the PR until the link is changed to the /pull/XXX form or the real PR URL.

Comment on lines +87 to +89
benchmark:
env:
CONC: ['64', '128', '256']

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🔴 DEP4 points on B300 and B200 may lose per-conversation rank affinity, so later turns land cold on other DP ranks and DEP4 throughput reads low. Both zip_override_dep4 blocks run a consistent_hash vllm-router but never set AIPERF_HTTP_X_SESSION_ID_FROM_CORRELATION_ID. Without it AIPerf sends no X-Session-ID header for the router to hash on. Fix: in both DEP4 overrides, add AIPERF_HTTP_X_SESSION_ID_FROM_CORRELATION_ID: '1' under benchmark.env, as the dsv4 vLLM DEP arms do. That keeps every turn of a conversation on the rank that holds its prefix cache.

Why this was flagged

The DEP4 matrix points (b300 conc 64/128/256, b200 conc 64/128) select zip_override_dep4. That block puts four DP ranks behind vllm-router with policy: consistent_hash. Its benchmark.env sets only CONC (b300-fp4/agentic.yaml:87-89, b200-fp4/agentic.yaml:88-90). The dsv4 vLLM DEP arms use the same router and policy, and they also set AIPERF_HTTP_X_SESSION_ID_FROM_CORRELATION_ID: '1' (dsv4/vllm/b300-fp4-mtp/agentic.yaml:135). perf-changelog.yaml:6598 notes that without affinity, turns scatter across DP ranks and the multi-turn workload collapses to cold prefill. The trigger is that vllm-router 0.1.14's consistent hash keys on X-Session-ID or X-User-ID and not on X-Correlation-ID. The dsv41flash vLLM DEP arms also leave the flag out, so the repo is split and nothing guards this. The new model has no earlier DEP4 numbers, so this is new coverage with possibly understated results, not a regression.

Verification: Trigger: every DEP4 point, as long as the vllm-router consistent_hash policy keys on X-Session-ID. In b300-fp4/agentic.yaml, zip_override_dep4 sets policy: consistent_hash (line 73). Its benchmark.env sets only CONC: ['64', '128', '256'] (lines 87-89). The B200 recipe has the same shape (consistent_hash at line 74, CONC only).

- { tp: 8, spec-decoding: mtp, kv-offloading: dram, kv-offload-backend: { name: native }, conc-list: [1, 5], srt-recipe: agentic.yaml }

minimaxm31flash-fp4-b300-vllm-agentic:
# Day-0 vLLM image for MiniMax-M3.1-Flash (vllm.ai/blog/2026-10-23-minimax-m31-flash).

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟡 nit (optional): several new comments record provenance or upstream sources, which the AGENTS.md comment rules forbid. sweep:Day-0 vLLM image|Serve flags follow the Blackwell|upstream Blackwell layout Examples: the "Day-0 vLLM image for MiniMax-M3.1-Flash (vllm.ai/blog/...)" line above image: here, and "Serve flags follow the Blackwell TP4 recipe at https://recipes.vllm.ai/..." above args: in the b300-fp4 and b200-fp4 recipes. Fix: delete these lines, or replace each with one short technical line saying why the value was chosen (for example, keep only the DSpark/golden-AL reason).

Why this was flagged

AGENTS.md "Code comments (mandatory)" applies to every file, including recipes and configs. It bans comments about where something was derived from ("No history") and comments describing the topology or config ("No header comments"). The diff adds comments that cite a blog post and an upstream recipe as the source of the image and serve flags (nvidia-master.yaml:5553 and :5571, and agentic.yaml lines 26-28 in both recipes). It also adds "TP4 is the upstream Blackwell layout" (master :5565/:5583, recipes :56), which states provenance rather than a technical reason. Maintainers inherit comments that go stale when the upstream recipe or blog changes. The base branch has none of these comments.

Verification: AGENTS.md:60-66 says the rule "Applies to every file, including recipes, configs". It bans comments about "where it was derived from" (No history) and comments that "describe what the file, topology, or config is" (No header comments). The diff adds comments of both kinds, such as the "Day-0 vLLM image" line at nvidia-master.yaml:5553. Nothing at runtime breaks.

Give b200-nscale a writable download root so script runs can fetch
unstaged checkpoints.
…k on the golden AL curve

Split B200 into its own PR. TP4 drafts 7 DSpark tokens (vLLM day-0 GB300
AgentX frontier), DEP4 drafts 3 (recipes.vllm.ai Blackwell); throughput
runs use the minimaxm31flash_dspark golden AL from #4425.
@Oseltamivir Oseltamivir changed the title [Klaud Cold] Add day-0 MiniMax-M3.1-Flash B200/B300 vLLM AgentX recipes / [Klaud Cold] 新增 MiniMax-M3.1-Flash B200/B300 vLLM AgentX 首日配方 [Klaud Cold] Add day-0 MiniMax-M3.1-Flash B300 vLLM AgentX DSpark recipe / [Klaud Cold] 新增 MiniMax-M3.1-Flash B300 vLLM AgentX DSpark 首日配方 Oct 23, 2026
@Oseltamivir Oseltamivir added the full-sweep-fail-fast Full sweep with canary gate; first failure cancels the rest of that matrix (recommended) label Oct 23, 2026
@Oseltamivir

Copy link
Copy Markdown
Collaborator Author

/use 39904040374

@github-actions

github-actions Bot commented Oct 25, 2026 •

Copy link
Copy Markdown
Contributor

@Oseltamivir staged run 39904040374.

This run remains available across future /stage-results requests. Staging the same run ID again updates its staged data. Staging workflow

@Oseltamivir
Oseltamivir merged commit e6cf37f into main Oct 25, 2026
26 checks passed
@Oseltamivir
Oseltamivir deleted the feat/minimaxm31flash-b200-b300-dayzero branch October 25, 2026 07:46
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

full-sweep-fail-fast Full sweep with canary gate; first failure cancels the rest of that matrix (recommended)

Projects

Status: Done

Development

Successfully merging this pull request may close these issues.

1 participant