Repository navigation
[Klaud Cold] minimaxm31flash-fp4-gb200-vllm-agentic-dspark: day-zero MiniMax-M3.1-Flash GB200 DSpark recipe - #4424
Oseltamivir wants to merge 3 commits into
Conversation
…Flash AgentX on GB200
|
Thanks for the contribution!
中文感谢你的贡献!
|
| - { tp: 4, kv-offloading: none, spec-decoding: mtp, conc-list: [1, 4, 8, 16, 32, 64], srt-recipe: agentic.yaml } | ||
| # DEP4 is native TP1 x DP4 + EP4 behind a consistent-hash vLLM Router. | ||
| - { tp: 4, ep: 4, dp-attn: true, kv-offloading: none, spec-decoding: mtp, conc-list: [64, 128, 256], router: { name: vllm-router, version: "0.1.14" }, srt-recipe: agentic.yaml } | ||
|
|
There was a problem hiding this comment.
🔴 Once merged, every throughput point of the new config fails before benchmarking with No committed golden curve minimaxm31flash_dspark.yaml. The PR description says the curve is collected separately. The new changelog entry still triggers result collection on merge. Fix: add infx/golden_al_distribution/minimaxm31flash_dspark.yaml with a thinking_on value for 3 draft tokens in this PR, or hold the config and changelog entry until that curve lands. [also at: inferencex-e2e/benchmarks/single_node/srt-slurm-recipes/minimaxm31flash/vllm/gb200-fp4-mtp/agentic.yaml:43 - The changelog entry starts a sweep whose recipe generation fails at every point, because the golden curve this config needs is not in the repo.]
Why this was flagged
The new entry sets model-prefix: minimaxm31flash, and the recipe's speculative-config has "method":"dspark","num_speculative_tokens":3. For throughput runs, infx/srt_slurm/synthetic_acceptance.py:131 calls golden_length(MODEL_PREFIX, spec, ...). curve_name (infx/golden_al_distribution/curves.py:47) resolves this to minimaxm31flash_dspark. load_curve (curves.py:67-68) then raises No committed golden curve minimaxm31flash_dspark.yaml because no such file exists in infx/golden_al_distribution/. The perf-changelog block appended by this PR triggers collection for minimaxm31flash-fp4-gb200-vllm-agentic-dspark, so all 9 search-space points (TP4 conc 1-64, DEP4 conc 64-256) error out instead of producing results. The PR description admits generation fails until the curve lands, but nothing in the change gates the config or the entry on it.
Verification: After merge, any agentic throughput run of the new config key hits the fault. The recipe (agentic.yaml:43) uses "method":"dspark", so curve_name (curves.py:41-62) returns minimaxm31flash_dspark. load_curve (curves.py:66-68) then raises No committed golden curve minimaxm31flash_dspark.yaml, because the diff adds no such file. All 9 search-space points fail at recipe generation.
| - "Serve flags follow the recipes.vllm.ai GB200 TP4 recipe: block size 128, nvfp4 KV and indexer cache, prefill query quantization, FULL_AND_PIECEWISE graphs, hybrid KV cache manager disabled, minimax_m3 reasoning and tool parsers, 1M context." | ||
| - "Search space: TP4 at conc 1-64 and DEP4 (TP1 x DP4 + EP4, vLLM Router) at conc 64-256, no KV offloading." | ||
| - "Three-token DSpark from the checkpoint's dspark/ head with the FLASH_ATTN drafter, probabilistic drafting and block verification (recipes.vllm.ai GB200 command). Throughput runs use the minimaxm31flash_dspark golden AL curve." | ||
| pr-link: TBD |
There was a problem hiding this comment.
🔴 The PR's perf-changelog check will reject this entry, and the main-branch check will reject it after merge. pr-link: TBD is not an accepted placeholder. validate_added_pr_link only accepts XXX, https://github.com/SemiAnalysisAI/InferenceX/pull/XXX or the PR's own canonical URL. Fix: set pr-link to https://github.com/SemiAnalysisAI/InferenceX/pull/XXX, or to the real PR URL, so the merge tooling can canonicalize it.
Why this was flagged
The new block ends with pr-link: TBD at inferencex-e2e/perf-changelog.yaml:10882. On PR runs, validate_added_pr_link (inferencex-e2e/infx/workflows/validate_perf_changelog.py:146-150) raises "new PR entry must use ... or an XXX placeholder; found 'TBD'". That is because PR_LINK_PLACEHOLDERS (line 22) contains only "XXX" and the XXX pull URL. On main (pr_number None), line 142 requires CANONICAL_PR_LINK and raises "new main-branch entry has invalid pr-link". No other entry in the file uses TBD, so this is the first one, and changelog validation fails where the base passes.
Verification: The trigger is that this PR goes through the supported reuse merge path, python -m infx.workflows.merge_with_reuse <pr>, while the new entry still has pr-link: TBD (inferencex-e2e/perf-changelog.yaml:10882). validate_added_pr_link (validate_perf_changelog.py:139-150) only accepts PR_LINK_PLACEHOLDERS or the PR's own URL, so "TBD" raises. The real call path is merge_with_reuse.py:631.
| # MiniMax-M3.1-Flash AgentX on GB200: day-zero vLLM image from | ||
| # https://vllm.ai/blog/2026-10-23-minimax-m31-flash, arm64 CUDA 13.4 build, | ||
| # verified 2026-10-23 (manifest sha256:86f582dae25ea92802353727164ae4eb03b37b18baf9119b805ffaec2a7ef07c). | ||
| # NVFP4 routed experts with MXFP8 dense layers and an NVFP4 KV cache, with | ||
| # DSpark from the checkpoint's dspark/ head (golden AL minimaxm31flash_dspark). |
There was a problem hiding this comment.
🟡 nit (optional): Maintainers get provenance and header comments that AGENTS.md "Code comments (mandatory)" forbids. One of them gives a digest that does not match the pinned image. The header above the new master entry cites manifest sha256:86f582da..., while image: pins sha256:5d763476.... Fix: remove the provenance and header comments, keeping only short technical line comments. This covers the 4 sites listed. Same pattern at 4 sites (nvidia-master.yaml:8600-8604, agentic.yaml:25-26, agentic.yaml:40-42, agentic.yaml:59).
Why this was flagged
AGENTS.md says there must be no header comments above a recipe, override or section. It also bans comments saying where something was derived from. The added block at nvidia-master.yaml:8600-8604 describes the entry and records its source blog, verification date and a manifest digest. That digest (86f5...) differs from the digest pinned at line 8606 (5d76...), so a reader cannot tell which image was verified. agentic.yaml:25-26 ("Serve flags follow the GB200 TP4 recipe at ...") and agentic.yaml:40-42 ("per the recipes.vllm.ai GB200 TP4 command") are provenance comments. agentic.yaml:59 ("TP4 is the upstream Blackwell layout...") is a header above an override. None of this changes runtime behaviour.
Verification: AGENTS.md:60-66, "Code comments (mandatory)", applies to "every file, including recipes, configs" and bans header comments above a recipe, override, or section and comments on where config was derived from. The nvidia-master.yaml header and the agentic.yaml comments break it. The comment cites sha256:86f582da..., while image: pins sha256:5d76347628973fb8....
|
View unofficial run (performance): https://inferencex.semianalysis.com/inference?unofficialRun=39903544156 View unofficial run (accuracy): https://inferencex.semianalysis.com/evaluation?unofficialRun=39903544156 |
|
/use 39903544156 |
|
@Oseltamivir staging run 39903544156. Existing staged runs will be preserved; a completion link will be posted here. |
Summary
Day-zero MiniMax-M3.1-Flash AgentX on GB200 with vLLM and DSpark, per the recipes.vllm.ai GB200 command and the vLLM launch blog. This was split out of #4419, which now covers GB300 only.
minimaxm31flash. It uses the uncapped 1M-context trace corpus.minimaxm31flash-fp4-gb200-vllm-agentic-dspark.vllm/vllm-openai:minimax-m31-flash-arm64-cu134, pinned to digestsha256:5d76347…. The tag was published 2026-10-23 and verified on Docker Hub.--block-size 128--kv-cache-dtype nvfp4attention-config {"indexer_kv_dtype":"nvfp4","use_prefill_query_quantization":true}--disable-hybrid-kv-cache-managerminimax_m3reasoning and tool parsersVLLM_FLOAT32_MATMUL_PRECISION=high--language-model-onlydspark/head, with theFLASH_ATTNdrafter, probabilistic drafting and block verification.Depends on the golden AL curve
Throughput runs need
infx/golden_al_distribution/minimaxm31flash_dspark.yamlwith athinking_oncell for 3 draft tokens. The curve is being collected separately. Until it lands, recipe generation fails withNo committed golden curve.Validation
infx generatewith a temporary placeholder curve binds all 9 points to exactly one variant each.