Repository navigation
[Klaud Cold] minimaxm31flash-fp4-gb300-vllm-agentic-dspark: day-zero MiniMax-M3.1-Flash GB300 DSpark recipe - #4419
Conversation
…lash AgentX on GB200/GB300
|
Thanks for the contribution!
中文感谢你的贡献!
|
| - "Add day-zero MiniMax-M3.1-Flash (MiniMaxAI/MiniMax-M3.1-Flash, NVFP4 experts + NVFP4 KV) AgentX on GB200 and GB300 with vLLM image minimax-m31-flash-arm64-cu134 (https://vllm.ai/blog/2026-10-23-minimax-m31-flash)." | ||
| - "Serve flags follow the recipes.vllm.ai GB200 TP4 recipe: block size 128, nvfp4 KV and indexer cache, prefill query quantization, FULL_AND_PIECEWISE graphs, hybrid KV cache manager disabled, minimax_m3 reasoning and tool parsers, 1M context." | ||
| - "Search space: TP4 at conc 1-64 and DEP4 (TP1 x DP4 + EP4, vLLM Router) at conc 64-256, no KV offloading. DSpark is off until a golden AL curve exists." | ||
| pr-link: TBD |
There was a problem hiding this comment.
🔴 The changelog check will fail on this PR and on main, so neither MiniMax-M3.1-Flash config key gets results collected. The new block uses pr-link: TBD. validate_added_pr_link accepts only the canonical PR URL or an XXX placeholder, and TBD is neither. Fix: set pr-link to https://github.com/SemiAnalysisAI/InferenceX/pull/<this PR number> or to the XXX placeholder, which merge_with_reuse rewrites to the canonical link.
Why this was flagged
The appended block at inferencex-e2e/perf-changelog.yaml:10882 sets pr-link: TBD. infx/workflows/validate_perf_changelog.py:139-150 (validate_added_pr_link) raises "new PR entry must use '…/pull/N' or an XXX placeholder; found 'TBD'" on PR runs. On main runs it raises "new main-branch entry has invalid pr-link" because TBD does not match CANONICAL_PR_LINK (line 21). PR_LINK_PLACEHOLDERS (lines 22-25) holds only XXX and https://github.com/SemiAnalysisAI/InferenceX/pull/XXX. So the changelog validation step fails, and the entry that is meant to trigger result collection for minimaxm31flash-fp4-gb200-vllm-agentic and minimaxm31flash-fp4-gb300-vllm-agentic never passes. The base branch has no such entry. The placeholder canonicalization in merge_with_reuse.py:478 only rewrites recognised placeholders, so it does not fix TBD either.
Verification: Trigger: the PR is merged through the documented merge_with_reuse tool. The new entry ends with pr-link: TBD (perf-changelog.yaml:10882), and validate_added_pr_link (validate_perf_changelog.py:139-150) rejects that value. merge_with_reuse.merge_pr calls canonicalize_changelog(pr, repo) (merge_with_reuse.py:631). As a result the standard merge-with-reuse flow aborts for this PR.
| # MiniMax-M3.1-Flash AgentX on GB200: day-zero vLLM image from | ||
| # https://vllm.ai/blog/2026-10-23-minimax-m31-flash, arm64 CUDA 13.4 build, | ||
| # verified 2026-10-23 (manifest sha256:86f582dae25ea92802353727164ae4eb03b37b18baf9119b805ffaec2a7ef07c). | ||
| # NVFP4 routed experts with MXFP8 dense layers and an NVFP4 KV cache. DSpark is | ||
| # off until a MiniMax-M3.1-Flash golden AL curve is committed. |
There was a problem hiding this comment.
🟡 nit (optional): New provenance/header comments break the AGENTS.md "No header comments" and "No history" rules, sweep:day-zero vLLM image from|verified 2026-10-23|Serve flags follow the GB200 TP4 recipe. The cited manifest sha256:86f582… also differs from the pinned digest sha256:5d76347…, which misleads readers. Fix: delete the blog/verified-date/manifest header above each master entry, and delete the "Serve flags follow…" lines in both recipes (gb200-fp4/agentic.yaml:26-28). Keep only per-line technical why-comments.
Why this was flagged
AGENTS.md:64-66 says comments may not sit above a recipe or section describing the config, and may not say where something was derived from. nvidia-master.yaml:8757-8761 and the matching gb300 header at about 8780-8784 describe the image source blog, the verification date and a manifest sha. The cited manifest sha (86f582…) does not match the image digest pinned at :8763 (5d76347…). Both recipes, at gb200-fp4/agentic.yaml:26-28 and gb300-fp4/agentic.yaml:26-28, say "Serve flags follow the GB200 TP4 recipe at https://recipes.vllm.ai/…", which is a provenance comment. There is no runtime impact. This is a convention breach that maintainers have to clean up.
Verification: nit: the rule is broken on every merge, but nothing breaks at runtime. The repo's CLAUDE.md says "All guidance ... lives in AGENTS.md", and AGENTS.md:60-66 "Code comments (mandatory)" applies to "every file, including recipes, configs". It says "No header comments ... above a recipe, override, or section. Do not describe what the file, topology, or config is" and "No history. Do not write comments that describe ... where it was derived from".
| model: MiniMaxAI/MiniMax-M3.1-Flash | ||
| model-prefix: minimaxm31flash | ||
| runner: cluster:gb200-nv | ||
| precision: fp4 | ||
| framework: vllm | ||
| multinode: false | ||
| srt-recipe-dir: minimaxm31flash/vllm/gb200-fp4 |
There was a problem hiding this comment.
🟡 (optional) MiniMax-M3 sweeps and eval campaigns will also pick up the two new MiniMax-M3.1-Flash configs. The full-sweep --model-prefix filter at infx/matrix/generate.py:1169 uses key.startswith(p), and the key minimaxm31flash-fp4-gb200-vllm-agentic starts with minimaxm3. Fix: have the filter compare against each config's own model-prefix field (exact match, or key.startswith(f"{p}-")), so minimaxm3, dsv4 and the other family names select only their own configs. The canonical full-sweep examples (--model-prefix dsr1) keep working either way.
Why this was flagged
An operator runs the documented MiniMax-M3 deployment-smoke command in infx/evals/EVALS.md:100-105 (full-sweep ... --model-prefix minimaxm3 --scenario-type agentic-coding --evals-only --all-evals), or any full-sweep --model-prefix minimaxm3. infx/matrix/generate.py:1169 keeps configs where any(key.startswith(p) for p in options.model_prefix). The new keys minimaxm31flash-fp4-gb200-vllm-agentic and minimaxm31flash-fp4-gb300-vllm-agentic (nvidia-master.yaml:8762, 8785) pass that test. The MiniMax-M3 matrix therefore gains GB200/GB300 MiniMax-M3.1-Flash rows. Those rows do not get the minimax_m3_full suite, because AUTOMATIC_AGENTIC_VENDOR_EVALS (generate.py:35-38) is keyed on the exact minimaxm3. On base the filter returns only minimaxm3 configs. docs/MODELS.md:133 calls the Prefix column the value the --model-prefix filter selects by. Nothing checks the model-prefix field itself. The existing dsv4/dsv41flash overlap shows this has gone unchecked; it is not a safeguard.
Verification: The trigger is any full-sweep --model-prefix minimaxm3 run. infx/matrix/generate.py:1169 keeps a config when any(key.startswith(p) for p in options.model_prefix). The two new keys at nvidia-master.yaml:8762 and 8785 both start with minimaxm3. After it, the MiniMax-M3 smoke matrix and any MiniMax-M3 full-sweep also schedule GB200 and GB300 MiniMax-M3.1-Flash jobs.
|
View unofficial run (performance): https://inferencex.semianalysis.com/inference?unofficialRun=39949684795 View unofficial run (accuracy): https://inferencex.semianalysis.com/evaluation?unofficialRun=39949684795 |
…gb200-gb300-dayzero
…gb200-gb300-dayzero # Conflicts: # inferencex-e2e/perf-changelog.yaml
…drafter; drop TP4 c64 and DEP4 c256, add DEP4 c96
|
/use 39949684795 |
|
@Oseltamivir staged run 39949684795. This run remains available across future |
Summary
Day-zero MiniMax-M3.1-Flash AgentX on GB300 with vLLM and DSpark. It follows the merged B200 recipe (#4426) with the vLLM blog's GB300 drafter and one extra DEP4 point. GB200 is #4433.
minimaxm31flash-fp4-gb300-vllm-agentic-dspark.minimaxm31flash/vllm/b200-fp4-mtp/agentic.yamlwith these changes:gpu_type: gb300andgpus_per_node: 4.VLLM_CHAT_TEMPLATE_RENDER_TIMEOUT: '600'. Under concurrent load, rendering the chat template for long AgentX conversations exceeded vLLM's 30 s default, and vLLM aborted those requests during warmup.vllm/vllm-openai:minimax-m31-flash-arm64-cu134@sha256:5d76347…, the arm64 build of B200's image.What changed from the first version of this PR
Validation
infx generatebinds all 8 points to exactly one variant each, with the expected speculative config, golden AL and render timeout.