Repository navigation
[Klaud Cold] Add day-0 MiniMax-M3.1-Flash B300 vLLM AgentX DSpark recipe / [Klaud Cold] 新增 MiniMax-M3.1-Flash B300 vLLM AgentX DSpark 首日配方 - #4421
Conversation
… vLLM AgentX recipes TP4 + DEP4 aggregate recipes on the vllm/vllm-openai:minimax-m31-flash day-0 image, serve flags from recipes.vllm.ai Blackwell TP4 (NVFP4 KV and indexer cache, block-size 128). DSpark stays off until a golden AL curve is committed for this model.
|
Thanks for the contribution!
中文感谢你的贡献!
|
|
View unofficial run (performance): https://inferencex.semianalysis.com/inference?unofficialRun=39904040374 View unofficial run (accuracy): https://inferencex.semianalysis.com/evaluation?unofficialRun=39904040374 |
Built-in DSpark drafter, levels 1-7, thinking_on (adaptive template), recipes.vllm.ai Blackwell TP4 serve flags.
| - "Add day-0 MiniMax-M3.1-Flash (MiniMaxAI/MiniMax-M3.1-Flash, NVFP4 experts + NVFP4 KV cache) AgentX recipes on B300 and B200 with the vLLM day-0 image vllm/vllm-openai:minimax-m31-flash." | ||
| - "Serve flags follow recipes.vllm.ai Blackwell TP4: block-size 128, nvfp4 KV and indexer cache, prefill query quantization, FULL_AND_PIECEWISE graphs, minimax_m3 parsers, text-only." | ||
| - "TP4 c1-c32 and DEP4 (TP1 x DP4 + EP4 behind vllm-router) c64-c256 on B300, c64-c128 on B200. DSpark is off until a golden AL curve exists for this model." | ||
| pr-link: TBD |
There was a problem hiding this comment.
🔴 The changelog check will fail this PR, and merge tooling will refuse it, because the new block's pr-link: TBD is not an accepted placeholder. validate_added_pr_link accepts only XXX, https://github.com/SemiAnalysisAI/InferenceX/pull/XXX, or the real PR URL. prepare_perf_changelog_merge raises "appended entry ... has unexpected pr-link" for any other value. Fix: set the line to pr-link: https://github.com/SemiAnalysisAI/InferenceX/pull/XXX or to this PR's canonical URL.
Why this was flagged
The appended block at inferencex-e2e/perf-changelog.yaml:10883 sets pr-link: TBD. In inferencex-e2e/infx/workflows/validate_perf_changelog.py:22-25, PR_LINK_PLACEHOLDERS holds only "XXX" and the /pull/XXX URL. On PR runs, validate_added_pr_link (lines 146-150) raises ChangelogValidationError "new PR entry must use ... or an XXX placeholder; found 'TBD'". On main runs it needs CANONICAL_PR_LINK instead. In inferencex-e2e/infx/workflows/prepare_perf_changelog_merge.py:105-111, a link not in PR_LINK_PLACEHOLDERS also raises "appended entry N has unexpected pr-link 'TBD'" instead of being rewritten to the PR URL. The changelog entry is what triggers result collection, so the two new config keys never get swept or merged through the normal path. The base branch has no such entry.
Verification: The appended block ends with pr-link: TBD (perf-changelog.yaml:10883). In inferencex-e2e/infx/workflows/validate_perf_changelog.py, on a PR run, validate_added_pr_link raises new PR entry must use ... or an XXX placeholder; found 'TBD'. Result: the changelog check fails and merge tooling refuses the PR until the link is changed to the /pull/XXX form or the real PR URL.
| benchmark: | ||
| env: | ||
| CONC: ['64', '128', '256'] |
There was a problem hiding this comment.
🔴 DEP4 points on B300 and B200 may lose per-conversation rank affinity, so later turns land cold on other DP ranks and DEP4 throughput reads low. Both zip_override_dep4 blocks run a consistent_hash vllm-router but never set AIPERF_HTTP_X_SESSION_ID_FROM_CORRELATION_ID. Without it AIPerf sends no X-Session-ID header for the router to hash on. Fix: in both DEP4 overrides, add AIPERF_HTTP_X_SESSION_ID_FROM_CORRELATION_ID: '1' under benchmark.env, as the dsv4 vLLM DEP arms do. That keeps every turn of a conversation on the rank that holds its prefix cache.
Why this was flagged
The DEP4 matrix points (b300 conc 64/128/256, b200 conc 64/128) select zip_override_dep4. That block puts four DP ranks behind vllm-router with policy: consistent_hash. Its benchmark.env sets only CONC (b300-fp4/agentic.yaml:87-89, b200-fp4/agentic.yaml:88-90). The dsv4 vLLM DEP arms use the same router and policy, and they also set AIPERF_HTTP_X_SESSION_ID_FROM_CORRELATION_ID: '1' (dsv4/vllm/b300-fp4-mtp/agentic.yaml:135). perf-changelog.yaml:6598 notes that without affinity, turns scatter across DP ranks and the multi-turn workload collapses to cold prefill. The trigger is that vllm-router 0.1.14's consistent hash keys on X-Session-ID or X-User-ID and not on X-Correlation-ID. The dsv41flash vLLM DEP arms also leave the flag out, so the repo is split and nothing guards this. The new model has no earlier DEP4 numbers, so this is new coverage with possibly understated results, not a regression.
Verification: Trigger: every DEP4 point, as long as the vllm-router consistent_hash policy keys on X-Session-ID. In b300-fp4/agentic.yaml, zip_override_dep4 sets policy: consistent_hash (line 73). Its benchmark.env sets only CONC: ['64', '128', '256'] (lines 87-89). The B200 recipe has the same shape (consistent_hash at line 74, CONC only).
| - { tp: 8, spec-decoding: mtp, kv-offloading: dram, kv-offload-backend: { name: native }, conc-list: [1, 5], srt-recipe: agentic.yaml } | ||
|
|
||
| minimaxm31flash-fp4-b300-vllm-agentic: | ||
| # Day-0 vLLM image for MiniMax-M3.1-Flash (vllm.ai/blog/2026-10-23-minimax-m31-flash). |
There was a problem hiding this comment.
🟡 nit (optional): several new comments record provenance or upstream sources, which the AGENTS.md comment rules forbid. sweep:Day-0 vLLM image|Serve flags follow the Blackwell|upstream Blackwell layout Examples: the "Day-0 vLLM image for MiniMax-M3.1-Flash (vllm.ai/blog/...)" line above image: here, and "Serve flags follow the Blackwell TP4 recipe at https://recipes.vllm.ai/..." above args: in the b300-fp4 and b200-fp4 recipes. Fix: delete these lines, or replace each with one short technical line saying why the value was chosen (for example, keep only the DSpark/golden-AL reason).
Why this was flagged
AGENTS.md "Code comments (mandatory)" applies to every file, including recipes and configs. It bans comments about where something was derived from ("No history") and comments describing the topology or config ("No header comments"). The diff adds comments that cite a blog post and an upstream recipe as the source of the image and serve flags (nvidia-master.yaml:5553 and :5571, and agentic.yaml lines 26-28 in both recipes). It also adds "TP4 is the upstream Blackwell layout" (master :5565/:5583, recipes :56), which states provenance rather than a technical reason. Maintainers inherit comments that go stale when the upstream recipe or blog changes. The base branch has none of these comments.
Verification: AGENTS.md:60-66 says the rule "Applies to every file, including recipes, configs". It bans comments about "where it was derived from" (No history) and comments that "describe what the file, topology, or config is" (No header comments). The diff adds comments of both kinds, such as the "Day-0 vLLM image" line at nvidia-master.yaml:5553. Nothing at runtime breaks.
Give b200-nscale a writable download root so script runs can fetch unstaged checkpoints.
…b200-b300-dayzero
…k on the golden AL curve Split B200 into its own PR. TP4 drafts 7 DSpark tokens (vLLM day-0 GB300 AgentX frontier), DEP4 drafts 3 (recipes.vllm.ai Blackwell); throughput runs use the minimaxm31flash_dspark golden AL from #4425.
|
/use 39904040374 |
|
@Oseltamivir staged run 39904040374. This run remains available across future |
Summary
Day-0 MiniMax-M3.1-Flash (
MiniMaxAI/MiniMax-M3.1-Flash, ~428B/23B active, MXFP8 dense + NVFP4 experts, QAT for an NVFP4 KV cache) AgentX recipe on B300 with DSpark, following the vLLM day-0 blog and recipes.vllm.ai. B200 is in a separate PR.minimaxm31flash-fp4-b300-vllm-agentic-dspark, recipeminimaxm31flash/vllm/b300-fp4-mtp/agentic.yaml.vllm/vllm-openai:minimax-m31-flash@sha256:86f582da…(day-0 multi-arch build, verified on Docker Hub).block-size 128,kv-cache-dtype nvfp4,attention-config {"indexer_kv_dtype":"nvfp4","use_prefill_query_quantization":true},FULL_AND_PIECEWISEcudagraphs,disable-hybrid-kv-cache-manager,minimax_m3reasoning/tool parsers,language-model-only,VLLM_FLOAT32_MATMUL_PRECISION=high.{"method":"dspark","attention_backend":"FLASH_ATTN","kv_cache_dtype":"fp8","draft_sample_method":"probabilistic","rejection_sample_method":"block"}. Throughput runs use synthetic acceptance from the golden curveminimaxm31flash_dspark.yaml([Klaud Cold] Add MiniMax-M3.1-Flash DSpark golden AL curve and SPEED-Bench collector / [Klaud Cold] 新增 MiniMax-M3.1-Flash DSpark 黄金 AL 曲线与 SPEED-Bench 采集脚本 #4425).minimaxm31flashtodocs/MODELS.md.