Repository navigation
[Klaud Cold] minimaxm31flash-fp4-gb300-dynamo-vllm-agentic-dspark-agg: MiniMax-M3.1-Flash DEP8 + CPU offload on two GB300 trays - #4436
Oseltamivir wants to merge 5 commits into
Conversation
…1-Flash TP8/TEP8/DEP8 on two GB300 trays
|
Thanks for the contribution!
中文感谢你的贡献!
|
| description: | ||
| - "Add MiniMax-M3.1-Flash AgentX on two GB300 trays (8 GPUs) aggregated through Dynamo vLLM, with the single-node GB300 serve flags (NVFP4 KV and indexer cache, 600 s chat-template render timeout) and the day-0 arm64 vLLM image." | ||
| - "TP8 c1-c32 and TEP8 (TP8 + EP8) c48-c64 with 7-token DSpark (golden AL 4.04); DEP8 (TP1 x DP8 + EP8, per-node DP launch) c72-c128 with 3-token DSpark (golden AL 2.88) and vLLM Simple CPU KV offload (179 GiB per DP rank, lazy). FlashInfer PrimiTS drafter throughout." | ||
| pr-link: TBD |
There was a problem hiding this comment.
🔴 The changelog check fails on this PR, and a main-branch merge would also be rejected. The new block's pr-link is TBD, which the validator does not accept. Accepted values are XXX, https://github.com/SemiAnalysisAI/InferenceX/pull/XXX, or the canonical PR URL. Fix: set pr-link: to this PR's canonical https://github.com/SemiAnalysisAI/InferenceX/pull/<N> URL, or to XXX until the number is known.
Why this was flagged
The new entry at inferencex-e2e/perf-changelog.yaml:10998 sets pr-link: TBD. On PR runs, validate_added_pr_link in inferencex-e2e/infx/workflows/validate_perf_changelog.py:147 accepts only PR_LINK_PLACEHOLDERS (XXX and https://github.com/SemiAnalysisAI/InferenceX/pull/XXX, lines 22-25) or the exact expected PR URL. Anything else raises "new PR entry must use ... or an XXX placeholder". On main-branch runs, line 142 requires CANONICAL_PR_LINK, so TBD raises "new main-branch entry has invalid pr-link". Changelog validation therefore fails, and the entry cannot trigger result collection for minimaxm31flash-fp4-gb300-dynamo-vllm-agentic-dspark-agg. On the base branch there is no such entry and the check passes.
Verification: The new block in inferencex-e2e/perf-changelog.yaml ends with pr-link: TBD. PR_LINK_PLACEHOLDERS (validate_perf_changelog.py lines 22-25) contains only "XXX" and ".../pull/XXX". At merge time, compare_entries(base, head, pr_number) at prepare_perf_changelog_merge.py:88 calls validate_added_pr_link with an int pr_number and raises on 'TBD'. CONTRIBUTING.md:169 names merge_with_reuse as the supported merge path, so it fails for this PR.
| # MiniMax-M3.1-Flash AgentX on two GB300 trays (8 GPUs), aggregated through | ||
| # Dynamo vLLM. Serve flags follow the single-node GB300 recipe | ||
| # (single_node/srt-slurm-recipes/minimaxm31flash/vllm/gb300-fp4-mtp); the | ||
| # Dynamo setup follows minimaxm3/vllm/gb200-fp4/agentx. |
There was a problem hiding this comment.
🟡 (optional) Maintainers get new header and provenance comments that AGENTS.md "Code comments (mandatory)" forbids in recipes and master configs. The recipe top block and the master-entry block describe the topology and say where flags were taken from. Fix: delete the comment blocks that sit above the file, override or entry, and keep only per-line technical reasons such as the existing block-size and render-timeout notes. That covers the 5 sites listed. Same pattern at 5 sites (agg-variants.yaml:3-6, agg-variants.yaml:95, agg-variants.yaml:107, agg-variants.yaml:121-124, nvidia-master.yaml:5611-5614).
Why this was flagged
AGENTS.md:64 says "Never add broad, overarching comments at the top of a file or above a recipe, override, or section. Do not describe what the file, topology, or config is." AGENTS.md:65 also bans comments about where code was derived from. agg-variants.yaml:3-6 describes the file ("MiniMax-M3.1-Flash AgentX on two GB300 trays...") and gives provenance ("Serve flags follow the single-node GB300 recipe...; the Dynamo setup follows minimaxm3/vllm/gb200-fp4/agentx"). The comments above override_tp8 (line 95), override_tep8 (line 107) and override_dep8_vllm_simple (lines 121-124) describe each override's topology. nvidia-master.yaml:5611-5614 describes the entry's layouts and says "as in the single-node GB300 recipe". The base branch has none of these comments, so the diff adds every one of them.
Verification: This happens on every merge, because the comments are newly added in this diff. AGENTS.md:64 forbids header comments "above a recipe, override, or section" that describe "what the file, topology, or config is". AGENTS.md:65 forbids comments about "where it was derived from". Nothing breaks at runtime.
|
View unofficial run (performance): https://inferencex.semianalysis.com/inference?unofficialRun=39987826992 View unofficial run (accuracy): https://inferencex.semianalysis.com/evaluation?unofficialRun=39987826992 |
…LM engine-helper rename
Summary
MiniMax-M3.1-Flash AgentX on two GB300 trays (8 GPUs), aggregated through Dynamo vLLM, for the high-concurrency end of the curve. Low and mid concurrency stay on the single-tray TP4 recipe (#4419).
dp_launch_mode: per_node)minimaxm31flash-fp4-gb300-dynamo-vllm-agentic-dspark-agg.multi_node/srt-slurm-recipes/minimaxm31flash/vllm/gb300-fp4/agentx/agg-variants.yaml.VLLM_CHAT_TEMPLATE_RENDER_TIMEOUT=600minimax_m3reasoning and tool parsers in the frontend. Earlier Dynamo builds fail at import on this vLLM, because vLLM renamedget_engine_zmq_addresses([Bugfix] Eliminate frontend ZMQ port TOCTOU with inherited listeners vllm-project/vllm#54113). fix(vllm): import engine-launch helpers lazily so workers start on new vLLM ai-dynamo/dynamo#16378 made that import lazy.dsv4GB200 DEP8 recipe.dram-utilization: 0.80gives 771 GB of CPU DRAM per tray, and each of the 4 DP ranks on a tray gets 179 GiB of it.vllm/vllm-openai:minimax-m31-flash-arm64-cu134@sha256:5d76347…, the same image as [Klaud Cold] minimaxm31flash-fp4-gb300-vllm-agentic-dspark: day-zero MiniMax-M3.1-Flash GB300 DSpark recipe #4419.Why there are no TP8/TEP8 points
The model has 64 query heads over 4 KV heads. vLLM's NVFP4 KV-cache attention kernels need 16 query heads per KV head on each rank. Attention TP8 leaves 8, and the engine refuses to start:
DEP8 runs attention at TP1, so it is unaffected.
Validation
infx generatebinds all 3 points to one variant each with the expected TP/DP/EP, draft tokens, golden AL and offload size, and srtctl accepts them.