Repository navigation
[Klaud Cold] minimaxm31flash-fp4-gb200-vllm-agentic-dspark: day-zero MiniMax-M3.1-Flash GB200 DSpark recipe #4424
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Changes from all commits
File filter
Filter by extension
Conversations
Jump to
Diff view
Diff view
There are no files selected for viewing
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,92 @@ | ||
| base: | ||
| schema: 2 | ||
| name: minimaxm31flash-fp4-gb200-vllm-agentic-dspark | ||
| resources: | ||
| gpu_type: gb200 | ||
| gpus_per_node: 4 | ||
| frontend: | ||
| type: vllm | ||
| enable_multiple_frontends: false | ||
| observability: | ||
| enabled: false | ||
| tachometer: | ||
| enabled: false | ||
| engine: | ||
| type: vllm | ||
| connector: null | ||
| # Cold loads of the 250 GB checkpoint from the hub cache run past half an hour. | ||
| health_check: | ||
| interval_seconds: 5 | ||
| max_attempts: 1440 | ||
| roles: | ||
| agg: | ||
| nodes: 1 | ||
| workers: 1 | ||
| # Serve flags follow the GB200 TP4 recipe at | ||
| # https://recipes.vllm.ai/MiniMaxAI/MiniMax-M3.1-Flash. | ||
| args: | ||
| served-model-name: MiniMaxAI/MiniMax-M3.1-Flash | ||
| language-model-only: true | ||
| trust-remote-code: true | ||
| reasoning-parser: minimax_m3 | ||
| tool-call-parser: minimax_m3 | ||
| enable-auto-tool-choice: true | ||
| # MSA sparse/index cache needs 128-token blocks. | ||
| block-size: 128 | ||
| # The checkpoint is QAT-trained for an NVFP4 KV cache. | ||
| kv-cache-dtype: nvfp4 | ||
| attention-config: '{"indexer_kv_dtype":"nvfp4","use_prefill_query_quantization":true}' | ||
| disable-hybrid-kv-cache-manager: true | ||
| # Three-token DSpark from the checkpoint's dspark/ head, per the | ||
| # recipes.vllm.ai GB200 TP4 command. Throughput runs replace | ||
| # block rejection with the golden acceptance length. | ||
| speculative-config: '{"method":"dspark","num_speculative_tokens":3,"attention_backend":"FLASH_ATTN","kv_cache_dtype":"fp8","draft_sample_method":"probabilistic","rejection_sample_method":"block"}' | ||
| enable-prefix-caching: true | ||
| max-model-len: 1048576 | ||
| max-num-batched-tokens: 8192 | ||
| compilation-config: '{"cudagraph_mode":"FULL_AND_PIECEWISE"}' | ||
| disable-uvicorn-access-log: true | ||
| env: | ||
| VLLM_FLOAT32_MATMUL_PRECISION: high | ||
| PYTHONUNBUFFERED: '1' | ||
| VLLM_ENGINE_READY_TIMEOUT_S: '7200' | ||
| benchmark: | ||
| env: | ||
| AIPERF_REQUIRED_SERVER_METRIC_PREFIX: 'vllm:' | ||
|
|
||
| # Native zip expansion creates one variant per concurrency. | ||
|
|
||
| # TP4 is the upstream Blackwell layout and covers low to mid concurrency. | ||
| zip_override_tp4: | ||
| roles: | ||
| agg: | ||
| gpus: 4 | ||
| args: | ||
| tensor-parallel-size: 4 | ||
| max-num-seqs: [8, 8, 16, 32, 64, 128] | ||
| benchmark: | ||
| env: | ||
| CONC: ['1', '4', '8', '16', '32', '64'] | ||
|
|
||
| # DEP4: TP1 x DP4 + EP4 behind a consistent-hash router for high concurrency. | ||
| zip_override_dep4: | ||
| frontend: | ||
| type: vllm-router | ||
| args: | ||
| policy: consistent_hash | ||
| prometheus-host: 127.0.0.1 | ||
| prometheus-port: 18000 | ||
| request-timeout-secs: 14400 | ||
| disable-retries: true | ||
| setup_script: vllm-router.sh | ||
| roles: | ||
| agg: | ||
| gpus: 4 | ||
| args: | ||
| tensor-parallel-size: 1 | ||
| data-parallel-size: 4 | ||
| enable-expert-parallel: true | ||
| max-num-seqs: [32, 64, 128] | ||
| benchmark: | ||
| env: | ||
| CONC: ['64', '128', '256'] |
| Original file line number | Diff line number | Diff line change |
|---|---|---|
|
|
@@ -8597,6 +8597,29 @@ dsv41flash-fp4-gb200-vllm-agentic-dspark: | |
| # DEP4 is native TP1 x DP4 + EP4 with the same settings. | ||
| - { tp: 4, ep: 4, dp-attn: true, kv-offloading: none, spec-decoding: mtp, conc-list: [64, 128], router: { name: vllm-router, version: "0.1.14" }, srt-recipe: agentic.yaml } | ||
|
|
||
| # MiniMax-M3.1-Flash AgentX on GB200: day-zero vLLM image from | ||
| # https://vllm.ai/blog/2026-10-23-minimax-m31-flash, arm64 CUDA 13.4 build, | ||
| # verified 2026-10-23 (manifest sha256:86f582dae25ea92802353727164ae4eb03b37b18baf9119b805ffaec2a7ef07c). | ||
| # NVFP4 routed experts with MXFP8 dense layers and an NVFP4 KV cache, with | ||
| # DSpark from the checkpoint's dspark/ head (golden AL minimaxm31flash_dspark). | ||
| minimaxm31flash-fp4-gb200-vllm-agentic-dspark: | ||
| image: vllm/vllm-openai:minimax-m31-flash-arm64-cu134@sha256:5d76347628973fb80519a919a2f2740b6645b62cba3fe2bd87b25f111bd0e948 | ||
| model: MiniMaxAI/MiniMax-M3.1-Flash | ||
| model-prefix: minimaxm31flash | ||
| runner: cluster:gb200-nv | ||
| precision: fp4 | ||
| framework: vllm | ||
| multinode: false | ||
| srt-recipe-dir: minimaxm31flash/vllm/gb200-fp4-mtp | ||
| scenarios: | ||
| agentic-coding: | ||
| - dram-utilization: 0.80 | ||
| search-space: | ||
| # TP4 is the upstream Blackwell layout. | ||
| - { tp: 4, kv-offloading: none, spec-decoding: mtp, conc-list: [1, 4, 8, 16, 32, 64], srt-recipe: agentic.yaml } | ||
| # DEP4 is native TP1 x DP4 + EP4 behind a consistent-hash vLLM Router. | ||
| - { tp: 4, ep: 4, dp-attn: true, kv-offloading: none, spec-decoding: mtp, conc-list: [64, 128, 256], router: { name: vllm-router, version: "0.1.14" }, srt-recipe: agentic.yaml } | ||
|
|
||
|
Contributor
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. 🔴 Once merged, every throughput point of the new config fails before benchmarking with Why this was flaggedThe new entry sets Verification: After merge, any agentic throughput run of the new config key hits the fault. The recipe (agentic.yaml:43) uses |
||
| # SGLang arm for DeepSeek-V4.1-Flash AgentX on GB200, from the SGLang cookbook | ||
| # (https://lmsysorg.mintlify.app/cookbook/autoregressive/DeepSeek/DeepSeek-V4_1). | ||
| # Official CUDA 13 nightly, verified 2026-10-14 (amd64 + arm64): | ||
|
|
||
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
🟡 nit (optional): Maintainers get provenance and header comments that AGENTS.md "Code comments (mandatory)" forbids. One of them gives a digest that does not match the pinned image. The header above the new master entry cites manifest
sha256:86f582da..., whileimage:pinssha256:5d763476.... Fix: remove the provenance and header comments, keeping only short technical line comments. This covers the 4 sites listed. Same pattern at 4 sites (nvidia-master.yaml:8600-8604, agentic.yaml:25-26, agentic.yaml:40-42, agentic.yaml:59).Why this was flagged
AGENTS.md says there must be no header comments above a recipe, override or section. It also bans comments saying where something was derived from. The added block at nvidia-master.yaml:8600-8604 describes the entry and records its source blog, verification date and a manifest digest. That digest (86f5...) differs from the digest pinned at line 8606 (5d76...), so a reader cannot tell which image was verified. agentic.yaml:25-26 ("Serve flags follow the GB200 TP4 recipe at ...") and agentic.yaml:40-42 ("per the recipes.vllm.ai GB200 TP4 command") are provenance comments. agentic.yaml:59 ("TP4 is the upstream Blackwell layout...") is a header above an override. None of this changes runtime behaviour.
Verification: AGENTS.md:60-66, "Code comments (mandatory)", applies to "every file, including recipes, configs" and bans header comments above a recipe, override, or section and comments on where config was derived from. The nvidia-master.yaml header and the agentic.yaml comments break it. The comment cites
sha256:86f582da..., whileimage:pinssha256:5d76347628973fb8....