Skip to content
Closed
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Original file line number Diff line number Diff line change
@@ -0,0 +1,92 @@
base:
schema: 2
name: minimaxm31flash-fp4-gb200-vllm-agentic-dspark
resources:
gpu_type: gb200
gpus_per_node: 4
frontend:
type: vllm
enable_multiple_frontends: false
observability:
enabled: false
tachometer:
enabled: false
engine:
type: vllm
connector: null
# Cold loads of the 250 GB checkpoint from the hub cache run past half an hour.
health_check:
interval_seconds: 5
max_attempts: 1440
roles:
agg:
nodes: 1
workers: 1
# Serve flags follow the GB200 TP4 recipe at
# https://recipes.vllm.ai/MiniMaxAI/MiniMax-M3.1-Flash.
args:
served-model-name: MiniMaxAI/MiniMax-M3.1-Flash
language-model-only: true
trust-remote-code: true
reasoning-parser: minimax_m3
tool-call-parser: minimax_m3
enable-auto-tool-choice: true
# MSA sparse/index cache needs 128-token blocks.
block-size: 128
# The checkpoint is QAT-trained for an NVFP4 KV cache.
kv-cache-dtype: nvfp4
attention-config: '{"indexer_kv_dtype":"nvfp4","use_prefill_query_quantization":true}'
disable-hybrid-kv-cache-manager: true
# Three-token DSpark from the checkpoint's dspark/ head, per the
# recipes.vllm.ai GB200 TP4 command. Throughput runs replace
# block rejection with the golden acceptance length.
speculative-config: '{"method":"dspark","num_speculative_tokens":3,"attention_backend":"FLASH_ATTN","kv_cache_dtype":"fp8","draft_sample_method":"probabilistic","rejection_sample_method":"block"}'
enable-prefix-caching: true
max-model-len: 1048576
max-num-batched-tokens: 8192
compilation-config: '{"cudagraph_mode":"FULL_AND_PIECEWISE"}'
disable-uvicorn-access-log: true
env:
VLLM_FLOAT32_MATMUL_PRECISION: high
PYTHONUNBUFFERED: '1'
VLLM_ENGINE_READY_TIMEOUT_S: '7200'
benchmark:
env:
AIPERF_REQUIRED_SERVER_METRIC_PREFIX: 'vllm:'

# Native zip expansion creates one variant per concurrency.

# TP4 is the upstream Blackwell layout and covers low to mid concurrency.
zip_override_tp4:
roles:
agg:
gpus: 4
args:
tensor-parallel-size: 4
max-num-seqs: [8, 8, 16, 32, 64, 128]
benchmark:
env:
CONC: ['1', '4', '8', '16', '32', '64']

# DEP4: TP1 x DP4 + EP4 behind a consistent-hash router for high concurrency.
zip_override_dep4:
frontend:
type: vllm-router
args:
policy: consistent_hash
prometheus-host: 127.0.0.1
prometheus-port: 18000
request-timeout-secs: 14400
disable-retries: true
setup_script: vllm-router.sh
roles:
agg:
gpus: 4
args:
tensor-parallel-size: 1
data-parallel-size: 4
enable-expert-parallel: true
max-num-seqs: [32, 64, 128]
benchmark:
env:
CONC: ['64', '128', '256']
23 changes: 23 additions & 0 deletions inferencex-e2e/configs/nvidia-master.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -8597,6 +8597,29 @@ dsv41flash-fp4-gb200-vllm-agentic-dspark:
# DEP4 is native TP1 x DP4 + EP4 with the same settings.
- { tp: 4, ep: 4, dp-attn: true, kv-offloading: none, spec-decoding: mtp, conc-list: [64, 128], router: { name: vllm-router, version: "0.1.14" }, srt-recipe: agentic.yaml }

# MiniMax-M3.1-Flash AgentX on GB200: day-zero vLLM image from
# https://vllm.ai/blog/2026-10-23-minimax-m31-flash, arm64 CUDA 13.4 build,
# verified 2026-10-23 (manifest sha256:86f582dae25ea92802353727164ae4eb03b37b18baf9119b805ffaec2a7ef07c).
# NVFP4 routed experts with MXFP8 dense layers and an NVFP4 KV cache, with
# DSpark from the checkpoint's dspark/ head (golden AL minimaxm31flash_dspark).
Comment on lines +8600 to +8604

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟡 nit (optional): Maintainers get provenance and header comments that AGENTS.md "Code comments (mandatory)" forbids. One of them gives a digest that does not match the pinned image. The header above the new master entry cites manifest sha256:86f582da..., while image: pins sha256:5d763476.... Fix: remove the provenance and header comments, keeping only short technical line comments. This covers the 4 sites listed. Same pattern at 4 sites (nvidia-master.yaml:8600-8604, agentic.yaml:25-26, agentic.yaml:40-42, agentic.yaml:59).

Why this was flagged

AGENTS.md says there must be no header comments above a recipe, override or section. It also bans comments saying where something was derived from. The added block at nvidia-master.yaml:8600-8604 describes the entry and records its source blog, verification date and a manifest digest. That digest (86f5...) differs from the digest pinned at line 8606 (5d76...), so a reader cannot tell which image was verified. agentic.yaml:25-26 ("Serve flags follow the GB200 TP4 recipe at ...") and agentic.yaml:40-42 ("per the recipes.vllm.ai GB200 TP4 command") are provenance comments. agentic.yaml:59 ("TP4 is the upstream Blackwell layout...") is a header above an override. None of this changes runtime behaviour.

Verification: AGENTS.md:60-66, "Code comments (mandatory)", applies to "every file, including recipes, configs" and bans header comments above a recipe, override, or section and comments on where config was derived from. The nvidia-master.yaml header and the agentic.yaml comments break it. The comment cites sha256:86f582da..., while image: pins sha256:5d76347628973fb8....

minimaxm31flash-fp4-gb200-vllm-agentic-dspark:
image: vllm/vllm-openai:minimax-m31-flash-arm64-cu134@sha256:5d76347628973fb80519a919a2f2740b6645b62cba3fe2bd87b25f111bd0e948
model: MiniMaxAI/MiniMax-M3.1-Flash
model-prefix: minimaxm31flash
runner: cluster:gb200-nv
precision: fp4
framework: vllm
multinode: false
srt-recipe-dir: minimaxm31flash/vllm/gb200-fp4-mtp
scenarios:
agentic-coding:
- dram-utilization: 0.80
search-space:
# TP4 is the upstream Blackwell layout.
- { tp: 4, kv-offloading: none, spec-decoding: mtp, conc-list: [1, 4, 8, 16, 32, 64], srt-recipe: agentic.yaml }
# DEP4 is native TP1 x DP4 + EP4 behind a consistent-hash vLLM Router.
- { tp: 4, ep: 4, dp-attn: true, kv-offloading: none, spec-decoding: mtp, conc-list: [64, 128, 256], router: { name: vllm-router, version: "0.1.14" }, srt-recipe: agentic.yaml }

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🔴 Once merged, every throughput point of the new config fails before benchmarking with No committed golden curve minimaxm31flash_dspark.yaml. The PR description says the curve is collected separately. The new changelog entry still triggers result collection on merge. Fix: add infx/golden_al_distribution/minimaxm31flash_dspark.yaml with a thinking_on value for 3 draft tokens in this PR, or hold the config and changelog entry until that curve lands. [also at: inferencex-e2e/benchmarks/single_node/srt-slurm-recipes/minimaxm31flash/vllm/gb200-fp4-mtp/agentic.yaml:43 - The changelog entry starts a sweep whose recipe generation fails at every point, because the golden curve this config needs is not in the repo.]

Why this was flagged

The new entry sets model-prefix: minimaxm31flash, and the recipe's speculative-config has "method":"dspark","num_speculative_tokens":3. For throughput runs, infx/srt_slurm/synthetic_acceptance.py:131 calls golden_length(MODEL_PREFIX, spec, ...). curve_name (infx/golden_al_distribution/curves.py:47) resolves this to minimaxm31flash_dspark. load_curve (curves.py:67-68) then raises No committed golden curve minimaxm31flash_dspark.yaml because no such file exists in infx/golden_al_distribution/. The perf-changelog block appended by this PR triggers collection for minimaxm31flash-fp4-gb200-vllm-agentic-dspark, so all 9 search-space points (TP4 conc 1-64, DEP4 conc 64-256) error out instead of producing results. The PR description admits generation fails until the curve lands, but nothing in the change gates the config or the entry on it.

Verification: After merge, any agentic throughput run of the new config key hits the fault. The recipe (agentic.yaml:43) uses "method":"dspark", so curve_name (curves.py:41-62) returns minimaxm31flash_dspark. load_curve (curves.py:66-68) then raises No committed golden curve minimaxm31flash_dspark.yaml, because the diff adds no such file. All 9 search-space points fail at recipe generation.

# SGLang arm for DeepSeek-V4.1-Flash AgentX on GB200, from the SGLang cookbook
# (https://lmsysorg.mintlify.app/cookbook/autoregressive/DeepSeek/DeepSeek-V4_1).
# Official CUDA 13 nightly, verified 2026-10-14 (amd64 + arm64):
Expand Down
10 changes: 10 additions & 0 deletions inferencex-e2e/perf-changelog.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -10871,3 +10871,13 @@
- "Update the vLLM ROCm image from v0.31.0 to v0.32.0."
pr-link: https://github.com/SemiAnalysisAI/InferenceX/pull/4395
scenario-type: [agentic-coding]

- config-keys:
- minimaxm31flash-fp4-gb200-vllm-agentic-dspark
description:
- "Add day-zero MiniMax-M3.1-Flash (MiniMaxAI/MiniMax-M3.1-Flash, NVFP4 experts + NVFP4 KV) AgentX on GB200 with vLLM image minimax-m31-flash-arm64-cu134 (https://vllm.ai/blog/2026-10-23-minimax-m31-flash)."
- "Serve flags follow the recipes.vllm.ai GB200 TP4 recipe: block size 128, nvfp4 KV and indexer cache, prefill query quantization, FULL_AND_PIECEWISE graphs, hybrid KV cache manager disabled, minimax_m3 reasoning and tool parsers, 1M context."
- "Search space: TP4 at conc 1-64 and DEP4 (TP1 x DP4 + EP4, vLLM Router) at conc 64-256, no KV offloading."
- "Three-token DSpark from the checkpoint's dspark/ head with the FLASH_ATTN drafter, probabilistic drafting and block verification (recipes.vllm.ai GB200 command). Throughput runs use the minimaxm31flash_dspark golden AL curve."
pr-link: https://github.com/SemiAnalysisAI/InferenceX/pull/4424
scenario-type: [agentic-coding]
Loading