Repository navigation
[NVIDIA][AgentX] MiniMax-M3 B200 vLLM digest pin, TP4 DRAM-offload tuning and capped-active c10 entries / [NVIDIA][AgentX] MiniMax-M3 B200 vLLM 按摘要固定镜像、调优 TP4 DRAM 卸载并新增限制活跃序列的 c10 配置 #4382
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Changes from all commits
fb296fc
7ddc333
ae58809
File filter
Filter by extension
Conversations
Jump to
Diff view
Diff view
There are no files selected for viewing
| Original file line number | Diff line number | Diff line change |
|---|---|---|
|
|
@@ -5558,7 +5558,7 @@ minimaxm3-fp4-b300-trtllm-agentic-mtp: | |
| # enabled by framework=dynamo-vllm. | ||
|
|
||
| minimaxm3-fp4-b200-vllm-agentic-mtp: | ||
| image: vllm/vllm-openai:nightly-af7f9488c2210d67e1033ecdc845b087ee7fe92b | ||
| image: vllm/vllm-openai:nightly-af7f9488c2210d67e1033ecdc845b087ee7fe92b@sha256:8e2ffbb9f866de678b22e7446903b1ee5a5766c51a948013277477c462a68a77 | ||
| model: nvidia/MiniMax-M3-NVFP4 | ||
| model-prefix: minimaxm3 | ||
| runner: cluster:b200-nscale | ||
|
|
@@ -5570,11 +5570,33 @@ minimaxm3-fp4-b200-vllm-agentic-mtp: | |
| agentic-coding: | ||
| - dram-utilization: 0.683 | ||
| search-space: | ||
| - { tp: 8, spec-decoding: mtp, kv-offloading: none, conc-list: [1], srt-recipe: agentic.yaml } | ||
| - { tp: 4, spec-decoding: mtp, kv-offloading: none, conc-list: [1, 5, 10, 15, 20], srt-recipe: agentic.yaml } | ||
| - { tp: 4, spec-decoding: mtp, kv-offloading: none, conc-list: [15], srt-recipe: agentic.yaml:override_tp4_c15 } | ||
| - dram-utilization: 1.0 | ||
| search-space: | ||
| - { tp: 4, spec-decoding: mtp, kv-offloading: dram, kv-offload-backend: { name: vllm-simple }, conc-list: [30], srt-recipe: agentic.yaml:override_tp4_c30_dram } | ||
| - { tp: 4, spec-decoding: mtp, kv-offloading: dram, kv-offload-backend: { name: vllm-simple }, conc-list: [32], srt-recipe: agentic.yaml:override_tp4_c32_dram } | ||
| - { tp: 4, spec-decoding: mtp, kv-offloading: dram, kv-offload-backend: { name: vllm-simple }, conc-list: [38], srt-recipe: agentic.yaml:override_tp4_c38_dram } | ||
|
Comment on lines
5570
to
+5578
Contributor
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. 🟡 (optional) Maintainers inherit 11 overrides in the B200 MiniMax-M3 agentic.yaml that no config runs anymore. The master config now names a selector for every point. Overrides left without a selector include override_tp8_c1, override_tp4_c1/c5/c10/c20, and the dram c15/c20/c25/c34/c36/c40 blocks. Fix: delete the overrides no config references, or keep them reachable from a search-space entry. AGENTS.md says to delete recipes and assets no active config uses rather than leave them in place. Why this was flaggedOn the base branch, both search-space groups used bare Verification: On the base, both search-space groups used bare |
||
| minimaxm3-fp4-b200-vllm-agentic-mtp-c10: | ||
| image: vllm/vllm-openai:nightly-ac68c3087215e0a4f3cdfa218508c6aada57235d@sha256:98adb2311118263a453ec0acd5d4b72bda3db26cecc0fe8ffc94b9ab03c951f9 | ||
| model: nvidia/MiniMax-M3-NVFP4 | ||
| model-prefix: minimaxm3 | ||
| runner: cluster:b200-nscale | ||
| precision: fp4 | ||
| framework: vllm | ||
| multinode: false | ||
| srt-recipe-dir: minimaxm3/vllm/b200-fp4-mtp | ||
| scenarios: | ||
| agentic-coding: | ||
| - dram-utilization: 0.683 | ||
| search-space: | ||
| - { tp: 4, spec-decoding: mtp, kv-offloading: none, conc-list: [10], srt-recipe: agentic.yaml:override_cap1_k5_tp4_c10 } | ||
| - { tp: 8, spec-decoding: mtp, kv-offloading: none, conc-list: [10], srt-recipe: agentic.yaml:override_cap1_k5_tp8_c10 } | ||
| - dram-utilization: 1.0 | ||
| search-space: | ||
| - { tp: 4, spec-decoding: mtp, kv-offloading: dram, kv-offload-backend: { name: vllm-simple }, conc-list: [15, 20, 25, 30, 32, 34, 36, 38, 40], srt-recipe: agentic.yaml } | ||
| - { tp: 4, spec-decoding: mtp, kv-offloading: dram, kv-offload-backend: { name: vllm-simple }, conc-list: [20], srt-recipe: agentic.yaml:override_cap4_tp4_c20_dram } | ||
| - { tp: 4, spec-decoding: mtp, kv-offloading: dram, kv-offload-backend: { name: vllm-simple }, conc-list: [20], srt-recipe: agentic.yaml:override_cap3_tp4_c20_dram } | ||
| - { tp: 4, spec-decoding: mtp, kv-offloading: dram, kv-offload-backend: { name: vllm-simple }, conc-list: [32], srt-recipe: agentic.yaml:override_cap6_tp4_c32_dram } | ||
| - { tp: 4, spec-decoding: mtp, kv-offloading: dram, kv-offload-backend: { name: vllm-simple }, conc-list: [32], srt-recipe: agentic.yaml:override_cap16_tp4_c32_dram } | ||
|
Comment on lines
+5596
to
+5599
Contributor
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. 🔴 Two pairs of new points can't be told apart by the tooling: cap4/cap3 at c20 and cap6/cap16 at c32. Their results collide, so reuse validation and per-experiment reruns fail. The c10 key reuses TP4 with the same conc and offload settings, and only the srt-recipe override differs. Exp names (generate.py:1062) and agentic_key (results/artifacts.py:123) both ignore the override. Fix: give every point a distinct identity, for example by keeping one capped variant per (tp, conc, offload), or by adding the recipe or override to the exp name and to agentic_key. Why this was flaggedIn minimaxm3-fp4-b200-vllm-agentic-mtp-c10, nvidia-master.yaml:5550-5551 are both tp 4, conc 20, dram, vllm-simple. Lines 5552-5553 are both tp 4, conc 32, dram, vllm-simple, and so is override_tp4_c32_dram at :5531 in the main key, on the same runner cluster:b200-nscale. generate.py:1062-1066 builds exp_name only from model_code, tp, conc, offload suffix and spec, so each pair gets the same exp_name. filter_exp_names (generate.py:1437-1441) then raises "Experiment name(s) matched multiple rows" when a test-config run targets either point. For single-node runs, agentic_key (results/artifacts.py:166-180) keys on hw, model prefix, framework, precision, tp, pp, dcp, pcp, ep, dp_attention, conc and offload, not the recipe. So once a sweep finishes, validate_agentic_artifacts reports "agentic point artifacts contain N duplicate row(s)". That breaks the Verification: In configs/nvidia-master.yaml, lines 5550 and 5551 differ only in |
||
| minimaxm31flash-fp4-b200-vllm-agentic-dspark: | ||
| image: vllm/vllm-openai:minimax-m31-flash@sha256:86f582dae25ea92802353727164ae4eb03b37b18baf9119b805ffaec2a7ef07c | ||
| model: MiniMaxAI/MiniMax-M3.1-Flash | ||
|
|
@@ -5612,7 +5634,6 @@ minimaxm31flash-fp4-gb200-vllm-agentic-dspark: | |
| - search-space: | ||
| - { tp: 4, spec-decoding: mtp, kv-offloading: none, conc-list: [1, 4, 8, 16, 32], srt-recipe: agentic.yaml } | ||
| - { tp: 4, ep: 4, dp-attn: true, spec-decoding: mtp, kv-offloading: none, conc-list: [64, 128], router: { name: vllm-router, version: "0.1.14" }, srt-recipe: agentic.yaml } | ||
|
|
||
| minimaxm3-fp4-b200-trtllm-agentic-mtp: | ||
| image: nvcr.io#nvidia/tensorrt-llm/release:1.3.0rc23.post1 | ||
| model: nvidia/MiniMax-M3-NVFP4 | ||
|
|
||
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
🟡 nit (optional): The new section comments break the repo's comment policy, which bans header comments above a recipe, override or section. sweep:
^# (Low-concurrency arms|TP4 DRAM concurrency 20|CONC 32 with active)Lines 291-293 say what the next overrides are and how they are selected, so they are a section header, not a technical note. Line 377 is the same kind of comment, and its "cut to" wording also describes history. Fix: delete these blocks. If a value needs a reason, keep only a one-line note on that line, for example whymax-num-active-seqssits below CONC.Why this was flagged
AGENTS.md "Code comments (mandatory)" applies to recipes and configs. It says "Never add broad, overarching comments ... above a recipe, override, or section" and "Do not describe what the file, topology, or config is". It also bans history wording. The diff adds three such comments in agentic.yaml. Lines 291-293 begin "# Low-concurrency arms that cap ... Selected by name from the master config." Lines 333-334 begin "# TP4 DRAM concurrency 20 with 4 and 3 active sequences on the c10 key's image". Line 377 begins "# CONC 32 with active sequences capped below CONC ... the CPU KV pool is cut to the 1,073 GB node allowance." Each one describes the override group below it and does not explain a specific value. The base branch has none of them.
Verification: AGENTS.md lines 60-67 apply to recipes and say: "Never add broad, overarching comments at the top of a file or above a recipe, override, or section." The diff adds such comments in agentic.yaml at lines 291-293 and 333-334, and line 377's "cut to" is history wording. Nothing breaks at runtime; the only effect is a breach of a rule the repository wrote down.