Repository navigation
minimaxm31flash-fp4-b200-sglang-agentic-dspark: tune DRAM offload points and add TP2 / minimaxm31flash-fp4-b200-sglang-agentic-dspark:调整 DRAM offload 点并新增 TP2 - #4536
Conversation
更新 MiniMax-M3.1-Flash B200 SGLang AgentX 配置。
填写 perf-changelog 的 pr-link。
…sglang-agentx-update # Conflicts: # inferencex-e2e/perf-changelog.yaml
DRAM offload 保持在分配额度内,exp-name 保持唯一。
…sglang-agentx-update # Conflicts: # inferencex-e2e/perf-changelog.yaml
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
# Conflicts: # inferencex-e2e/perf-changelog.yaml
…nts and add TP2 / minimaxm31flash-fp4-b200-sglang-agentic-dspark:调整 DRAM offload 点并新增 TP2 Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
|
Thanks for the contribution!
中文感谢你的贡献!
|
| - config-keys: | ||
| - minimaxm31flash-fp4-b200-sglang-agentic-dspark | ||
| description: | ||
| - "Update the larger-concurrency points from TP8 + EP8 to TP4 and TP4 + EP4 with DRAM KV offload, and adjust the memory settings." | ||
| - "Add TP4 DRAM KV offload points at conc 40, 48 and 56." | ||
| pr-link: https://github.com/SemiAnalysisAI/InferenceX/pull/4490 | ||
| scenario-type: [agentic-coding] | ||
|
|
||
| - config-keys: | ||
| - minimaxm31flash-fp4-b200-sglang-agentic-dspark | ||
| description: | ||
| - "Raise the TP4 DRAM KV offload points to mem-fraction-static 0.85 with a 16384 prefill chunk, 260 GB of HiCache per rank at DRAM utilization 1.0, and a scheduler batch of 1.5x conc." | ||
| - "Add TP4 DRAM KV offload points at conc 24 and 44, and drop the TP4 DRAM conc 64 and TP4 + EP4 conc 72 and 80 points." | ||
| - "Add TP2 points with DRAM KV offload at conc 16, 24 and 32." | ||
| pr-link: https://github.com/SemiAnalysisAI/InferenceX/pull/XXX | ||
| scenario-type: [agentic-coding] |
There was a problem hiding this comment.
🟡 (optional) Result readers get two changelog blocks for one config key, and they describe points this diff never had (TP4 + EP4, TP4 DRAM conc 64, conc 72/80). AGENTS.md allows one block per PR, so the second block (pr-link .../pull/4490 plus the XXX one) breaks the rule. Measured against the base, the text is also wrong: the base runs TP8 + EP8 at conc 64/128. Fix: merge both into one tail block that describes the net change from the base. That means removing the TP8 + EP8 conc 64/128 points, adding TP4 DRAM HiCache at conc 24-56 (0.85, 16384, 260 GB, 1.5x conc), and adding TP2 at conc 16/24/32.
Why this was flagged
The diff appends two blocks for minimaxm31flash-fp4-b200-sglang-agentic-dspark at perf-changelog.yaml:11214-11220 (pr-link 4490) and 11222-11229 (pr-link XXX). AGENTS.md:88 says "One PR has at most one block ... revise it rather than adding another". On the base, configs/nvidia-master.yaml has only the TP4 none row and a TP8/EP8 row at conc [64, 128], with overrides override_tp8_ep8_c64/c128. The diff swaps these for TP4 dram at [24, 32, 40, 44, 48, 56] and TP2 at [16, 24, 32]. Yet the blocks say "TP4 and TP4 + EP4" and "drop the TP4 DRAM conc 64 and TP4 + EP4 conc 72 and 80 points", and the master config and recipe contain no such rows. After merge, the append-only, never-edited record would permanently misstate what changed against main. That misleads anyone reading result collection for this key. Once 4490 merges separately, it also leaves a duplicate 4490 block.
Verification: The diff appends two tail blocks for minimaxm31flash-fp4-b200-sglang-agentic-dspark to inferencex-e2e/perf-changelog.yaml, one with pr-link: .../pull/4490 and one with pr-link: .../pull/XXX. AGENTS.md:88 says: "One PR has at most one block (listing one or more config-keys); revise it rather than adding another." No TP4+EP4 row and no TP4 DRAM conc 64/72/80 point exists in the base or in HEAD.
| - dram-utilization: 1.0 | ||
| search-space: | ||
| - { tp: 4, spec-decoding: mtp, kv-offloading: dram, kv-offload-backend: { name: hicache }, conc-list: [24, 32, 40, 44, 48, 56], srt-recipe: agentic.yaml } | ||
| # TP2 with L2 HiCache: 210 GB per rank at conc 16-32. |
There was a problem hiding this comment.
🟡 nit (optional): this comment restates the row below it (TP2, conc 16-32) and copies a recipe value (210 GB) that can drift out of sync. The TP2 override comment in agentic.yaml:247-248 also reads as a header that describes the override. Fix: delete the master-config comment, and keep only the why-text in the recipe (for example, why 210 GB fits the 536 GB share).
Why this was flagged
AGENTS.md:56 forbids header comments that describe a config, override or section, and says comments must never restate the code. The comment at nvidia-master.yaml:5642 repeats tp: 2, the HiCache backend and the conc-list [16, 24, 32] from the line below it. It also copies hicache-size 210 from the recipe, so it goes stale as soon as the recipe changes. There is no runtime impact; this is a convention slip that the diff adds.
Verification: At inferencex-e2e/configs/nvidia-master.yaml:5642 the diff adds # TP2 with L2 HiCache: 210 GB per rank at conc 16-32. directly above the tp 2 hicache row. The comment repeats tp 2, the hicache backend and the conc range from that line. AGENTS.md:58 says to "never restate the code". Nothing changes at runtime.
|
View unofficial run (performance): https://inferencex.semianalysis.com/inference?unofficialRun=40336087375 View unofficial run (accuracy): https://inferencex.semianalysis.com/evaluation?unofficialRun=40336087375 |
…sglang-c44-tp2 # Conflicts: # inferencex-e2e/benchmarks/single_node/srt-slurm-recipes/minimaxm31flash/sglang/b200-fp4-mtp/agentic.yaml # inferencex-e2e/configs/nvidia-master.yaml # inferencex-e2e/perf-changelog.yaml
|
[by Claude Code] The SGLang cookbook integration of this PR's AgentX config ( This PR has no authorized |
|
[by Claude Code] sgl-project/sglang#46807 has been merged. This PR's AgentX config ( This PR has no authorized |
|
[by Claude Code] The SGLang cookbook update of this PR's AgentX config ( This PR has no authorized |
|
[by Claude Code] sgl-project/sglang#46948 has been merged. This PR's AgentX config ( This PR has no authorized |
Tune the B200 SGLang AgentX DRAM KV offload points and add TP2 points. Builds on #4490.
中文
调整 B200 SGLang AgentX 的 DRAM KV offload 点,并新增 TP2 点。基于 #4490。
AI model disclosure
Prepared with Claude Code using
claude-opus-5-5. Reviewed by the human author before submission.🤖 Generated with Claude Code