Skip to content

minimaxm31flash-fp4-b200-sglang-agentic-dspark: tune DRAM offload points and add TP2 / minimaxm31flash-fp4-b200-sglang-agentic-dspark:调整 DRAM offload 点并新增 TP2 - #4536

Open
zijiexia wants to merge 12 commits into
mainfrom
minimaxm31flash-b200-sglang-c44-tp2
Open

zijiexia wants to merge 12 commits into
mainfrom
minimaxm31flash-b200-sglang-c44-tp2

Conversation

@zijiexia

Copy link
Copy Markdown
Collaborator

Tune the B200 SGLang AgentX DRAM KV offload points and add TP2 points. Builds on #4490.

中文

调整 B200 SGLang AgentX 的 DRAM KV offload 点,并新增 TP2 点。基于 #4490。

AI model disclosure

Prepared with Claude Code using claude-opus-5-5. Reviewed by the human author before submission.

🤖 Generated with Claude Code

ispobock and others added 9 commits October 25, 2026 15:00
更新 MiniMax-M3.1-Flash B200 SGLang AgentX 配置。
填写 perf-changelog 的 pr-link。
…sglang-agentx-update

# Conflicts:
#	inferencex-e2e/perf-changelog.yaml
DRAM offload 保持在分配额度内,exp-name 保持唯一。
…sglang-agentx-update

# Conflicts:
#	inferencex-e2e/perf-changelog.yaml
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
# Conflicts:
#	inferencex-e2e/perf-changelog.yaml
…nts and add TP2 / minimaxm31flash-fp4-b200-sglang-agentic-dspark:调整 DRAM offload 点并新增 TP2

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@github-actions

Copy link
Copy Markdown
Contributor

Thanks for the contribution!

  • Review: If this PR changes files owned by someone other than a repository admin or @SemiAnalysisAI/core, ask one eligible CODEOWNER to complete the latest PR_REVIEW_CHECKLIST.md before contacting a core maintainer on Slack. Follow the template exactly, including As a PR reviewer and CODEOWNER, I have reviewed this and have, so sign-off verification triggers.
  • PR verification: Sweeps run only when the PR appends an inferencex-e2e/perf-changelog.yaml entry and carries exactly one primary label: full-sweep-fail-fast (strongly recommended; canary plus per-matrix fail-fast), full-sweep-enabled (canary; matrix jobs continue after a failure), or non-canary-full-sweep-enabled (no canary or fail-fast). The modifiers all-evals, evals-only, and agentx-fast require a primary label. On fork PRs, a maintainer applies the label. See sweep labels and reuse.
  • After merging: PR authors must ensure all GitHub Actions jobs pass. Transient failures often pass on rerun; see how to rerun failed jobs.
中文

感谢你的贡献!

  • **审阅:**如果 PR 修改的文件归属于仓库管理员及 @SemiAnalysisAI/core 之外的 CODEOWNER,请先联系一位有资格的 CODEOWNER 填写最新的 PR_REVIEW_CHECKLIST.md,再通过 Slack 联系核心维护者。必须严格遵循模板,并保留 As a PR reviewer and CODEOWNER, I have reviewed this and have,才能触发签核验证。
  • **PR 验证:**只有当 PR 在 inferencex-e2e/perf-changelog.yaml 末尾追加了条目,并且恰好带有一个主标签时,才会运行扫描:full-sweep-fail-fast(强烈推荐;canary 加逐矩阵 fail-fast)、full-sweep-enabled(有 canary;矩阵任务在失败后继续运行)或 non-canary-full-sweep-enabled(无 canary,也无 fail-fast)。修饰标签 all-evals、evals-only 和 agentx-fast 必须与主标签一起使用。fork PR 的标签由维护者添加。参见扫描标签与复用。
  • **合并后:**PR 作者必须确保所有 GitHub Actions 任务通过。临时性失败通常可以通过重新运行恢复;参见重新运行失败任务的说明。

@zijiexia zijiexia added the full-sweep-fail-fast Full sweep with canary gate; first failure cancels the rest of that matrix (recommended) label Oct 26, 2026

@claude claude Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nothing blocking. The comments below are optional suggestions. There is no need to push a fix for them before merging.

Comment on lines +11214 to +11229
- config-keys:
- minimaxm31flash-fp4-b200-sglang-agentic-dspark
description:
- "Update the larger-concurrency points from TP8 + EP8 to TP4 and TP4 + EP4 with DRAM KV offload, and adjust the memory settings."
- "Add TP4 DRAM KV offload points at conc 40, 48 and 56."
pr-link: https://github.com/SemiAnalysisAI/InferenceX/pull/4490
scenario-type: [agentic-coding]

- config-keys:
- minimaxm31flash-fp4-b200-sglang-agentic-dspark
description:
- "Raise the TP4 DRAM KV offload points to mem-fraction-static 0.85 with a 16384 prefill chunk, 260 GB of HiCache per rank at DRAM utilization 1.0, and a scheduler batch of 1.5x conc."
- "Add TP4 DRAM KV offload points at conc 24 and 44, and drop the TP4 DRAM conc 64 and TP4 + EP4 conc 72 and 80 points."
- "Add TP2 points with DRAM KV offload at conc 16, 24 and 32."
pr-link: https://github.com/SemiAnalysisAI/InferenceX/pull/XXX
scenario-type: [agentic-coding]

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟡 (optional) Result readers get two changelog blocks for one config key, and they describe points this diff never had (TP4 + EP4, TP4 DRAM conc 64, conc 72/80). AGENTS.md allows one block per PR, so the second block (pr-link .../pull/4490 plus the XXX one) breaks the rule. Measured against the base, the text is also wrong: the base runs TP8 + EP8 at conc 64/128. Fix: merge both into one tail block that describes the net change from the base. That means removing the TP8 + EP8 conc 64/128 points, adding TP4 DRAM HiCache at conc 24-56 (0.85, 16384, 260 GB, 1.5x conc), and adding TP2 at conc 16/24/32.

Why this was flagged

The diff appends two blocks for minimaxm31flash-fp4-b200-sglang-agentic-dspark at perf-changelog.yaml:11214-11220 (pr-link 4490) and 11222-11229 (pr-link XXX). AGENTS.md:88 says "One PR has at most one block ... revise it rather than adding another". On the base, configs/nvidia-master.yaml has only the TP4 none row and a TP8/EP8 row at conc [64, 128], with overrides override_tp8_ep8_c64/c128. The diff swaps these for TP4 dram at [24, 32, 40, 44, 48, 56] and TP2 at [16, 24, 32]. Yet the blocks say "TP4 and TP4 + EP4" and "drop the TP4 DRAM conc 64 and TP4 + EP4 conc 72 and 80 points", and the master config and recipe contain no such rows. After merge, the append-only, never-edited record would permanently misstate what changed against main. That misleads anyone reading result collection for this key. Once 4490 merges separately, it also leaves a duplicate 4490 block.

Verification: The diff appends two tail blocks for minimaxm31flash-fp4-b200-sglang-agentic-dspark to inferencex-e2e/perf-changelog.yaml, one with pr-link: .../pull/4490 and one with pr-link: .../pull/XXX. AGENTS.md:88 says: "One PR has at most one block (listing one or more config-keys); revise it rather than adding another." No TP4+EP4 row and no TP4 DRAM conc 64/72/80 point exists in the base or in HEAD.

- dram-utilization: 1.0
search-space:
- { tp: 4, spec-decoding: mtp, kv-offloading: dram, kv-offload-backend: { name: hicache }, conc-list: [24, 32, 40, 44, 48, 56], srt-recipe: agentic.yaml }
# TP2 with L2 HiCache: 210 GB per rank at conc 16-32.

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟡 nit (optional): this comment restates the row below it (TP2, conc 16-32) and copies a recipe value (210 GB) that can drift out of sync. The TP2 override comment in agentic.yaml:247-248 also reads as a header that describes the override. Fix: delete the master-config comment, and keep only the why-text in the recipe (for example, why 210 GB fits the 536 GB share).

Why this was flagged

AGENTS.md:56 forbids header comments that describe a config, override or section, and says comments must never restate the code. The comment at nvidia-master.yaml:5642 repeats tp: 2, the HiCache backend and the conc-list [16, 24, 32] from the line below it. It also copies hicache-size 210 from the recipe, so it goes stale as soon as the recipe changes. There is no runtime impact; this is a convention slip that the diff adds.

Verification: At inferencex-e2e/configs/nvidia-master.yaml:5642 the diff adds # TP2 with L2 HiCache: 210 GB per rank at conc 16-32. directly above the tp 2 hicache row. The comment repeats tp 2, the hicache backend and the conc range from that line. AGENTS.md:58 says to "never restate the code". Nothing changes at runtime.

@github-actions

github-actions Bot commented Oct 26, 2026 •

Copy link
Copy Markdown
Contributor

…sglang-c44-tp2

# Conflicts:
#	inferencex-e2e/benchmarks/single_node/srt-slurm-recipes/minimaxm31flash/sglang/b200-fp4-mtp/agentic.yaml
#	inferencex-e2e/configs/nvidia-master.yaml
#	inferencex-e2e/perf-changelog.yaml
@ispobock ispobock added full-sweep-fail-fast Full sweep with canary gate; first failure cancels the rest of that matrix (recommended) and removed full-sweep-fail-fast Full sweep with canary gate; first failure cancels the rest of that matrix (recommended) labels Oct 26, 2026
@nvpohanh

Copy link
Copy Markdown
Collaborator

[by Claude Code] The SGLang cookbook integration of this PR's AgentX config (minimaxm31flash-fp4-b200-sglang-agentic-dspark) will happen in sgl-project/sglang#46807, in the Agentic Long-Context Deployment section of the MiniMax-M3.1-Flash cookbook page.

This PR has no authorized /use <run_id> comment yet, so the cookbook shows the recipe at this PR's head (42e9b43e9) as "in progress". A later cookbook PR switches it to the measured run once a run is pinned or this PR merges.

@nvpohanh

Copy link
Copy Markdown
Collaborator

[by Claude Code] sgl-project/sglang#46807 has been merged. This PR's AgentX config (minimaxm31flash-fp4-b200-sglang-agentic-dspark) is now in the Agentic Long-Context Deployment section of the SGLang MiniMax-M3.1-Flash cookbook: https://docs.sglang.io/cookbook/autoregressive/MiniMax/MiniMax-M3.1#agentic-long-context-deployment

This PR has no authorized /use <run_id> comment yet, so the cookbook shows the recipe at this PR's head (42e9b43e9) as "in progress". A later cookbook PR switches it to the measured run once a run is pinned or this PR merges.

@nvpohanh

Copy link
Copy Markdown
Collaborator

[by Claude Code] The SGLang cookbook update of this PR's AgentX config (minimaxm31flash-fp4-b200-sglang-agentic-dspark) will happen in sgl-project/sglang#46948, in the Agentic Long-Context Deployment section of the MiniMax-M3.1-Flash cookbook page. The config has been live in that section since sgl-project/sglang#46807; sgl-project/sglang#46948 refreshes it with the pin below.

This PR has no authorized /use <run_id> comment yet, so the cookbook shows the recipe at this PR's head (42e9b43e9) as "in progress". A later cookbook PR switches it to the measured run once a run is pinned or this PR merges.

@nvpohanh

Copy link
Copy Markdown
Collaborator

[by Claude Code] sgl-project/sglang#46948 has been merged. This PR's AgentX config (minimaxm31flash-fp4-b200-sglang-agentic-dspark) is now in the Agentic Long-Context Deployment section of the SGLang MiniMax-M3.1-Flash cookbook: https://docs.sglang.io/cookbook/autoregressive/MiniMax/MiniMax-M3.1#agentic-long-context-deployment

This PR has no authorized /use <run_id> comment yet, so the cookbook shows the recipe at this PR's head (42e9b43e9) as "in progress". A later cookbook PR switches it to the measured run once a run is pinned or this PR merges.

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

full-sweep-fail-fast Full sweep with canary gate; first failure cancels the rest of that matrix (recommended)

Projects

Status: No status

Development

Successfully merging this pull request may close these issues.

3 participants