Skip to content

[AgentX] MiniMax-M3.1-Flash GB200 SGLang: update config / [AgentX] MiniMax-M3.1-Flash GB200 SGLang:更新配置 - #4695

Open
zijiexia wants to merge 43 commits into
mainfrom
minimaxm31flash-gb200-sglang-pd
Open

zijiexia wants to merge 43 commits into
mainfrom
minimaxm31flash-gb200-sglang-pd

Conversation

@zijiexia

Copy link
Copy Markdown
Collaborator

Update config.

中文

更新配置。

AI model disclosure

Prepared with Claude Opus 5.5 (claude-opus-5-5).

🤖 Generated with Claude Code

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@zijiexia
zijiexia requested a review from RohitNagraj as a code owner October 31, 2026 20:11
@github-actions

Copy link
Copy Markdown
Contributor

Thanks for the contribution!

  • Review: If this PR changes files owned by someone other than a repository admin or @SemiAnalysisAI/core, ask one eligible CODEOWNER to complete the latest PR_REVIEW_CHECKLIST.md before contacting a core maintainer on Slack. Follow the template exactly, including As a PR reviewer and CODEOWNER, I have reviewed this and have, so sign-off verification triggers.
  • PR verification: Sweeps run only when the PR appends an inferencex-e2e/perf-changelog.yaml entry and carries exactly one primary label: full-sweep-fail-fast (strongly recommended; canary plus per-matrix fail-fast), full-sweep-enabled (canary; matrix jobs continue after a failure), or non-canary-full-sweep-enabled (no canary or fail-fast). The modifiers all-evals, evals-only, and agentx-fast require a primary label. On fork PRs, a maintainer applies the label. See sweep labels and reuse.
  • After merging: PR authors must ensure all GitHub Actions jobs pass. Transient failures often pass on rerun; see how to rerun failed jobs.
中文

感谢你的贡献!

  • **审阅:**如果 PR 修改的文件归属于仓库管理员及 @SemiAnalysisAI/core 之外的 CODEOWNER,请先联系一位有资格的 CODEOWNER 填写最新的 PR_REVIEW_CHECKLIST.md,再通过 Slack 联系核心维护者。必须严格遵循模板,并保留 As a PR reviewer and CODEOWNER, I have reviewed this and have,才能触发签核验证。
  • **PR 验证:**只有当 PR 在 inferencex-e2e/perf-changelog.yaml 末尾追加了条目,并且恰好带有一个主标签时,才会运行扫描:full-sweep-fail-fast(强烈推荐;canary 加逐矩阵 fail-fast)、full-sweep-enabled(有 canary;矩阵任务在失败后继续运行)或 non-canary-full-sweep-enabled(无 canary,也无 fail-fast)。修饰标签 all-evals、evals-only 和 agentx-fast 必须与主标签一起使用。fork PR 的标签由维护者添加。参见扫描标签与复用。
  • **合并后:**PR 作者必须确保所有 GitHub Actions 任务通过。临时性失败通常可以通过重新运行恢复;参见重新运行失败任务的说明。

@zijiexia zijiexia added full-sweep-fail-fast Full sweep with canary gate; first failure cancels the rest of that matrix (recommended) and removed full-sweep-fail-fast Full sweep with canary gate; first failure cancels the rest of that matrix (recommended) labels Oct 31, 2026

@claude claude Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Findings marked 🟡 are optional suggestions and need no follow-up push.

Comment thread inferencex-e2e/perf-changelog.yaml Outdated
- "Add disaggregated SGLang behind Dynamo for MiniMax-M3.1-Flash AgentX on GB200 with lmsysorg/sglang:nightly-cu134-20261030-e371881 (Dynamo 1.7.0.dev20261030, Mooncake transfer, minimax-msa kernels)."
- "2 DEP4 prefill workers with a DEP8 decode at conc 224 and 256, or 3 DEP4 prefill workers with a DEP16 decode at conc 320 and 360; MegaMoE, DSpark block 7 with fp8 draft KV, Dynamo KV router with fastokens for prefill and least-loaded routing across decode DP ranks."
- "Prefill: shortest-prefill-first, chunk 8192, mem-fraction-static 0.85, HiCache 180 GB per rank write_back."
pr-link: https://github.com/SemiAnalysisAI/InferenceX/pull/XXXX

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🔴 The changelog check will fail on this PR, so the new config never gets scheduled. Its pr-link ends in pull/XXXX, with four X's. The validator only accepts XXX, https://github.com/SemiAnalysisAI/InferenceX/pull/XXX, or the PR's real URL (validate_perf_changelog.py:21-24). Fix: set pr-link to this PR's real URL, or to the exact .../pull/XXX placeholder.

Why this was flagged

The new entry at inferencex-e2e/perf-changelog.yaml:11750 sets pr-link: https://github.com/SemiAnalysisAI/InferenceX/pull/XXXX. On PR runs, validate_added_pr_link (inferencex-e2e/infx/workflows/validate_perf_changelog.py:105-116) accepts only a link in PR_LINK_PLACEHOLDERS ("XXX" or "https://github.com/SemiAnalysisAI/InferenceX/pull/XXX") or the PR's own URL. Four X's matches none of these, so it raises "new PR entry must use ... or an XXX placeholder". On main, CANONICAL_PR_LINK needs pull/\d+ and rejects it too. The changelog validation fails, so the sweep for minimaxm31flash-fp4-gb200-dynamo-sglang-agentic-dspark-disagg is never scheduled and the PR cannot get the /use <run_id> acceptance it needs.

Verification: Trigger: the PR goes through merge_with_reuse. The entry at inferencex-e2e/perf-changelog.yaml:11750 uses pull/XXXX, which matches neither placeholder (validate_perf_changelog.py:21-24). merge_with_reuse.py:688 calls canonicalize_changelog(pr, repo) and does not catch the exception, so the merge aborts. The PR-run changelog check does not call this validator.

multinode: true
disagg: true
srt-recipe-dir: minimaxm31flash/sglang/gb200-fp4/agentx
scenarios:

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟡 (optional) Published results for this config will claim 901 GB of host KV per node, while the recipe only pins 720 GB. dram-utilization: 1.0 makes matrix generation record the whole gb200-nv node share (860_160 MiB, about 901 GB) as total-cpu-dram-gb. The prefill recipe pins hicache-size: 180 on 4 ranks, which is 720 GB. Fix: set dram-utilization: 0.80, which resolves to 721 GB. The single-node sibling minimaxm31flash-fp4-gb200-sglang-agentic-dspark uses that value for the same 180 GB-per-rank pool.

Why this was flagged

agentic_dram_offload_gb (infx/matrix/generate.py:475-518) computes the prefill share as available-cpu-dram-mib 860_160 (runners.yaml:572) × utilization × 4/4 GPUs. With 1.0 that comes to 901 GB. The value reaches TOTAL_CPU_DRAM_GB through infx/srt_slurm/generate.py:59. It is published as allocated_cpu_dram_gb (infx/results/agentic/init.py:205). The decode side also gets 901 GB as decode_allocated_cpu_dram_gb, even though decode runs no HiCache. The recipe really allocates 180 GB × 4 ranks = 720 GB per prefill node; its own comment says "720 of the node's ~900 GB host allowance". The same-cluster, same-pool sibling uses dram-utilization 0.80. Its recipe records TOTAL_CPU_DRAM_GB '721' (benchmarks/single_node/.../minimaxm31flash/sglang/gb200-fp4-mtp/agentic.yaml:148). So results with identical host pools would report different DRAM budgets, and this one overstates its pool by about 25%. Nothing checks the recorded value against hicache-size, because the recipe never reads TOTAL_CPU_DRAM_GB.

Verification: nit. This happens on every DRAM-offload point of the new config. In agentic_dram_offload_gb, share_gb returns 860160 MiB × 1.0 × 4/4, which is about 901 GB. infx/results/agentic/__init__.py:205 publishes this as allocated_cpu_dram_gb. The recipe really pins hicache-size: 180 on 4 ranks per prefill node, which is 720 GB. Nothing fails at runtime.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@github-actions

github-actions Bot commented Oct 31, 2026 •

Copy link
Copy Markdown
Contributor

zijiexia and others added 6 commits October 31, 2026 18:23
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…-sglang-pd

# Conflicts:
#	inferencex-e2e/perf-changelog.yaml
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…-sglang-pd

# Conflicts:
#	inferencex-e2e/perf-changelog.yaml
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@nvpohanh

nvpohanh commented Nov 2, 2026

Copy link
Copy Markdown
Collaborator

/use 41271615328

@github-actions

github-actions Bot commented Nov 2, 2026

Copy link
Copy Markdown
Contributor

@nvpohanh staging run 41271615328. Existing staged runs will be preserved; a completion link will be posted here.

@nvpohanh

nvpohanh commented Nov 2, 2026

Copy link
Copy Markdown
Collaborator

[by Claude Code] The SGLang cookbook integration of this PR's AgentX configs (minimaxm31flash-fp4-gb200-dynamo-sglang-agentic-dspark-agg, minimaxm31flash-fp4-gb200-dynamo-sglang-agentic-dspark-disagg) will happen in sgl-project/sglang#47929, in the Agentic Long-Context Deployment section of the MiniMax-M3.1-Flash cookbook page.

The commands mirror the launch measured in run 41271615328, pinned by this PR's /use comment.

zijiexia and others added 2 commits November 2, 2026 17:27
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@nvpohanh

nvpohanh commented Nov 3, 2026

Copy link
Copy Markdown
Collaborator

[by Claude Code] sgl-project/sglang#47929 has been merged. This PR's AgentX configs (minimaxm31flash-fp4-gb200-dynamo-sglang-agentic-dspark-agg, minimaxm31flash-fp4-gb200-dynamo-sglang-agentic-dspark-disagg) are now in the Agentic Long-Context Deployment section of the SGLang MiniMax-M3.1-Flash cookbook: https://docs.sglang.io/cookbook/autoregressive/MiniMax/MiniMax-M3.1#agentic-long-context-deployment

The commands mirror the launch measured in run 41271615328, pinned by this PR's /use comment.

zijiexia and others added 16 commits November 12, 2026 23:27
…nal linker on the 2026-11-12 nightly

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…ts L3 share in decode-node stores

The linker needs the decode radix cache, which DP attention marks experimental.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…9 nightlies to bisect a decode slowdown

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…-sglang-l3

# Conflicts:
#	inferencex-e2e/perf-changelog.yaml
The image's bundled Dynamo imports a class its SGLang does not have yet.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…isect points

Their bundled Dynamo imports MultimodalProcessorOutput from schedule_batch,
which SGLang moved to req_types on 11-02.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…nd 11-02 nightlies

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…-deep-gemm 0.2.3.post2

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
… DP rank (1P DEP8 chunk 131072, 2P DEP4); drop the deep-gemm downgrade point

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…-sglang-l3

# Conflicts:
#	inferencex-e2e/perf-changelog.yaml
…ax linker points with 16384 tokens per DP rank

Both points OOMed during prefill startup at 0.88.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…stances and no attention DP

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…-sglang-l3

# Conflicts:
#	inferencex-e2e/perf-changelog.yaml
… decode

MiniMax sparse PD requires the same attention TP on prefill and decode; the
DEP8 decode (attention TP 1) failed against TP2 prefill.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…1111

Every fast decode point ran Dynamo 1.7.0.dev20261030 and every slow one a newer
Dynamo, so this separates Dynamo from the SGLang commits in b45a365..40a64e1.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…1030

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@zijiexia
zijiexia requested a review from cquil11 as a code owner November 16, 2026 00:07
zijiexia and others added 5 commits November 15, 2026 18:02
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…-sglang-l3

# Conflicts:
#	inferencex-e2e/perf-changelog.yaml
…s on the linker

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…Kernel c1 point and drop the diagnostic points

The linker point installs Dynamo 1.7.0.dev20261030 over the 11-12 image's
1.7.0.dev20261111, which decodes about 0.75 ms per step slower, and adds
conc 256. The agg key gets the TP4 MSA-V1 MonoKernel point at conc 1 that
GB300 runs. The nightly bisect keys and the losing HiCache, L3 and linker
variants are removed.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…-sglang-l3

# Conflicts:
#	inferencex-e2e/perf-changelog.yaml
@zijiexia zijiexia added full-sweep-fail-fast Full sweep with canary gate; first failure cancels the rest of that matrix (recommended) and removed full-sweep-fail-fast Full sweep with canary gate; first failure cancels the rest of that matrix (recommended) labels Nov 16, 2026
…e or two DEP8 prefill workers

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@zijiexia zijiexia added full-sweep-fail-fast Full sweep with canary gate; first failure cancels the rest of that matrix (recommended) and removed full-sweep-fail-fast Full sweep with canary gate; first failure cancels the rest of that matrix (recommended) labels Nov 16, 2026
…oint

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@zijiexia zijiexia added full-sweep-fail-fast Full sweep with canary gate; first failure cancels the rest of that matrix (recommended) and removed full-sweep-fail-fast Full sweep with canary gate; first failure cancels the rest of that matrix (recommended) labels Nov 16, 2026

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

full-sweep-fail-fast Full sweep with canary gate; first failure cancels the rest of that matrix (recommended)

Projects

Status: No status

Development

Successfully merging this pull request may close these issues.

2 participants