Skip to content

[NVIDIA][AgentX] MiniMax-M3.1-Flash GB300 SGLang AgentX: day-zero recipe with low-latency c1 and L2 HiCache / [NVIDIA][AgentX] MiniMax-M3.1-Flash GB300 SGLang AgentX:首发配置,含低延迟 c1 与 L2 HiCache - #4456

Merged
edwingao28 merged 6 commits into
SemiAnalysisAI:mainfrom
zijiexia:feat/minimaxm31flash-gb300-sglang-hicache-ll
Oct 25, 2026

Conversation

@zijiexia

@zijiexia zijiexia commented Oct 24, 2026 •

Copy link
Copy Markdown
Collaborator

Summary

Day-zero MiniMax-M3.1-Flash (NVFP4) AgentX on GB300 with SGLang, using the SGLang cookbook's GB300 configs. Replaces #4443.

  • Balanced TP4: c1–c32
  • Low-Latency (MonoKernel): c1
  • High-Throughput TP4 + EP4: c32–c128, with L2 HiCache above c32
AI model disclosure
  • Model/version: Claude Opus 5.5 (claude-opus-5-5)
  • Role: wrote the recipe, master-config and perf-changelog changes and drafted this description; the author chose the settings and reviewed the diff.
PR checklist and change type

Type of Change

  • Bug fix
  • New feature
  • Configuration change
  • Documentation update
  • Other (please describe)

Checklist

  • I have completed the AI model disclosure and kept it current
  • I have tested my changes locally
  • I have updated documentation if necessary
  • If python -m infx.matrix.changed --base origin/main lists affected config keys, I have appended one new entry covering them to the physical end of inferencex-e2e/perf-changelog.yaml; if it lists none, I have added none. I have not edited historical entries
  • If this PR appends an inferencex-e2e/perf-changelog.yaml entry, it carries exactly one primary sweep label (a maintainer applies it on fork PRs): full-sweep-fail-fast (recommended), full-sweep-enabled, or non-canary-full-sweep-enabled. Optional modifiers all-evals, evals-only, and agentx-fast require a primary label; the last two block reuse while applied.
  • Before merging via reuse, an authorized maintainer (OWNER/MEMBER/COLLABORATOR) has commented /use <run_id> (or the legacy /reuse-sweep-run) on this PR. Do this only once there is a final full sweep that is all green with evals passing, since after this comment the primary sweep label will no longer automatically kick off new sweeps. Remove and re-add the primary sweep label to force a new sweep.
中文

按 SGLang cookbook 的 GB300 配置,新增 MiniMax-M3.1-Flash(NVFP4)在 GB300 上的 SGLang AgentX 首发配置,取代 #4443。

  • Balanced TP4:c1–c32
  • Low-Latency(MonoKernel):c1
  • High-Throughput TP4 + EP4:c32–c128,c32 以上启用 L2 HiCache

AI 模型使用说明:Claude Opus 5.5(claude-opus-5-5)编写了 recipe、master config 和 perf-changelog 的改动并起草本说明;参数由作者决定,diff 也经作者审阅。

🤖 Generated with Claude Code

@claude claude Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Claude Code Review

This pull request is from a fork — automated review is disabled. A repository maintainer can comment @claude review to run a one-time review.

… c1 and L2 HiCache

Day-zero MiniMax-M3.1-Flash (NVFP4) AgentX on GB300 with the SGLang
cookbook's GB300 cells: Balanced TP4 at conc 1-32, Low-Latency MSA-V1
MonoKernel at conc 1 (agentic-low-latency.yaml), and High-Throughput
TP4 + EP4 at conc 32-128 with L2 HiCache (write_back, hicache-size 175 GB
per rank) above conc 32, within the gb300-nv 920,000 MiB step memory cap.

新增 MiniMax-M3.1-Flash(NVFP4)在 GB300 上的 SGLang AgentX 首发配置,
采用 SGLang cookbook 的 GB300 配置:Balanced TP4 覆盖 conc 1-32;
Low-Latency(MSA-V1 MonoKernel)用于 conc 1(agentic-low-latency.yaml);
High-Throughput TP4 + EP4 覆盖 conc 32-128,conc 大于 32 时启用 L2 HiCache
(write_back,每个 rank 175 GB),保持在 gb300-nv 920,000 MiB 的 step 内存上限内。

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@zijiexia
zijiexia force-pushed the feat/minimaxm31flash-gb300-sglang-hicache-ll branch from 97984ec to 688df9a Compare October 24, 2026 00:25
@zijiexia zijiexia changed the title [NVIDIA][AgentX] MiniMax-M3.1-Flash GB300 SGLang: low-latency c1, HT c32 and L2 HiCache / [NVIDIA][AgentX] MiniMax-M3.1-Flash GB300 SGLang:新增低延迟 c1、HT c32 与 L2 HiCache [NVIDIA][AgentX] MiniMax-M3.1-Flash GB300 SGLang AgentX: day-zero recipe with low-latency c1 and L2 HiCache / [NVIDIA][AgentX] MiniMax-M3.1-Flash GB300 SGLang AgentX:首发配置,含低延迟 c1 与 L2 HiCache Oct 24, 2026
@Fridge003 Fridge003 added the full-sweep-fail-fast Full sweep with canary gate; first failure cancels the rest of that matrix (recommended) label Oct 24, 2026
@github-actions

Copy link
Copy Markdown
Contributor

Dispatched trusted sweep run for approved external revision 688df9a2bd6b71d74dfcd26ad8e46d7ac4697d2b.
New commits are not trusted automatically; remove and re-add the primary sweep label to approve a new SHA.

已为获批的外部提交 688df9a2bd6b71d74dfcd26ad8e46d7ac4697d2b 调度trusted sweep run。
后续新提交不会自动获得信任;如需批准新的 SHA,请移除并重新添加主扫描标签。

Resolve the perf-changelog conflict with SemiAnalysisAI#4444 via prepare_perf_changelog_merge.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@zijiexia zijiexia added full-sweep-fail-fast Full sweep with canary gate; first failure cancels the rest of that matrix (recommended) and removed full-sweep-fail-fast Full sweep with canary gate; first failure cancels the rest of that matrix (recommended) labels Oct 24, 2026
@github-actions

Copy link
Copy Markdown
Contributor

Dispatched trusted sweep run for approved external revision ce12d5f41062c56b2dac427803d3c9830769a487.
New commits are not trusted automatically; remove and re-add the primary sweep label to approve a new SHA.

已为获批的外部提交 ce12d5f41062c56b2dac427803d3c9830769a487 调度trusted sweep run。
后续新提交不会自动获得信任;如需批准新的 SHA,请移除并重新添加主扫描标签。

zijiexia and others added 2 commits October 24, 2026 16:05
Resolve the perf-changelog conflict via prepare_perf_changelog_merge.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Balanced TP4 with TRT-LLM routed MoE, scheduler and decode graph batch 128,
and write_back HiCache at 180 GB per rank.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@zijiexia zijiexia added full-sweep-fail-fast Full sweep with canary gate; first failure cancels the rest of that matrix (recommended) and removed full-sweep-fail-fast Full sweep with canary gate; first failure cancels the rest of that matrix (recommended) labels Oct 24, 2026
@github-actions

Copy link
Copy Markdown
Contributor

Dispatched trusted sweep run for approved external revision 75745991a66ccab8b759f8067b2e26b9e46eab18.
New commits are not trusted automatically; remove and re-add the primary sweep label to approve a new SHA.

已为获批的外部提交 75745991a66ccab8b759f8067b2e26b9e46eab18 调度trusted sweep run。
后续新提交不会自动获得信任;如需批准新的 SHA,请移除并重新添加主扫描标签。

Resolve the perf-changelog conflict via prepare_perf_changelog_merge.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…ride

Pair CONC with max-running-requests and cuda-graph-max-bs-decode in one
zip_override_tp4 block; every resolved recipe and matrix fingerprint is unchanged.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@edwingao28

Copy link
Copy Markdown
Collaborator

Supplementary MonoKernel C1 evaluation passed full GSM8K: 96.59% across 1,319 questions, using the same pinned image and unchanged MonoKernel recipe. Retained GPU traces confirm actual MonoKernel execution during evaluation.

@edwingao28
edwingao28 merged commit 45d4c72 into SemiAnalysisAI:main Oct 25, 2026
38 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

full-sweep-fail-fast Full sweep with canary gate; first failure cancels the rest of that matrix (recommended)

Projects

Status: Done

Development

Successfully merging this pull request may close these issues.

3 participants