Skip to content

[Klaud Cold] Update DeepSeek-V4-Pro FP4 B200 Dynamo-SGLang AgentX disagg image to v0.5.21-cu130 / [Klaud Cold] 将 DeepSeek-V4-Pro FP4 B200 Dynamo-SGLang AgentX 分离式镜像更新至 v0.5.21-cu130 - #4176

Merged
adibarra merged 6 commits into
mainfrom
klaud/auto-1a5ea7133ab99854-3af5706b64e41f45
Oct 21, 2026
Merged

adibarra merged 6 commits into
mainfrom
klaud/auto-1a5ea7133ab99854-3af5706b64e41f45

Conversation

@Klaud-Cold

@Klaud-Cold Klaud-Cold commented Oct 16, 2026 •

Copy link
Copy Markdown
Collaborator

Goal: Update SGLang image from lmsysorg/sglang:nightly-dev-20260916-c9a8fba9 to lmsysorg/sglang:v0.5.21-cu130@sha256:b1259f3ea3275f66237c498ea388919729018bc9f01c3d638391e06e2cf3f469.
Baseline: 2026-09-22 · lmsysorg/sglang:nightly-dev-20260916-c9a8fba9
Mean latency · Sources: API 1, API 2

Point Total tok/s/GPU ↑ Output tok/s/GPU ↑ TTFT ms ↓ TPOT ms ↓
AgentX c64 PTP8x1/DTP8x1 379a02 25,134.99 181.41 2,358.22 7.32
AgentX c128 PTP8x1/DTP8x1 6fb351 38,023.26 305.47 4,445.61 8.47
AgentX c256 PTP8x2/DTP8x1 e936ea 47,447.77 345.79 6,016.56 11.22

Eval: N/A

中文

**目标:**将 SGLang 镜像从 lmsysorg/sglang:nightly-dev-20260916-c9a8fba9 更新至 lmsysorg/sglang:v0.5.21-cu130@sha256:b1259f3ea3275f66237c498ea388919729018bc9f01c3d638391e06e2cf3f469。
**基线:**2026-09-22 · lmsysorg/sglang:nightly-dev-20260916-c9a8fba9
平均延迟 · 来源: API 1, API 2;数值及异常说明见上表。

…-cu130

Pin lmsysorg/sglang:v0.5.21-cu130 by digest for
dsv4-fp4-b200-dynamo-sglang-agentic-disagg and drop
SGLANG_OPT_SWA_SPLIT_LEAF_ON_INSERT, whose only reader was removed
upstream with the legacy SWA radix cache (sgl-project/sglang#40313).

将 dsv4-fp4-b200-dynamo-sglang-agentic-disagg 的镜像按摘要固定为
lmsysorg/sglang:v0.5.21-cu130,并移除 SGLANG_OPT_SWA_SPLIT_LEAF_ON_INSERT;
上游已随旧版 SWA radix cache 一并删除其唯一读取方(sgl-project/sglang#40313)。

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@github-actions

Copy link
Copy Markdown
Contributor

Thanks for the contribution!

  • Review: If this PR changes files owned by someone other than a repository admin or @SemiAnalysisAI/core, ask one eligible CODEOWNER to complete the latest PR_REVIEW_CHECKLIST.md before contacting a core maintainer on Slack. Follow the template exactly, including As a PR reviewer and CODEOWNER, I have reviewed this and have, so sign-off verification triggers.
  • PR verification: Sweeps run only when the PR appends an inferencex-e2e/perf-changelog.yaml entry and carries exactly one primary label: full-sweep-fail-fast (strongly recommended; canary plus per-matrix fail-fast), full-sweep-enabled (canary; matrix jobs continue after a failure), or non-canary-full-sweep-enabled (no canary or fail-fast). The modifiers all-evals, evals-only, and agentx-fast require a primary label. On fork PRs, a maintainer applies the label. See sweep labels and reuse.
  • After merging: PR authors must ensure all GitHub Actions jobs pass. Transient failures often pass on rerun; see how to rerun failed jobs.
中文

感谢你的贡献!

  • **审阅:**如果 PR 修改的文件归属于仓库管理员及 @SemiAnalysisAI/core 之外的 CODEOWNER,请先联系一位有资格的 CODEOWNER 填写最新的 PR_REVIEW_CHECKLIST.md,再通过 Slack 联系核心维护者。必须严格遵循模板,并保留 As a PR reviewer and CODEOWNER, I have reviewed this and have,才能触发签核验证。
  • **PR 验证:**只有当 PR 在 inferencex-e2e/perf-changelog.yaml 末尾追加了条目,并且恰好带有一个主标签时,才会运行扫描:full-sweep-fail-fast(强烈推荐;canary 加逐矩阵 fail-fast)、full-sweep-enabled(有 canary;矩阵任务在失败后继续运行)或 non-canary-full-sweep-enabled(无 canary,也无 fail-fast)。修饰标签 all-evals、evals-only 和 agentx-fast 必须与主标签一起使用。fork PR 的标签由维护者添加。参见扫描标签与复用。
  • **合并后:**PR 作者必须确保所有 GitHub Actions 任务通过。临时性失败通常可以通过重新运行恢复;参见重新运行失败任务的说明。

@Klaud-Cold

Klaud-Cold commented Oct 16, 2026 •

Copy link
Copy Markdown
Collaborator Author

Initial attempt · Passed · Run 38905008779 / attempt 1 · 2026-10-16 15:36 UTC
lmsysorg/sglang:v0.5.21-cu130@sha256:b1259f3ea3275f66237c498ea388919729018bc9f01c3d638391e06e2cf3f469 · 92e9f4ff96c7 · Mean latency
Change: Measure branch head f326795: pin SGLang v0.5.21 (e00930c5, 773 commits after c9a8fba9) by digest and drop SGLANG_OPT_SWA_SPLIT_LEAF_ON_INSERT, whose only reader left with the legacy SWA radix cache in sgl-project/sglang#40313; all recipe CLI flags and choices (DSPARK, megamoe, w4a4 MXFP4 MegaMoE, FP4 indexer, HiCache page_first_direct) exist unchanged at v0.5.21, upstream now defaults SGLANG_DISAGGREGATION_DEFERRED_DECODE_KV_RELEASE on (#41023), and the published baseline gsm8k em_strict from run 35345548475 was 97.65% at c64 and 97.57% at c128 and c256 over 1319 samples.

Point Output tok/s/GPU ↑ TTFT ms ↓ TPOT ms ↓
AgentX c64 PTP8x1/DTP8x1 379a02 180.28 (N/A) 2,389.57 (N/A) 7.23 (N/A)
AgentX c128 PTP8x1/DTP8x1 6fb351 286.75 (N/A) 7,245.38 (N/A) 8.39 (N/A)
AgentX c256 PTP8x2/DTP8x1 e936ea 351.2 (N/A) 5,555.54 (N/A) 11.12 (N/A)

Note: All rows: Δ N/A: dataset/scenario/statistic mismatch.

Eval Score ↑ Samples
gsm8k/em_strict · c64 97.19% (N/A) N/A/1,319 (old/new)
gsm8k/em_strict · c128 97.65% (N/A) N/A/1,319 (old/new)
gsm8k/em_strict · c256 97.04% (N/A) N/A/1,319 (old/new)

Note: All rows: Δ N/A: no matched eval baseline.

Next: Append the perf-changelog entry and run the full sweep on the updated image with full-sweep-fail-fast.

中文

初次尝试 · 已通过 · Run 38905008779 / attempt 1 · 2026-10-16 15:36 UTC
lmsysorg/sglang:v0.5.21-cu130@sha256:b1259f3ea3275f66237c498ea388919729018bc9f01c3d638391e06e2cf3f469 · 92e9f4ff96c7 · 平均延迟
**变更:**测量分支提交 f326795:按摘要固定 SGLang v0.5.21(e00930c5,比 c9a8fba9 新 773 个提交),并移除 SGLANG_OPT_SWA_SPLIT_LEAF_ON_INSERT,其唯一读取方已随旧版 SWA radix cache 在 sgl-project/sglang#40313 中删除;配方中的所有 CLI 参数和取值(DSPARK、megamoe、w4a4 MXFP4 MegaMoE、FP4 indexer、HiCache page_first_direct)在 v0.5.21 中均保持不变,上游现已默认开启 SGLANG_DISAGGREGATION_DEFERRED_DECODE_KV_RELEASE(#41023),已发布基线运行 35345548475 的 gsm8k em_strict 在 c64 为 97.65%,在 c128 和 c256 为 97.57%,样本数 1319。实测数值及异常说明见上表。
**下一步:**追加 perf-changelog 条目,并使用 full-sweep-fail-fast 在新镜像上运行完整扫描。

github-actions Bot and others added 2 commits October 16, 2026 15:36
合并 origin/main。

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…date

Append the perf-changelog entry for
dsv4-fp4-b200-dynamo-sglang-agentic-disagg.

为 dsv4-fp4-b200-dynamo-sglang-agentic-disagg 追加 perf-changelog 条目。

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@Klaud-Cold Klaud-Cold added the full-sweep-fail-fast Full sweep with canary gate; first failure cancels the rest of that matrix (recommended) label Oct 16, 2026
@Klaud-Cold

Klaud-Cold commented Oct 16, 2026 •

Copy link
Copy Markdown
Collaborator Author

Final full sweep · Passed · Run 38925477096 / attempt 1 · 2026-10-16 18:27 UTC
lmsysorg/sglang:v0.5.21-cu130@sha256:b1259f3ea3275f66237c498ea388919729018bc9f01c3d638391e06e2cf3f469 · 8b1ac122ff6b · Mean latency
Change: Validate the complete updated-image family at this head.

Point Output tok/s/GPU ↑ TTFT ms ↓ TPOT ms ↓
AgentX c64 PTP8x1/DTP8x1 379a02 180.12 (N/A) 2,333.82 (N/A) 7.2 (N/A)
AgentX c128 PTP8x1/DTP8x1 6fb351 276.88 (N/A) 8,990.59 (N/A) 8.32 (N/A)
AgentX c256 PTP8x2/DTP8x1 e936ea 348.12 (N/A) 5,528.19 (N/A) 11.19 (N/A)

Note: All rows: Δ N/A: dataset/scenario/statistic mismatch.

Eval Score ↑ Samples
gsm8k/em_strict · c64 97.04% (N/A) N/A/1,319 (old/new)
gsm8k/em_strict · c128 97.57% (N/A) N/A/1,319 (old/new)
gsm8k/em_strict · c256 97.35% (N/A) N/A/1,319 (old/new)

Note: All rows: Δ N/A: no matched eval baseline.

Next: Mark ready for maintainer review.

中文

最终完整 sweep · 已通过 · Run 38925477096 / attempt 1 · 2026-10-16 18:27 UTC
lmsysorg/sglang:v0.5.21-cu130@sha256:b1259f3ea3275f66237c498ea388919729018bc9f01c3d638391e06e2cf3f469 · 8b1ac122ff6b · 平均延迟
**变更:**验证此提交更新镜像后的完整配置族。实测数值及异常说明见上表。
**下一步:**标记为就绪,等待维护者审查。

@github-actions

Copy link
Copy Markdown
Contributor

@Klaud-Cold
Klaud-Cold marked this pull request as ready for review October 16, 2026 18:27
@Klaud-Cold
Klaud-Cold requested a review from a team October 16, 2026 18:27
@Klaud-Cold

Copy link
Copy Markdown
Collaborator Author

validated · Repairs: 0 · Runs: 38905008779, 38925392693, 38925477096
All owned runs ended. Full sweep verified; ready for review.

中文

validated · 修复次数:0 · 运行:38905008779, 38925392693, 38925477096
所有自有运行均已结束。完整 sweep 已验证;已就绪,等待审查。

@Klaud-Cold

Copy link
Copy Markdown
Collaborator Author

/use 38925477096

@github-actions

github-actions Bot commented Oct 16, 2026 •

Copy link
Copy Markdown
Contributor

@Klaud-Cold staged run 38925477096.

This run remains available across future /stage-results requests. Staging the same run ID again updates its staged data. Staging workflow

@claude claude Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I reviewed this PR and didn't find any bugs. The NVIDIA master config and the B200 srt-slurm recipes have designated code owners, so one of them should still sign off.

What was reviewed:

  • The image bump for dsv4-fp4-b200-dynamo-sglang-agentic-disagg. The digest is pinned, and it differs from the ARM64 v0.5.21-cu130 digest the GB300 Qwen entry uses, which is what you'd expect for an x86 B200 runner.
  • The SGLANG_OPT_SWA_SPLIT_LEAF_ON_INSERT removal from both worker env blocks of the B200 disagg recipe. The var is still in the B200 agg recipe and the GB200/GB300 recipes, but those use other config keys and images, so this PR doesn't affect them.
  • The perf-changelog entry. It only adds lines at the end of the file, covers the one affected key, and correctly leaves out append-only, since this change alters existing points.
Extended reasoning...

The change touches three YAML files: an image tag and digest in nvidia-master.yaml, two env-var removals in one B200 FP4 AgentX disagg recipe, and a new perf-changelog entry. It has no security-sensitive surface. The change is small and mechanical, but CODEOWNERS assigns both nvidia-master.yaml and the b200-* srt-slurm recipes to specific owners, so approval is left to them.

Resolve perf-changelog.yaml by taking main's file and re-appending this PR's entry.

中文:合并 origin/main;perf-changelog.yaml 采用 main 版本,并在末尾重新追加本 PR 的条目。
Resolve conflicts: take main's refactored recipes and re-apply this PR's non-image edits; take main's perf-changelog.yaml and re-append this PR's entry.

中文:合并 origin/main;配方采用 main 重构后的版本并重新应用本 PR 的非镜像改动,perf-changelog.yaml 采用 main 版本并在末尾重新追加本 PR 的条目。
@adibarra

Copy link
Copy Markdown
Collaborator

/reuse-sweep-run 38925477096

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

full-sweep-fail-fast Full sweep with canary gate; first failure cancels the rest of that matrix (recommended)

Projects

Status: Done

Development

Successfully merging this pull request may close these issues.

3 participants