Skip to content

[Klaud Cold] Update GLM-5.2 FP8 B200 SGLang AgentX MTP image to nightly-dev-20261024-5994c559 / [Klaud Cold] 将 GLM-5.2 FP8 B200 SGLang AgentX MTP 镜像更新至 nightly-dev-20261024-5994c559 - #4471

Closed
Klaud-Cold wants to merge 2 commits into
mainfrom
klaud/auto-fb56d2a358d2743c-0dd16aeb42d3dd5b
Closed

Klaud-Cold wants to merge 2 commits into
mainfrom
klaud/auto-fb56d2a358d2743c-0dd16aeb42d3dd5b

Conversation

@Klaud-Cold

@Klaud-Cold Klaud-Cold commented Oct 24, 2026 •

Copy link
Copy Markdown
Collaborator

Goal: Update SGLang image from lmsysorg/sglang:nightly-dev-20261020-ab580b2e@sha256:0af411bb320a95bb5e1c58b4c20c9852920a19aa9f12f148f69731b0dca3b576 to lmsysorg/sglang:nightly-dev-20261024-5994c559@sha256:40da9cac78a558237426971c048ab0f4f1a4b13bfb1b6e7a48d4d980e4163c8e.
Baseline: 2026-10-22 · lmsysorg/sglang:nightly-dev-20261020-ab580b2e@sha256:0af411bb320a95bb5e1c58b4c20c9852920a19aa9f12f148f69731b0dca3b576
AgentX · TP8/EP1 · Mean latency · Sources: API 1, API 2, API 3, API 4

Concurrency Total tok/s/GPU ↑ Output tok/s/GPU ↑ TTFT ms ↓ TPOT ms ↓
1 2,433.44 17.84 611.6 3.51
4 3,737.18 26.21 422.47 4.12
8 6,566.69 48.42 511.95 5.13
12 7,674.6 60.12 934.38 7.13
16 10,166.66 72.68 1,301.32 10.43
17 10,945.66 78.82 1,440.64 10.94
18 12,172.31 85.75 1,448.3 12.69
24 13,884.74 97.23 1,818.06 16.46
Eval Score ↑ Samples
gsm8k/em_strict · c24 96.66% 1,319

Stop: Add the klaud-abort label at any time to stop this candidate now; Klaud cancels its runs, closes this PR and does not retry it. Remove the label to let the planner pick it up again.

中文

**目标:**将 SGLang 镜像从 lmsysorg/sglang:nightly-dev-20261020-ab580b2e@sha256:0af411bb320a95bb5e1c58b4c20c9852920a19aa9f12f148f69731b0dca3b576 更新至 lmsysorg/sglang:nightly-dev-20261024-5994c559@sha256:40da9cac78a558237426971c048ab0f4f1a4b13bfb1b6e7a48d4d980e4163c8e。
**基线:**2026-10-22 · lmsysorg/sglang:nightly-dev-20261020-ab580b2e@sha256:0af411bb320a95bb5e1c58b4c20c9852920a19aa9f12f148f69731b0dca3b576
AgentX · TP8/EP1 · 平均延迟 · 来源: API 1, API 2, API 3, API 4;数值及异常说明见上表。
**停止:**随时添加 klaud-abort 标签即可立即停止该候选:Klaud 会取消其运行、关闭此 PR 且不再重试。移除该标签后,规划器会在下次扫描时重新考虑它。

…-20261024-5994c559

将 GLM-5.2 FP8 B200 SGLang AgentX MTP 镜像更新至 nightly-dev-20261024-5994c559。

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@github-actions

Copy link
Copy Markdown
Contributor

Thanks for the contribution!

  • Review: If this PR changes files owned by someone other than a repository admin or @SemiAnalysisAI/core, ask one eligible CODEOWNER to complete the latest PR_REVIEW_CHECKLIST.md before contacting a core maintainer on Slack. Follow the template exactly, including As a PR reviewer and CODEOWNER, I have reviewed this and have, so sign-off verification triggers.
  • PR verification: Sweeps run only when the PR appends an inferencex-e2e/perf-changelog.yaml entry and carries exactly one primary label: full-sweep-fail-fast (strongly recommended; canary plus per-matrix fail-fast), full-sweep-enabled (canary; matrix jobs continue after a failure), or non-canary-full-sweep-enabled (no canary or fail-fast). The modifiers all-evals, evals-only, and agentx-fast require a primary label. On fork PRs, a maintainer applies the label. See sweep labels and reuse.
  • After merging: PR authors must ensure all GitHub Actions jobs pass. Transient failures often pass on rerun; see how to rerun failed jobs.
中文

感谢你的贡献!

  • **审阅:**如果 PR 修改的文件归属于仓库管理员及 @SemiAnalysisAI/core 之外的 CODEOWNER,请先联系一位有资格的 CODEOWNER 填写最新的 PR_REVIEW_CHECKLIST.md,再通过 Slack 联系核心维护者。必须严格遵循模板,并保留 As a PR reviewer and CODEOWNER, I have reviewed this and have,才能触发签核验证。
  • **PR 验证:**只有当 PR 在 inferencex-e2e/perf-changelog.yaml 末尾追加了条目,并且恰好带有一个主标签时,才会运行扫描:full-sweep-fail-fast(强烈推荐;canary 加逐矩阵 fail-fast)、full-sweep-enabled(有 canary;矩阵任务在失败后继续运行)或 non-canary-full-sweep-enabled(无 canary,也无 fail-fast)。修饰标签 all-evals、evals-only 和 agentx-fast 必须与主标签一起使用。fork PR 的标签由维护者添加。参见扫描标签与复用。
  • **合并后:**PR 作者必须确保所有 GitHub Actions 任务通过。临时性失败通常可以通过重新运行恢复;参见重新运行失败任务的说明。

@Klaud-Cold

Klaud-Cold commented Oct 24, 2026 •

Copy link
Copy Markdown
Collaborator Author

Initial attempt · Passed · Run 40121743981 / attempt 1 · 2026-10-24 20:18 UTC
lmsysorg/sglang:nightly-dev-20261024-5994c559@sha256:40da9cac78a558237426971c048ab0f4f1a4b13bfb1b6e7a48d4d980e4163c8e · a0287f0b3617 · AgentX · TP8/EP1 · Mean latency
Change: Bump the master image only; the SGLang diff ab580b2e..5994c559 (sgl-project/sglang@ab580b2...5994c55) keeps every recipe flag, the glm47/glm45 parsers, CUDA 13.0.3, FlashInfer 0.7.2, sgl-kernel 0.4.9 and NCCL 2.30.7, while adding a GLM-5.2 CUDA-graph side-stream fix, a HiCache write-through race fix, MLA prefill graph max_bs min(8192, chunked) and NCCL graph-buffer registration off for multi-rank capture, whose performance effect is unmeasured.

Concurrency Output tok/s/GPU ↑ TTFT ms ↓ TPOT ms ↓
1 17.97 (+0.7%) 614.56 (+0.5%) 3.47 (-1.1%)
4 26.68 (+1.8%) 428.7 (+1.5%) 4 (-2.9%)
8 48.27 (-0.3%) 522.77 (+2.1%) 5.29 (+3.1%)
12 61.24 (+1.9%) 948.34 (+1.5%) 7.15 (+0.3%)
16 72.71 (+0.1%) 1,259.84 (-3.2%) 10.2 (-2.2%)
17 79.19 (+0.5%) 1,415.78 (-1.7%) 10.86 (-0.7%)
24 97.22 (-0%) 1,920.59 (+5.6%) 17.58 (+6.8%)

Note: c4, c24: 2 request errors.
Note: c8, c16, c17: 1 request error.
Note: c12: 3 request errors.

Eval Score ↑ Samples
gsm8k/em_strict · c24 92.12% (N/A) 1,319

Note: All rows: Δ N/A: no matched eval baseline.

Next: Smoke benchmarks and GSM8K c24 pass, but 68 of 1319 GSM8K questions (both repeats) return only '!' up to the 16384-token cap with identical resolved server args, so repair 1 tests NCCL_GRAPH_REGISTER=1, which the new image otherwise forces to 0 for multi-rank graph capture.

中文

初次尝试 · 已通过 · Run 40121743981 / attempt 1 · 2026-10-24 20:18 UTC
lmsysorg/sglang:nightly-dev-20261024-5994c559@sha256:40da9cac78a558237426971c048ab0f4f1a4b13bfb1b6e7a48d4d980e4163c8e · a0287f0b3617 · AgentX · TP8/EP1 · 平均延迟
**变更:**仅更新主配置镜像;SGLang 差异 ab580b2e..5994c559 (sgl-project/sglang@ab580b2...5994c55) 保留所有配方参数、glm47/glm45 解析器、CUDA 13.0.3、FlashInfer 0.7.2、sgl-kernel 0.4.9 与 NCCL 2.30.7,同时加入 GLM-5.2 CUDA graph 侧流修复、HiCache write-through 竞争修复、MLA prefill graph max_bs 改为 min(8192, chunked),以及多卡捕获时关闭 NCCL graph buffer 注册,其性能影响尚未测量。实测数值及异常说明见上表。
**下一步:**冒烟基准和 GSM8K c24 均通过,但在已解析 server args 相同的情况下,1319 道 GSM8K 题中有 68 道(两次重复均是)只输出 '!' 直到 16384 token 上限,因此修复 1 测试 NCCL_GRAPH_REGISTER=1,新镜像在多卡 graph 捕获时会将其强制设为 0。

…tX MTP

SGLang 5994c559 forces NCCL_GRAPH_REGISTER=0 for any multi-rank graph
capture; with it the DCP8 c24 GSM8K eval returned only '!' for 68 of 1319
questions. Set it explicitly to 1, which the image honors.

SGLang 5994c559 对任何多卡 graph 捕获强制设置 NCCL_GRAPH_REGISTER=0;
在此设置下 DCP8 c24 GSM8K 评测中 1319 道题有 68 道只输出 '!'。
显式设置为 1,镜像会遵循该设置。

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@Klaud-Cold

Klaud-Cold commented Oct 24, 2026 •

Copy link
Copy Markdown
Collaborator Author

Repair 1/5 · Passed · Run 40127359781 / attempt 1 · 2026-10-24 21:40 UTC
lmsysorg/sglang:nightly-dev-20261024-5994c559@sha256:40da9cac78a558237426971c048ab0f4f1a4b13bfb1b6e7a48d4d980e4163c8e · a0287f0b3617 · AgentX · TP8/EP1 · Mean latency
Change: Measured commit 74754a8 sets NCCL_GRAPH_REGISTER=1 in the recipe env, which sglang parallel_hook._disable_nccl_graph_buffer_registration honors as an explicit override, restoring the old image's registration for graph-captured collectives; whether registration is the cause of the '!' outputs is unproven, and upstream sgl-project/sglang#45367 reports DCP a2a corruption only with registration on and the a2a backend.

Concurrency Output tok/s/GPU ↑ TTFT ms ↓ TPOT ms ↓
1 17.98 (+0.8%) 608.9 (-0.4%) 3.45 (-1.7%)
4 26.8 (+2.3%) 428.41 (+1.4%) 4.54 (+10.2%)
8 48.13 (-0.6%) 501.68 (-2%) 5.1 (-0.6%)
12 60.14 (+0%) 1,007.36 (+7.8%) 7.31 (+2.5%)
16 72.85 (+0.2%) 1,276.1 (-1.9%) 10.33 (-1%)
17 79.17 (+0.5%) 1,416.98 (-1.6%) 10.5 (-4%)
24 97.05 (-0.2%) 1,909.12 (+5%) 16.53 (+0.4%)

Note: c4, c8: 1 request error.
Note: c12: 4 request errors.
Note: c16: 3 request errors.
Note: c17, c24: 2 request errors.

Eval Score ↑ Samples
gsm8k/em_strict · c24 92.57% (N/A) 1,319

Note: All rows: Δ N/A: no matched eval baseline.

Next: Stopping as incompatible: with identical resolved server args and NCCL_GRAPH_REGISTER=1 applied, 5994c559 still returns only '!' for 56 of 1319 GSM8K c24 questions (68 in the initial attempt, a different batch-dependent set), versus none on the baseline image, so fixing it needs an SGLang engine change rather than a recipe flag.

中文

修复 1/5 · 已通过 · Run 40127359781 / attempt 1 · 2026-10-24 21:40 UTC
lmsysorg/sglang:nightly-dev-20261024-5994c559@sha256:40da9cac78a558237426971c048ab0f4f1a4b13bfb1b6e7a48d4d980e4163c8e · a0287f0b3617 · AgentX · TP8/EP1 · 平均延迟
**变更:**测量提交 74754a8 在配方 env 中设置 NCCL_GRAPH_REGISTER=1,sglang parallel_hook._disable_nccl_graph_buffer_registration 将其视为显式覆盖,从而恢复旧镜像对 graph 捕获集合通信的注册;注册是否为 '!' 输出的原因尚未证实,上游 sgl-project/sglang#45367 报告的 DCP a2a 损坏仅出现在开启注册且使用 a2a 后端时。实测数值及异常说明见上表。
**下一步:**以不兼容结束:在已解析 server args 相同且已应用 NCCL_GRAPH_REGISTER=1 的情况下,5994c559 在 GSM8K c24 的 1319 道题中仍有 56 道只输出 '!'(初次尝试为 68 道,且为与批次相关的不同题目),而基线镜像没有此类输出,因此修复需要更改 SGLang 引擎,而非配方参数。

@Klaud-Cold

Copy link
Copy Markdown
Collaborator Author

incompatible · Cleanup pending. Stop and confirm owned runs before closing.

中文

incompatible · 清理待完成。先停止并确认自有运行结束,再关闭 PR。

@Klaud-Cold Klaud-Cold closed this Oct 24, 2026
@Klaud-Cold
Klaud-Cold deleted the klaud/auto-fb56d2a358d2743c-0dd16aeb42d3dd5b branch October 24, 2026 21:41
@Klaud-Cold

Copy link
Copy Markdown
Collaborator Author

incompatible · Repairs: 1 · Runs: 40121743981, 40127359781
Reason: Stopping as incompatible: with identical resolved server args and NCCL_GRAPH_REGISTER=1 applied, 5994c559 still returns only '!' for 56 of 1319 GSM8K c24 questions (68 in the initial attempt, a different batch-dependent set), versus none on the baseline image, so fixing it needs an SGLang engine change rather than a recipe flag.
All owned runs ended. PR closed; branch deleted for retry.

中文

incompatible · 修复次数:1 · 运行:40121743981, 40127359781
**原因:**以不兼容结束:在已解析 server args 相同且已应用 NCCL_GRAPH_REGISTER=1 的情况下,5994c559 在 GSM8K c24 的 1319 道题中仍有 56 道只输出 '!'(初次尝试为 68 道,且为与批次相关的不同题目),而基线镜像没有此类输出,因此修复需要更改 SGLang 引擎,而非配方参数。
所有自有运行均已结束。PR 已关闭;分支已删除,可重新尝试。

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

Status: Done

Development

Successfully merging this pull request may close these issues.

1 participant