Skip to content

[Klaud Cold] Update DeepSeek-V4-Pro FP4 GB200 Dynamo-SGLang AgentX disagg image to v0.5.22-cu130 / [Klaud Cold] 将 DeepSeek-V4-Pro FP4 GB200 Dynamo-SGLang AgentX 分离式镜像更新至 v0.5.22-cu130 - #4254

Closed
Klaud-Cold wants to merge 1 commit into
mainfrom
klaud/auto-8a3c6cb2221f87df-3af5706b64e41f45
Closed

Klaud-Cold wants to merge 1 commit into
mainfrom
klaud/auto-8a3c6cb2221f87df-3af5706b64e41f45

Conversation

@Klaud-Cold

@Klaud-Cold Klaud-Cold commented Oct 17, 2026 •

Copy link
Copy Markdown
Collaborator

Goal: Update SGLang image from lmsysorg/sglang:nightly-dev-20260916-c9a8fba9 to lmsysorg/sglang:v0.5.22-cu130@sha256:e3a4f06001a29542fc4a8638536d9ffc5eb9c9942b08603c09b19a56eae4a1ae.
Baseline: 2026-10-12 · lmsysorg/sglang:nightly-dev-20260916-c9a8fba9
Mean latency · Sources: API 1, API 2, API 3

Point Total tok/s/GPU ↑ Output tok/s/GPU ↑ TTFT ms ↓ TPOT ms ↓
AgentX c64 PTP8x1/DTP16x1 944059 17,795.41 125.98 2,403.01 6.07
AgentX c128 PTP8x1/DTP16x1 880874 26,081.27 213.62 4,161.5 6.66
AgentX c256 PTP16x1/DTP32x1 97c52f 28,040.06 198.88 4,864.66 7.36
AgentX c768 PTP16x2/DTP32x1 c62df2 53,176.06 395.08 7,070.43 10.49
AgentX c1024 PTP16x2/DTP32x1 a4bf48 64,538.24 482.77 9,122.37 12.05
AgentX c1280 PTP16x2/DTP32x1 48862c 72,718 549.15 11,759.75 13.32
Eval Score ↑ Samples
gsm8k/em_strict · c64 97.57% 1,319
gsm8k/em_strict · c128 97.8% 1,319
gsm8k/em_strict · c256 97.57% 1,319
gsm8k/em_strict · c768 97.57% 1,319
gsm8k/em_strict · c1024 97.65% 1,319
gsm8k/em_strict · c1280 97.57% 1,319
中文

**目标:**将 SGLang 镜像从 lmsysorg/sglang:nightly-dev-20260916-c9a8fba9 更新为 lmsysorg/sglang:v0.5.22-cu130@sha256:e3a4f06001a29542fc4a8638536d9ffc5eb9c9942b08603c09b19a56eae4a1ae。
**基线:**2026-10-12 · lmsysorg/sglang:nightly-dev-20260916-c9a8fba9
平均延迟 · 来源: API 1, API 2, API 3;数值及异常说明见上表。

…2-cu130

Pin lmsysorg/sglang:v0.5.22-cu130 by digest and drop
SGLANG_OPT_SWA_SPLIT_LEAF_ON_INSERT, whose SWA radix cache was removed
upstream.

将 GB200 Dynamo-SGLang AgentX 分离式配置的镜像更新至 v0.5.22-cu130,
按摘要固定,并移除上游已删除的 SWA radix cache 所用的
SGLANG_OPT_SWA_SPLIT_LEAF_ON_INSERT。

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@github-actions

Copy link
Copy Markdown
Contributor

Thanks for the contribution!

  • Review: If this PR changes files owned by someone other than a repository admin or @SemiAnalysisAI/core, ask one eligible CODEOWNER to complete the latest PR_REVIEW_CHECKLIST.md before contacting a core maintainer on Slack. Follow the template exactly, including As a PR reviewer and CODEOWNER, I have reviewed this and have, so sign-off verification triggers.
  • PR verification: Sweeps run only when the PR appends an inferencex-e2e/perf-changelog.yaml entry and carries exactly one primary label: full-sweep-fail-fast (strongly recommended; canary plus per-matrix fail-fast), full-sweep-enabled (canary; matrix jobs continue after a failure), or non-canary-full-sweep-enabled (no canary or fail-fast). The modifiers all-evals, evals-only, and agentx-fast require a primary label. On fork PRs, a maintainer applies the label. See sweep labels and reuse.
  • After merging: PR authors must ensure all GitHub Actions jobs pass. Transient failures often pass on rerun; see how to rerun failed jobs.
中文

感谢你的贡献!

  • **审阅:**如果 PR 修改的文件归属于仓库管理员及 @SemiAnalysisAI/core 之外的 CODEOWNER,请先联系一位有资格的 CODEOWNER 填写最新的 PR_REVIEW_CHECKLIST.md,再通过 Slack 联系核心维护者。必须严格遵循模板,并保留 As a PR reviewer and CODEOWNER, I have reviewed this and have,才能触发签核验证。
  • **PR 验证:**只有当 PR 在 inferencex-e2e/perf-changelog.yaml 末尾追加了条目,并且恰好带有一个主标签时,才会运行扫描:full-sweep-fail-fast(强烈推荐;canary 加逐矩阵 fail-fast)、full-sweep-enabled(有 canary;矩阵任务在失败后继续运行)或 non-canary-full-sweep-enabled(无 canary,也无 fail-fast)。修饰标签 all-evals、evals-only 和 agentx-fast 必须与主标签一起使用。fork PR 的标签由维护者添加。参见扫描标签与复用。
  • **合并后:**PR 作者必须确保所有 GitHub Actions 任务通过。临时性失败通常可以通过重新运行恢复;参见重新运行失败任务的说明。

@Klaud-Cold

Klaud-Cold commented Oct 17, 2026 •

Copy link
Copy Markdown
Collaborator Author

Initial attempt · Failed · Run 39034150815 / attempt 1 · 2026-10-17 15:44 UTC
lmsysorg/sglang:v0.5.22-cu130@sha256:e3a4f06001a29542fc4a8638536d9ffc5eb9c9942b08603c09b19a56eae4a1ae · 144e40f60c73
Change: Measure b1029f3: pin v0.5.22-cu130 (tag commit baa9befa, descendant of c9a8fba9) and drop SGLANG_OPT_SWA_SPLIT_LEAF_ON_INSERT, removed with the SWA radix cache in sgl-project/sglang#40313; recipe flags, FP8 KV default, DSpark and MegaMoE budget checks were audited in sgl-project/sglang@c9a8fba...v0.5.22, while torch 2.14.1, flashinfer 0.7.1, DeepGEMM 0.2.2, Mooncake 0.3.13.post1 and deprecated DP-attention spelling under Dynamo 1.5.0.dev20260910 await smoke.

Eval Score ↑ Samples
gsm8k/em_strict · c64 97.73% (+0.15 pp) 1,319 each
gsm8k/em_strict · c128 97.65% (-0.15 pp) 1,319 each
gsm8k/em_strict · c256 97.65% (+0.08 pp) 1,319 each
gsm8k/em_strict · c768 97.27% (-0.30 pp) 1,319 each
gsm8k/em_strict · c1024 97.57% (-0.08 pp) 1,319 each
gsm8k/em_strict · c1280 97.5% (-0.08 pp) 1,319 each

Next: All six evals passed, but the c64 benchmark failed on a transient pip IncompleteRead while installing Dynamo wheel dependencies in frontend_3 and fail-fast cancelled the other benchmarks, so rerun the failed and cancelled jobs on the same head as an infrastructure retry.

中文

初次尝试 · 失败 · Run 39034150815 / attempt 1 · 2026-10-17 15:44 UTC
lmsysorg/sglang:v0.5.22-cu130@sha256:e3a4f06001a29542fc4a8638536d9ffc5eb9c9942b08603c09b19a56eae4a1ae · 144e40f60c73
**变更:**测量 b1029f3:固定 v0.5.22-cu130(标签提交 baa9befa,为 c9a8fba9 的后续提交),并移除随 SWA radix cache 在 sgl-project/sglang#40313 中删除的 SGLANG_OPT_SWA_SPLIT_LEAF_ON_INSERT;已在 sgl-project/sglang@c9a8fba...v0.5.22 中核对配方参数、FP8 KV 默认值、DSpark 与 MegaMoE 预算校验,torch 2.14.1、flashinfer 0.7.1、DeepGEMM 0.2.2、Mooncake 0.3.13.post1 以及 Dynamo 1.5.0.dev20260910 下已弃用的 DP-attention 写法待冒烟验证。实测数值及异常说明见上表。
**下一步:**六个评测全部通过,但 c64 基准测试因 frontend_3 安装 Dynamo wheel 依赖时 pip 出现瞬时 IncompleteRead 而失败,fail-fast 取消了其余基准测试,因此在同一提交上将失败和被取消的任务作为基础设施重试重新运行。

@Klaud-Cold

Klaud-Cold commented Oct 17, 2026 •

Copy link
Copy Markdown
Collaborator Author

Infrastructure retry 1 · Running · Run 39034150815 / attempt 2 · 2026-10-17 16:42 UTC
lmsysorg/sglang:v0.5.22-cu130@sha256:e3a4f06001a29542fc4a8638536d9ffc5eb9c9942b08603c09b19a56eae4a1ae · 144e40f60c73
Change: Rerun the failed c64 benchmark and the five fail-fast-cancelled benchmarks of run 39034150815 on the same head b1029f3 after a transient pip IncompleteRead during the Dynamo wheel install; no recipe change.

Eval Score ↑ Samples
gsm8k/em_strict · c64 97.73% (+0.15 pp) 1,319 each
gsm8k/em_strict · c128 97.65% (-0.15 pp) 1,319 each
gsm8k/em_strict · c256 97.65% (+0.08 pp) 1,319 each
gsm8k/em_strict · c768 97.27% (-0.30 pp) 1,319 each
gsm8k/em_strict · c1024 97.57% (-0.08 pp) 1,319 each
gsm8k/em_strict · c1280 97.5% (-0.08 pp) 1,319 each

Next: The rerun c64 and c128 benchmarks have been serving for about 45 minutes without errors and c256 through c1280 are queued, so keep waiting.

中文

基础设施重试 1 · 运行中 · Run 39034150815 / attempt 2 · 2026-10-17 16:42 UTC
lmsysorg/sglang:v0.5.22-cu130@sha256:e3a4f06001a29542fc4a8638536d9ffc5eb9c9942b08603c09b19a56eae4a1ae · 144e40f60c73
**变更:**由于 Dynamo wheel 安装期间出现瞬时 pip IncompleteRead,在同一提交 b1029f3 上重新运行 39034150815 中失败的 c64 基准测试及被 fail-fast 取消的五个基准测试;配方无改动。实测数值及异常说明见上表。
**下一步:**重新运行的 c64 与 c128 基准测试已无错误运行约 45 分钟,c256 至 c1280 仍在排队,继续等待。

@Klaud-Cold

Copy link
Copy Markdown
Collaborator Author

unexpected-error · Cleanup pending. Stop and confirm owned runs before closing.

中文

unexpected-error · 清理待完成。先停止并确认自有运行结束,再关闭 PR。

@Klaud-Cold Klaud-Cold closed this Oct 18, 2026
@Klaud-Cold
Klaud-Cold deleted the klaud/auto-8a3c6cb2221f87df-3af5706b64e41f45 branch October 18, 2026 00:35
@Klaud-Cold

Copy link
Copy Markdown
Collaborator Author

unexpected-error · Repairs: unknown · Runs: 39034150815
All owned runs ended. PR closed; branch deleted for retry.

中文

unexpected-error · 修复次数:unknown · 运行:39034150815
所有自有运行均已结束。PR 已关闭;分支已删除,可重新尝试。

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

Status: Done

Development

Successfully merging this pull request may close these issues.

1 participant