Skip to content

[Klaud Cold] Update DeepSeek-V4-Pro FP4 GB200 Dynamo-SGLang AgentX disagg image to v0.5.22-cu130 / [Klaud Cold] 将 DeepSeek-V4-Pro FP4 GB200 Dynamo-SGLang AgentX 分离式镜像更新至 v0.5.22-cu130 - #4329

Closed
Klaud-Cold wants to merge 3 commits into
mainfrom
klaud/auto-8a3c6cb2221f87df-3af5706b64e41f45
Closed

Klaud-Cold wants to merge 3 commits into
mainfrom
klaud/auto-8a3c6cb2221f87df-3af5706b64e41f45

Conversation

@Klaud-Cold

@Klaud-Cold Klaud-Cold commented Oct 19, 2026 •

Copy link
Copy Markdown
Collaborator

Goal: Update SGLang image from lmsysorg/sglang:nightly-dev-20260916-c9a8fba9 to lmsysorg/sglang:v0.5.22-cu130.
Baseline: 2026-10-12 · lmsysorg/sglang:nightly-dev-20260916-c9a8fba9
Mean latency · Sources: API 1, API 2, API 3

Point Total tok/s/GPU ↑ Output tok/s/GPU ↑ TTFT ms ↓ TPOT ms ↓
AgentX c64 PTP8x1/DTP16x1 944059 17,795.41 125.98 2,403.01 6.07
AgentX c128 PTP8x1/DTP16x1 880874 26,081.27 213.62 4,161.5 6.66
AgentX c256 PTP16x1/DTP32x1 97c52f 28,040.06 198.88 4,864.66 7.36
AgentX c768 PTP16x2/DTP32x1 c62df2 53,176.06 395.08 7,070.43 10.49
AgentX c1024 PTP16x2/DTP32x1 a4bf48 64,538.24 482.77 9,122.37 12.05
AgentX c1280 PTP16x2/DTP32x1 48862c 72,718 549.15 11,759.75 13.32
Eval Score ↑ Samples
gsm8k/em_strict · c64 97.57% 1,319
gsm8k/em_strict · c128 97.8% 1,319
gsm8k/em_strict · c256 97.57% 1,319
gsm8k/em_strict · c768 97.57% 1,319
gsm8k/em_strict · c1024 97.65% 1,319
gsm8k/em_strict · c1280 97.57% 1,319

Stop: Add the klaud-abort label at any time to stop this candidate now; Klaud cancels its runs, closes this PR and does not retry it. Remove the label to let the planner pick it up again.

中文

**目标:**将 SGLang 镜像从 lmsysorg/sglang:nightly-dev-20260916-c9a8fba9 更新至 lmsysorg/sglang:v0.5.22-cu130。
**基线:**2026-10-12 · lmsysorg/sglang:nightly-dev-20260916-c9a8fba9
平均延迟 · 来源: API 1, API 2, API 3;数值及异常说明见上表。
**停止:**随时添加 klaud-abort 标签即可立即停止该候选:Klaud 会取消其运行、关闭此 PR 且不再重试。移除该标签后,规划器会在下次扫描时重新考虑它。

github-actions Bot and others added 2 commits October 19, 2026 18:46
…2-cu130

Pin lmsysorg/sglang:v0.5.22-cu130 by digest for
dsv4-fp4-gb200-dynamo-sglang-agentic-disagg and drop
SGLANG_OPT_SWA_SPLIT_LEAF_ON_INSERT, which upstream removed with the
legacy SWA radix cache (sgl-project/sglang#40313).

将 dsv4-fp4-gb200-dynamo-sglang-agentic-disagg 的镜像更新为按摘要固定的
lmsysorg/sglang:v0.5.22-cu130,并删除上游随旧版 SWA radix cache 一并移除的
SGLANG_OPT_SWA_SPLIT_LEAF_ON_INSERT 环境变量(sgl-project/sglang#40313)。

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
SGLang v0.5.22 resolves DP attention into attn_dp_size and leaves dp_size=1
(sgl-project/sglang#41818). Dynamo 1.5.0.dev20260910 derives local DP ranks
from dp_size only, so the KV router sent every prefill to DP rank 0.
Use Dynamo 1.6.0.dev20261014, which counts ranks from attn_dp_size
(ai-dynamo/dynamo#16005). Raise c128 prefill mem-fraction-static to 0.815
because the v0.5.22 weight footprint left only 334k prefill KV tokens.

SGLang v0.5.22 将 DP attention 解析为 attn_dp_size,并保持 dp_size=1
(sgl-project/sglang#41818)。Dynamo 1.5.0.dev20260910 只根据 dp_size
推导本地 DP rank,导致 KV router 把所有 prefill 请求都发往 DP rank 0。
改用按 attn_dp_size 计数的 Dynamo 1.6.0.dev20261014(ai-dynamo/dynamo#16005)。
由于 v0.5.22 权重占用增加后 prefill KV 仅剩 334k token,将 c128 的 prefill
mem-fraction-static 提高到 0.815。

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@github-actions

Copy link
Copy Markdown
Contributor

Thanks for the contribution!

  • Review: If this PR changes files owned by someone other than a repository admin or @SemiAnalysisAI/core, ask one eligible CODEOWNER to complete the latest PR_REVIEW_CHECKLIST.md before contacting a core maintainer on Slack. Follow the template exactly, including As a PR reviewer and CODEOWNER, I have reviewed this and have, so sign-off verification triggers.
  • PR verification: Sweeps run only when the PR appends an inferencex-e2e/perf-changelog.yaml entry and carries exactly one primary label: full-sweep-fail-fast (strongly recommended; canary plus per-matrix fail-fast), full-sweep-enabled (canary; matrix jobs continue after a failure), or non-canary-full-sweep-enabled (no canary or fail-fast). The modifiers all-evals, evals-only, and agentx-fast require a primary label. On fork PRs, a maintainer applies the label. See sweep labels and reuse.
  • After merging: PR authors must ensure all GitHub Actions jobs pass. Transient failures often pass on rerun; see how to rerun failed jobs.
中文

感谢你的贡献!

  • **审阅:**如果 PR 修改的文件归属于仓库管理员及 @SemiAnalysisAI/core 之外的 CODEOWNER,请先联系一位有资格的 CODEOWNER 填写最新的 PR_REVIEW_CHECKLIST.md,再通过 Slack 联系核心维护者。必须严格遵循模板,并保留 As a PR reviewer and CODEOWNER, I have reviewed this and have,才能触发签核验证。
  • **PR 验证:**只有当 PR 在 inferencex-e2e/perf-changelog.yaml 末尾追加了条目,并且恰好带有一个主标签时,才会运行扫描:full-sweep-fail-fast(强烈推荐;canary 加逐矩阵 fail-fast)、full-sweep-enabled(有 canary;矩阵任务在失败后继续运行)或 non-canary-full-sweep-enabled(无 canary,也无 fail-fast)。修饰标签 all-evals、evals-only 和 agentx-fast 必须与主标签一起使用。fork PR 的标签由维护者添加。参见扫描标签与复用。
  • **合并后:**PR 作者必须确保所有 GitHub Actions 任务通过。临时性失败通常可以通过重新运行恢复;参见重新运行失败任务的说明。

@Klaud-Cold

Klaud-Cold commented Oct 19, 2026 •

Copy link
Copy Markdown
Collaborator Author

Initial attempt · Running · Run 39308319786 / attempt 1 · 2026-10-19 19:42 UTC
lmsysorg/sglang:v0.5.22-cu130@sha256:e3a4f06001a29542fc4a8638536d9ffc5eb9c9942b08603c09b19a56eae4a1ae · fa4b13f10313
Change: Measure 2b925b7: pin v0.5.22-cu130 (baa9befa, descendant of c9a8fba9: sgl-project/sglang@c9a8fba...v0.5.22); drop SGLANG_OPT_SWA_SPLIT_LEAF_ON_INSERT (removed, sgl-project/sglang#40313); use Dynamo 1.6.0.dev20261014 because sgl-project/sglang#41818 sets dp_size=1 with attn_dp_size=8 and 1.5.0.dev20260910 then sent every prefill to DP rank 0 (fixed in ai-dynamo/dynamo#16005, checked in both wheels); raise c128 prefill mem-fraction-static 0.80 to 0.815 since v0.5.22 weights left 334k KV tokens and 19 longer warmup prompts never returned in closed run 39034150815.

Next: The c128 eval passed and its prefill logs show batches spread across DP ranks 0-7 with Dynamo 1.6.0.dev20261014, but prefill KV holds only 540k tokens at 0.815, below the 670k-token warmup prompts, so repair 1 raises c128 to 0.83; the earlier c128 benchmark failure was a pyxis nginx import error on watchtower-navy-cn05, and the remaining evals here stay queued.

中文

初次尝试 · 运行中 · Run 39308319786 / attempt 1 · 2026-10-19 19:42 UTC
lmsysorg/sglang:v0.5.22-cu130@sha256:e3a4f06001a29542fc4a8638536d9ffc5eb9c9942b08603c09b19a56eae4a1ae · fa4b13f10313
**变更:**测量 2b925b7:固定 v0.5.22-cu130(baa9befa,为 c9a8fba9 的后续提交:sgl-project/sglang@c9a8fba...v0.5.22);移除已删除的 SGLANG_OPT_SWA_SPLIT_LEAF_ON_INSERT(sgl-project/sglang#40313);因 sgl-project/sglang#41818 设定 dp_size=1、attn_dp_size=8,1.5.0.dev20260910 会把所有 prefill 发往 DP rank 0,故改用 Dynamo 1.6.0.dev20261014(ai-dynamo/dynamo#16005 已修复,两个 wheel 均已核实);因 v0.5.22 权重使 KV 仅剩 334k token,已关闭的运行 39034150815 中 19 个更长的 warmup 请求未返回,将 c128 prefill mem-fraction-static 从 0.80 提高到 0.815。实测数值及异常说明见上表。
**下一步:**c128 评测已通过,其 prefill 日志显示在 Dynamo 1.6.0.dev20261014 下批次分布于 DP rank 0-7;但在 0.815 时 prefill KV 仅 540k token,低于 670k token 的 warmup 请求,因此修复 1 将 c128 提高到 0.83;此前 c128 基准测试失败源于 watchtower-navy-cn05 上 pyxis 导入 nginx 出错,本运行中其余评测继续排队。

At mem-fraction-static 0.815 the c128 prefill pool holds 540k tokens,
below the 670k-token AgentX warmup prompts; 0.83 restores about the
baseline's 735k tokens and stays below the c64 point's 0.85.

在 mem-fraction-static 0.815 时,c128 prefill KV 池仅 540k token,低于
670k token 的 AgentX warmup 请求;0.83 可恢复到约基线的 735k token,且
仍低于 c64 点使用的 0.85。

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@Klaud-Cold

Klaud-Cold commented Oct 19, 2026 •

Copy link
Copy Markdown
Collaborator Author

Repair 1/5 · Running · Run 39313265645 / attempt 1 · 2026-10-20 03:36 UTC
lmsysorg/sglang:v0.5.22-cu130@sha256:e3a4f06001a29542fc4a8638536d9ffc5eb9c9942b08603c09b19a56eae4a1ae · fa4b13f10313
Change: Measure 01d1d14: raise c128 prefill mem-fraction-static from 0.815 to 0.83, because the c128 eval in run 39308319786 showed 540k prefill KV tokens at 0.815 (about 137k tokens per 0.01) and 0.83 restores roughly the baseline's 735k while staying below c64's 0.85 on the same prefill shape; Dynamo 1.6.0.dev20261014 and the other settings are unchanged.

Next: The repair c128 smoke benchmark started on gb200-nv at about 03:40 UTC after eight hours queued, and the initial run c64 eval is also running; wait for both.

中文

修复 1/5 · 运行中 · Run 39313265645 / attempt 1 · 2026-10-20 03:36 UTC
lmsysorg/sglang:v0.5.22-cu130@sha256:e3a4f06001a29542fc4a8638536d9ffc5eb9c9942b08603c09b19a56eae4a1ae · fa4b13f10313
**变更:**测量 01d1d14:将 c128 prefill mem-fraction-static 从 0.815 提高到 0.83;运行 39308319786 中的 c128 评测显示 0.815 时 prefill KV 为 540k token(约每 0.01 对应 137k token),0.83 可恢复到约基线的 735k,且仍低于相同 prefill 形态下 c64 使用的 0.85;Dynamo 1.6.0.dev20261014 及其他设置不变。实测数值及异常说明见上表。
**下一步:**修复的 c128 冒烟基准测试在排队约八小时后于 03:40 UTC 左右在 gb200-nv 上开始运行,初次运行的 c64 评测也在运行中;等待两者完成。

@Klaud-Cold

Copy link
Copy Markdown
Collaborator Author

resumed · The session ended before finishing; session 2 continues it in a new run of its planner.

中文

resumed · 会话未完成即已结束;第 2 个会话将在其规划运行的新运行中继续。

@Klaud-Cold

Copy link
Copy Markdown
Collaborator Author

Repair 1/5 · Queued · Run 39313265645 / attempt 2 · 2026-10-20 03:51 UTC
lmsysorg/sglang:v0.5.22-cu130@sha256:e3a4f06001a29542fc4a8638536d9ffc5eb9c9942b08603c09b19a56eae4a1ae · fa4b13f10313
Change: Measure 01d1d14: raise c128 prefill mem-fraction-static from 0.815 to 0.83, because the c128 eval in run 39308319786 showed 540k prefill KV tokens at 0.815 (about 137k tokens per 0.01) and 0.83 restores roughly the baseline's 735k while staying below c64's 0.85 on the same prefill shape; Dynamo 1.6.0.dev20261014 and the other settings are unchanged.

Next: The dashboard bot cancelled all of run 39313265645 attempt 1 at 03:46 UTC, including the c128 benchmark 14 minutes after it started and before any result, so the preempted jobs were rerun on the same head as attempt 2 and are queued again.

中文

修复 1/5 · 排队中 · Run 39313265645 / attempt 2 · 2026-10-20 03:51 UTC
lmsysorg/sglang:v0.5.22-cu130@sha256:e3a4f06001a29542fc4a8638536d9ffc5eb9c9942b08603c09b19a56eae4a1ae · fa4b13f10313
**变更:**测量 01d1d14:将 c128 prefill mem-fraction-static 从 0.815 提高到 0.83;运行 39308319786 中的 c128 评测显示 0.815 时 prefill KV 为 540k token(约每 0.01 对应 137k token),0.83 可恢复到约基线的 735k,且仍低于相同 prefill 形态下 c64 使用的 0.85;Dynamo 1.6.0.dev20261014 及其他设置不变。实测数值及异常说明见上表。
**下一步:**仪表盘机器人于 03:46 UTC 取消了运行 39313265645 的第 1 次尝试的全部任务,其中 c128 基准测试在启动 14 分钟后、尚未产出结果时被取消;被抢占的任务已在同一 head 上作为第 2 次尝试重新运行,并再次排队。

@Klaud-Cold

Klaud-Cold commented Oct 20, 2026 •

Copy link
Copy Markdown
Collaborator Author

Repair 1/5 · Running · Run 39313265645 / attempt 3 · 2026-10-20 14:32 UTC
lmsysorg/sglang:v0.5.22-cu130@sha256:e3a4f06001a29542fc4a8638536d9ffc5eb9c9942b08603c09b19a56eae4a1ae · fa4b13f10313 · AgentX · PTP8x1/DTP16x1 · Mean latency
Change: Measure 01d1d14: raise c128 prefill mem-fraction-static from 0.815 to 0.83, because the c128 eval in run 39308319786 showed 540k prefill KV tokens at 0.815 (about 137k tokens per 0.01) and 0.83 restores roughly the baseline's 735k while staying below c64's 0.85 on the same prefill shape; Dynamo 1.6.0.dev20261014 and the other settings are unchanged.

Concurrency Output tok/s/GPU ↑ TTFT ms ↓ TPOT ms ↓
64 128.46 (N/A) 2,103.32 (N/A) 6.7 (N/A)
128 220.65 (N/A) 3,880.07 (N/A) 8.04 (N/A)

Note: All rows: Δ N/A: dataset/scenario/statistic mismatch.

Eval Score ↑ Samples
gsm8k/em_strict · c64 97.35% (-0.23 pp) 1,319 each
gsm8k/em_strict · c128 97.35% (-0.45 pp) 1,319 each

Next: The c64 and c128 smoke benchmarks and evals passed on run 39313265645 attempt 3 with zero request errors; the c256, c768, c1024 and c1280 benchmarks and evals remain queued on gb200-nv, so keep waiting.

中文

修复 1/5 · 运行中 · Run 39313265645 / attempt 3 · 2026-10-20 14:32 UTC
lmsysorg/sglang:v0.5.22-cu130@sha256:e3a4f06001a29542fc4a8638536d9ffc5eb9c9942b08603c09b19a56eae4a1ae · fa4b13f10313 · AgentX · PTP8x1/DTP16x1 · 平均延迟
**变更:**测量 01d1d14:将 c128 prefill mem-fraction-static 从 0.815 提高到 0.83;运行 39308319786 中的 c128 评测显示 0.815 时 prefill KV 为 540k token(约每 0.01 对应 137k token),0.83 可恢复到约基线的 735k,且仍低于相同 prefill 形态下 c64 使用的 0.85;Dynamo 1.6.0.dev20261014 及其他设置不变。实测数值及异常说明见上表。
**下一步:**运行 39313265645 第 3 次尝试中 c64 与 c128 冒烟基准测试和评测均已通过,请求错误为零;c256、c768、c1024 和 c1280 的基准测试与评测仍在 gb200-nv 上排队,继续等待。

@Klaud-Cold

Copy link
Copy Markdown
Collaborator Author

resumed · The session ended before finishing; session 3 continues it in a new run of its planner.

中文

resumed · 会话未完成即已结束;第 3 个会话将在其规划运行的新运行中继续。

@Klaud-Cold

Copy link
Copy Markdown
Collaborator Author

resumed · The session ended before finishing; session 4 continues it in a new run of its planner.

中文

resumed · 会话未完成即已结束;第 4 个会话将在其规划运行的新运行中继续。

@Klaud-Cold

Copy link
Copy Markdown
Collaborator Author

Repair 1/5 · Queued · Run 39313265645 / attempt 4 · 2026-10-20 15:40 UTC
lmsysorg/sglang:v0.5.22-cu130@sha256:e3a4f06001a29542fc4a8638536d9ffc5eb9c9942b08603c09b19a56eae4a1ae · fa4b13f10313 · AgentX · PTP8x1/DTP16x1 · Mean latency
Change: Measure 01d1d14: raise c128 prefill mem-fraction-static from 0.815 to 0.83, because the c128 eval in run 39308319786 showed 540k prefill KV tokens at 0.815 (about 137k tokens per 0.01) and 0.83 restores roughly the baseline's 735k while staying below c64's 0.85 on the same prefill shape; Dynamo 1.6.0.dev20261014 and the other settings are unchanged.

Concurrency Output tok/s/GPU ↑ TTFT ms ↓ TPOT ms ↓
64 128.46 (N/A) 2,103.32 (N/A) 6.7 (N/A)
128 220.65 (N/A) 3,880.07 (N/A) 8.04 (N/A)

Note: All rows: Δ N/A: dataset/scenario/statistic mismatch.

Eval Score ↑ Samples
gsm8k/em_strict · c64 97.35% (-0.23 pp) 1,319 each
gsm8k/em_strict · c128 97.35% (-0.45 pp) 1,319 each

Next: The dashboard bot cancelled the queued c256-c1280 jobs (and the c768 benchmark 16 minutes after it started) at 15:35 UTC and reran them as attempt 4; c64 and c128 results from attempt 3 are kept, and the four remaining benchmarks and evals are queued again.

中文

修复 1/5 · 排队中 · Run 39313265645 / attempt 4 · 2026-10-20 15:40 UTC
lmsysorg/sglang:v0.5.22-cu130@sha256:e3a4f06001a29542fc4a8638536d9ffc5eb9c9942b08603c09b19a56eae4a1ae · fa4b13f10313 · AgentX · PTP8x1/DTP16x1 · 平均延迟
**变更:**测量 01d1d14:将 c128 prefill mem-fraction-static 从 0.815 提高到 0.83;运行 39308319786 中的 c128 评测显示 0.815 时 prefill KV 为 540k token(约每 0.01 对应 137k token),0.83 可恢复到约基线的 735k,且仍低于相同 prefill 形态下 c64 使用的 0.85;Dynamo 1.6.0.dev20261014 及其他设置不变。实测数值及异常说明见上表。
**下一步:**仪表盘机器人于 15:35 UTC 取消了排队中的 c256-c1280 任务(以及启动 16 分钟的 c768 基准测试),并作为第 4 次尝试重新运行;保留第 3 次尝试中 c64 与 c128 的结果,其余四个基准测试与评测再次排队。

@Klaud-Cold

Copy link
Copy Markdown
Collaborator Author

unexpected-error · Cleanup pending. Stop and confirm owned runs before closing.

中文

unexpected-error · 清理待完成。先停止并确认自有运行结束,再关闭 PR。

@Klaud-Cold Klaud-Cold closed this Oct 21, 2026
@Klaud-Cold
Klaud-Cold deleted the klaud/auto-8a3c6cb2221f87df-3af5706b64e41f45 branch October 21, 2026 18:18
@Klaud-Cold

Copy link
Copy Markdown
Collaborator Author

unexpected-error · Repairs: unknown · Runs: 39308319786, 39313265645
All owned runs ended. PR closed; branch deleted for retry.

中文

unexpected-error · 修复次数:unknown · 运行:39308319786, 39313265645
所有自有运行均已结束。PR 已关闭;分支已删除,可重新尝试。

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

Status: Done

Development

Successfully merging this pull request may close these issues.

1 participant