Skip to content

[Klaud Cold] Update DSR1 FP8 MI300X SGLang image to v0.5.21-rocm720-mi30x / [Klaud Cold] 将 DSR1 FP8 MI300X SGLang 镜像更新至 v0.5.21-rocm720-mi30x - #4212

Closed
Klaud-Cold wants to merge 2 commits into
mainfrom
klaud/auto-4bfcce0abcfcb357-58417c62beb2fe3c
Closed

Klaud-Cold wants to merge 2 commits into
mainfrom
klaud/auto-4bfcce0abcfcb357-58417c62beb2fe3c

Conversation

@Klaud-Cold

@Klaud-Cold Klaud-Cold commented Oct 16, 2026 •

Copy link
Copy Markdown
Collaborator

Goal: Update SGLang image from lmsysorg/sglang:v0.5.12-rocm700-mi30x to lmsysorg/sglang:v0.5.21-rocm720-mi30x.
Baseline: 2026-05-18 · lmsysorg/sglang:v0.5.12-rocm700-mi30x
Mean latency · Sources: API 1, API 2

Point Total tok/s/GPU ↑ Output tok/s/GPU ↑ TTFT ms ↓ TPOT ms ↓
1k/1k c4 TP8 EP1 14b375 63.6 31.64 449.55 14.86
1k/1k c8 TP8 EP1 c5e335 109.26 54.83 428.19 17.42
1k/1k c16 TP8 EP1 a288ec 181.91 90.48 617.5 21.01
1k/1k c32 TP8 EP1 fb7ddf 270.16 135.3 545.68 28.37
1k/1k c64 TP8 EP1 e94574 376.97 188.44 655.21 40.85
8k/1k c4 TP8 EP1 3e8523 210.09 23.37 1,441.16 19.41
8k/1k c8 TP8 EP1 66bec9 354.57 39.86 1,411.27 23.15
8k/1k c16 TP8 EP1 21e483 558.37 61.87 1,499.09 29.85
8k/1k c32 TP8 EP1 3639eb 736.45 82.36 1,705.3 45.74
8k/1k c64 TP8 EP1 4231ac 891.71 98.93 2,481.95 77.05

Note: 1k/1k c4 TP8 EP1 14b375, 1k/1k c8 TP8 EP1 c5e335, 1k/1k c16 TP8 EP1 a288ec, 1k/1k c32 TP8 EP1 fb7ddf, 1k/1k c64 TP8 EP1 e94574: Single-turn 1k1k retired 2026-07-17 in #2263 after this baseline; excluded from coverage.

Eval: N/A

中文

**目标:**将 SGLang 镜像从 lmsysorg/sglang:v0.5.12-rocm700-mi30x 更新到 lmsysorg/sglang:v0.5.21-rocm720-mi30x。
**基线:**2026-05-18 · lmsysorg/sglang:v0.5.12-rocm700-mi30x
平均延迟 · 来源: API 1, API 2;数值及异常说明见上表。

Pin lmsysorg/sglang:v0.5.21-rocm720-mi30x by digest and rename the removed
--cuda-graph-max-bs alias to --cuda-graph-max-bs-decode (sglang#38375).

将 MI300X FP8 SGLang 镜像更新至 v0.5.21-rocm720-mi30x(按 digest 固定),
并将已移除的 --cuda-graph-max-bs 别名改为 --cuda-graph-max-bs-decode(sglang#38375)。

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@github-actions

Copy link
Copy Markdown
Contributor

Thanks for the contribution!

  • Review: If this PR changes files owned by someone other than a repository admin or @SemiAnalysisAI/core, ask one eligible CODEOWNER to complete the latest PR_REVIEW_CHECKLIST.md before contacting a core maintainer on Slack. Follow the template exactly, including As a PR reviewer and CODEOWNER, I have reviewed this and have, so sign-off verification triggers.
  • PR verification: Sweeps run only when the PR appends an inferencex-e2e/perf-changelog.yaml entry and carries exactly one primary label: full-sweep-fail-fast (strongly recommended; canary plus per-matrix fail-fast), full-sweep-enabled (canary; matrix jobs continue after a failure), or non-canary-full-sweep-enabled (no canary or fail-fast). The modifiers all-evals, evals-only, and agentx-fast require a primary label. On fork PRs, a maintainer applies the label. See sweep labels and reuse.
  • After merging: PR authors must ensure all GitHub Actions jobs pass. Transient failures often pass on rerun; see how to rerun failed jobs.
中文

感谢你的贡献!

  • **审阅:**如果 PR 修改的文件归属于仓库管理员及 @SemiAnalysisAI/core 之外的 CODEOWNER,请先联系一位有资格的 CODEOWNER 填写最新的 PR_REVIEW_CHECKLIST.md,再通过 Slack 联系核心维护者。必须严格遵循模板,并保留 As a PR reviewer and CODEOWNER, I have reviewed this and have,才能触发签核验证。
  • **PR 验证:**只有当 PR 在 inferencex-e2e/perf-changelog.yaml 末尾追加了条目,并且恰好带有一个主标签时,才会运行扫描:full-sweep-fail-fast(强烈推荐;canary 加逐矩阵 fail-fast)、full-sweep-enabled(有 canary;矩阵任务在失败后继续运行)或 non-canary-full-sweep-enabled(无 canary,也无 fail-fast)。修饰标签 all-evals、evals-only 和 agentx-fast 必须与主标签一起使用。fork PR 的标签由维护者添加。参见扫描标签与复用。
  • **合并后:**PR 作者必须确保所有 GitHub Actions 任务通过。临时性失败通常可以通过重新运行恢复;参见重新运行失败任务的说明。

@Klaud-Cold

Klaud-Cold commented Oct 16, 2026 •

Copy link
Copy Markdown
Collaborator Author

Initial attempt · Passed · Run 38964094385 / attempt 1 · 2026-10-16 21:23 UTC
lmsysorg/sglang:v0.5.21-rocm720-mi30x@sha256:0534eb382626ef55ace8be24f7d81cc6eb7451acac8a5727848397a5e9309d91 · f8ba650b6fb1 · 8k/1k · TP8/EP1 · Mean latency
Change: Measured ref 49d9e49: Pin lmsysorg/sglang:v0.5.21-rocm720-mi30x by digest (no rocm700 build after v0.5.19) and rename cuda-graph-max-bs to cuda-graph-max-bs-decode, since v0.5.21 removed the deprecated alias (sgl-project/sglang#38375, commit db272201a2; v0.5.12 server_args.py treated it as the decode graph cap); other recipe flags and SGLANG_AITER_MLA_PERSIST/SGLANG_USE_AITER are still parsed at v0.5.21 e00930c5.

Concurrency Output tok/s/GPU ↑ TTFT ms ↓ TPOT ms ↓
4 21.59 (-7.6%) 1,599.85 (+11%) 20.97 (+8%)

Note: All rows: request errors unavailable.

Eval Score ↑ Samples
gsm8k/em_strict · c32 95.75% (N/A) N/A/1,319 (old/new)

Note: All rows: Δ N/A: no matched eval baseline.

Next: Append the exact-family perf-changelog entry and run the full-sweep-fail-fast sweep against all five unretired 8k/1k baseline points.

中文

初次尝试 · 已通过 · Run 38964094385 / attempt 1 · 2026-10-16 21:23 UTC
lmsysorg/sglang:v0.5.21-rocm720-mi30x@sha256:0534eb382626ef55ace8be24f7d81cc6eb7451acac8a5727848397a5e9309d91 · f8ba650b6fb1 · 8k/1k · TP8/EP1 · 平均延迟
**变更:**测量 ref 49d9e49:按 digest 固定 lmsysorg/sglang:v0.5.21-rocm720-mi30x(v0.5.19 之后不再提供 rocm700 构建),并将 cuda-graph-max-bs 改为 cuda-graph-max-bs-decode,因为 v0.5.21 移除了该弃用别名(sgl-project/sglang#38375,commit db272201a2;v0.5.12 的 server_args.py 中它即 decode 图上限);其余 recipe 参数以及 SGLANG_AITER_MLA_PERSIST/SGLANG_USE_AITER 在 v0.5.21 e00930c5 中仍被解析。实测数值及异常说明见上表。
**下一步:**追加该配置族的 perf-changelog 条目,并以 full-sweep-fail-fast 运行完整 sweep,覆盖全部 5 个未退役的 8k/1k 基线点。

…pdate

Append the perf-changelog entry for the MI300X FP8 SGLang image update.

为 MI300X FP8 SGLang 镜像更新追加 perf-changelog 条目。

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@Klaud-Cold Klaud-Cold added the full-sweep-fail-fast Full sweep with canary gate; first failure cancels the rest of that matrix (recommended) label Oct 16, 2026
@Klaud-Cold

Klaud-Cold commented Oct 16, 2026 •

Copy link
Copy Markdown
Collaborator Author

Final full sweep · Failed · Run 38969255712 / attempt 1 · 2026-10-16 22:21 UTC
lmsysorg/sglang:v0.5.21-rocm720-mi30x@sha256:0534eb382626ef55ace8be24f7d81cc6eb7451acac8a5727848397a5e9309d91 · 26586e5ec160
Change: Smoke passed on 49d9e49 (8k/1k c4 and gsm8k c32), so the exact-family perf-changelog entry was appended and the full family was swept with full-sweep-fail-fast; the 8k/1k c64 server died while resolving weights on a Hugging Face Hub 429 api rate limit, and fail-fast cancelled the other four benchmarks.

Note: 1k/1k c4 TP8 EP1 14b375, 1k/1k c8 TP8 EP1 c5e335, 1k/1k c16 TP8 EP1 a288ec, 1k/1k c32 TP8 EP1 fb7ddf, 1k/1k c64 TP8 EP1 e94574: Single-turn 1k1k retired 2026-07-17 in #2263 after this baseline; excluded from coverage.

Next: Rerun the failed and cancelled benchmarks on the same head as an infrastructure retry, reusing the passed c32/c64 evals.

中文

最终完整 sweep · 失败 · Run 38969255712 / attempt 1 · 2026-10-16 22:21 UTC
lmsysorg/sglang:v0.5.21-rocm720-mi30x@sha256:0534eb382626ef55ace8be24f7d81cc6eb7451acac8a5727848397a5e9309d91 · 26586e5ec160
**变更:**49d9e497 上的冒烟测试(8k/1k c4 与 gsm8k c32)已通过,因此追加了该配置族的 perf-changelog 条目,并以 full-sweep-fail-fast 进行完整 sweep;8k/1k c64 的服务在解析权重时因 Hugging Face Hub 429 api 限流退出,fail-fast 随后取消了其余四个基准。实测数值及异常说明见上表。
**下一步:**在同一提交上作为基础设施重试重新运行失败和被取消的基准,复用已通过的 c32/c64 评测。

@github-actions

Copy link
Copy Markdown
Contributor

@Klaud-Cold

Klaud-Cold commented Oct 16, 2026 •

Copy link
Copy Markdown
Collaborator Author

Infrastructure retry 1 · Failed · Run 38969255712 / attempt 2 · 2026-10-16 22:34 UTC
lmsysorg/sglang:v0.5.21-rocm720-mi30x@sha256:0534eb382626ef55ace8be24f7d81cc6eb7451acac8a5727848397a5e9309d91 · 26586e5ec160
Change: Rerun the failed 8k/1k c64 benchmark and the four fail-fast-cancelled benchmarks on the same head 26586e5 after a transient Hugging Face Hub 429 api rate limit during weight resolution; c8 and c32 then failed on nodes e07-12 and e07-40, whose node-local DeepSeek-R1-0528 snapshot is missing 161 and 157 of 163 shards, and fail-fast cancelled c4/c16/c64.

Next: Run the second and last infrastructure retry of the failed and cancelled benchmarks on the same head; the c32/c64 evals stay reused.

中文

基础设施重试 1 · 失败 · Run 38969255712 / attempt 2 · 2026-10-16 22:34 UTC
lmsysorg/sglang:v0.5.21-rocm720-mi30x@sha256:0534eb382626ef55ace8be24f7d81cc6eb7451acac8a5727848397a5e9309d91 · 26586e5ec160
**变更:**在权重解析期间遇到临时的 Hugging Face Hub 429 api 限流后,在同一提交 26586e5 上重新运行失败的 8k/1k c64 基准以及被 fail-fast 取消的四个基准;随后 c8 和 c32 在节点 e07-12 和 e07-40 上失败,这两个节点本地的 DeepSeek-R1-0528 快照分别缺少 163 个分片中的 161 个和 157 个,fail-fast 取消了 c4/c16/c64。实测数值及异常说明见上表。
**下一步:**在同一提交上对失败和被取消的基准进行第二次也是最后一次基础设施重试;继续复用 c32/c64 评测。

@Klaud-Cold

Klaud-Cold commented Oct 16, 2026 •

Copy link
Copy Markdown
Collaborator Author

Infrastructure retry 2 · Failed · Run 38969255712 / attempt 3 · 2026-10-16 22:44 UTC
lmsysorg/sglang:v0.5.21-rocm720-mi30x@sha256:0534eb382626ef55ace8be24f7d81cc6eb7451acac8a5727848397a5e9309d91 · 26586e5ec160
Change: Second and last infrastructure retry of the five 8k/1k benchmarks on the same head 26586e5; c64 failed again on node e07-31, whose node-local DeepSeek-R1-0528 snapshot is missing 148 of 163 shards, and fail-fast cancelled the other four, while the c32/c64 gsm8k evals and the smoke c4 point passed on nodes with complete weights.

Next: Stop as readiness-blocked: MI300X nodes e07-12, e07-31 and e07-40 need complete staged DeepSeek-R1-0528 weights before this image update can be validated.

中文

基础设施重试 2 · 失败 · Run 38969255712 / attempt 3 · 2026-10-16 22:44 UTC
lmsysorg/sglang:v0.5.21-rocm720-mi30x@sha256:0534eb382626ef55ace8be24f7d81cc6eb7451acac8a5727848397a5e9309d91 · 26586e5ec160
**变更:**在同一提交 26586e5 上对 5 个 8k/1k 基准进行第二次也是最后一次基础设施重试;c64 再次在节点 e07-31 上失败,该节点本地的 DeepSeek-R1-0528 快照缺少 163 个分片中的 148 个,fail-fast 取消了其余四个基准;c32/c64 gsm8k 评测和冒烟 c4 点在权重完整的节点上均已通过。实测数值及异常说明见上表。
**下一步:**以 readiness-blocked 结束:需要先在 MI300X 节点 e07-12、e07-31 和 e07-40 上补全 DeepSeek-R1-0528 权重,才能验证此次镜像更新。

@Klaud-Cold

Copy link
Copy Markdown
Collaborator Author

readiness-blocked · Cleanup pending. Stop and confirm owned runs before closing.

中文

readiness-blocked · 清理待完成。先停止并确认自有运行结束,再关闭 PR。

@Klaud-Cold Klaud-Cold removed the full-sweep-fail-fast Full sweep with canary gate; first failure cancels the rest of that matrix (recommended) label Oct 16, 2026
@Klaud-Cold Klaud-Cold closed this Oct 16, 2026
@Klaud-Cold
Klaud-Cold deleted the klaud/auto-4bfcce0abcfcb357-58417c62beb2fe3c branch October 16, 2026 22:45
@Klaud-Cold

Copy link
Copy Markdown
Collaborator Author

readiness-blocked · Repairs: 0 · Runs: 38964094385, 38969177491, 38969255712
All owned runs ended. PR closed; branch deleted for retry.

中文

readiness-blocked · 修复次数:0 · 运行:38964094385, 38969177491, 38969255712
所有自有运行均已结束。PR 已关闭;分支已删除,可重新尝试。

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

Status: Done

Development

Successfully merging this pull request may close these issues.

1 participant