Skip to content

[Klaud Cold] Update DSR1 FP8 B300 Dynamo-TRT image to 1.5.1 / [Klaud Cold] 将 DSR1 FP8 B300 Dynamo-TRT 镜像更新至 1.5.1 - #4218

Closed
Klaud-Cold wants to merge 2 commits into
mainfrom
klaud/auto-c9ea66acb3a4b265-71fd8d13974a868c
Closed

Klaud-Cold wants to merge 2 commits into
mainfrom
klaud/auto-c9ea66acb3a4b265-71fd8d13974a868c

Conversation

@Klaud-Cold

@Klaud-Cold Klaud-Cold commented Oct 16, 2026 •

Copy link
Copy Markdown
Collaborator

Goal: Update Dynamo TensorRT-LLM image from nvcr.io/nvidia/ai-dynamo/tensorrtllm-runtime:0.8.1.post1 to nvcr.io/nvidia/ai-dynamo/tensorrtllm-runtime:1.5.1@sha256:9a563fc8a59b9cc7b7281a52427077d6542ecd752d70af20dbb0508c0486d3ef.
Baseline: 2026-02-12 · nvcr.io/nvidia/ai-dynamo/tensorrtllm-runtime:0.8.1.post1
Mean latency · Sources: API 1, API 2

Point Total tok/s/GPU ↑ Output tok/s/GPU ↑ TTFT ms ↓ TPOT ms ↓
1k/1k c10 PTP4x1/DTP8x8 57e98f N/A N/A N/A N/A
1k/1k c12 PTP4x1/DTP8x8 da7d5c N/A N/A N/A N/A
1k/1k c128 PTP4x1/DTP8x8 3a620f 286.91 152.25 402.59 12.22
1k/1k c160 PTP4x1/DTP8x8 c8dc67 N/A N/A N/A N/A
1k/1k c384 PTP4x1/DTP8x8 3059c8 573.39 304.72 720.91 18.24
1k/1k c720 PTP4x1/DTP8x5 55f019 N/A N/A N/A N/A
1k/1k c1280 PTP4x1/DTP8x3 06380a 2,518.5 1,469.54 4,303.08 29.93
1k/1k c2112 PTP4x1/DTP8x1 a36add 5,727.29 4,296.6 4,047.92 54.82
1k/1k c2560 PTP4x1/DTP8x2 e1bc84 N/A N/A N/A N/A
1k/1k c3072 PTP4x1/DTP8x1 a2173b N/A N/A N/A N/A
1k/1k c3072 PTP4x1/DTP8x2 07358f 4,107.33 2,566.71 32,617.51 36.1
1k/1k c11264 PTP4x3/DTP8x2 c38df5 N/A N/A N/A N/A
1k/1k c16384 PTP4x2/DTP8x1 aa56fa 6,736.07 6,735.1 134,950.12 146.12
8k/1k c8 PTP4x1/DTP8x4 4f26dd N/A N/A N/A N/A
8k/1k c16 PTP4x1/DTP8x8 ba16d5 N/A N/A N/A N/A
8k/1k c20 PTP4x1/DTP8x4 b22644 N/A N/A N/A N/A
8k/1k c40 PTP4x1/DTP8x2 c6a23f N/A N/A N/A N/A
8k/1k c64 PTP4x1/DTP8x4 d371e7 1,051.42 131.21 1,901.25 12.59
8k/1k c72 PTP4x1/DTP8x1 d0b9dc N/A N/A N/A N/A
8k/1k c144 PTP4x2/DTP8x1 7332cd N/A N/A N/A N/A
8k/1k c256 PTP4x2/DTP8x1 771f27 4,313.9 958.84 2,298.37 29.46
8k/1k c256 PTP4x3/DTP8x5 6bba98 1,907.33 275.56 1,897.35 20.42
8k/1k c512 PTP4x3/DTP8x1 cc9596 5,841.77 1,622.6 3,001.27 34.53
8k/1k c512 PTP4x4/DTP8x1 511d7f N/A N/A N/A N/A
8k/1k c1075 PTP4x5/DTP8x1 1fe4d8 6,236.9 2,425.51 4,418.32 48.8
8k/1k c3072 PTP4x7/DTP8x1 939020 7,727.31 3,863.29 7,467.88 88.31

Note: 1k/1k c10 PTP4x1/DTP8x8 57e98f, 1k/1k c12 PTP4x1/DTP8x8 da7d5c, 1k/1k c160 PTP4x1/DTP8x8 c8dc67, 1k/1k c720 PTP4x1/DTP8x5 55f019, 1k/1k c2560 PTP4x1/DTP8x2 e1bc84, 1k/1k c3072 PTP4x1/DTP8x1 a2173b, 1k/1k c11264 PTP4x3/DTP8x2 c38df5, 8k/1k c8 PTP4x1/DTP8x4 4f26dd, 8k/1k c16 PTP4x1/DTP8x8 ba16d5, 8k/1k c20 PTP4x1/DTP8x4 b22644, 8k/1k c40 PTP4x1/DTP8x2 c6a23f, 8k/1k c72 PTP4x1/DTP8x1 d0b9dc, 8k/1k c144 PTP4x2/DTP8x1 7332cd, 8k/1k c512 PTP4x4/DTP8x1 511d7f: unavailable.
Note: 1k/1k c10 PTP4x1/DTP8x8 57e98f, 1k/1k c12 PTP4x1/DTP8x8 da7d5c, 1k/1k c128 PTP4x1/DTP8x8 3a620f, 1k/1k c160 PTP4x1/DTP8x8 c8dc67, 1k/1k c384 PTP4x1/DTP8x8 3059c8, 1k/1k c720 PTP4x1/DTP8x5 55f019, 1k/1k c1280 PTP4x1/DTP8x3 06380a, 1k/1k c2112 PTP4x1/DTP8x1 a36add, 1k/1k c2560 PTP4x1/DTP8x2 e1bc84, 1k/1k c3072 PTP4x1/DTP8x1 a2173b, 1k/1k c3072 PTP4x1/DTP8x2 07358f, 1k/1k c11264 PTP4x3/DTP8x2 c38df5, 1k/1k c16384 PTP4x2/DTP8x1 aa56fa: Single-turn 1k1k retired 2026-07-17 in #2263 after this baseline; excluded from coverage.

Eval: N/A

中文

**目标:**将 Dynamo TensorRT-LLM 镜像从 nvcr.io/nvidia/ai-dynamo/tensorrtllm-runtime:0.8.1.post1 更新至 nvcr.io/nvidia/ai-dynamo/tensorrtllm-runtime:1.5.1@sha256:9a563fc8a59b9cc7b7281a52427077d6542ecd752d70af20dbb0508c0486d3ef。
**基线:**2026-02-12 · nvcr.io/nvidia/ai-dynamo/tensorrtllm-runtime:0.8.1.post1
平均延迟 · 来源: API 1, API 2;数值及异常说明见上表。

将 dsr1-fp8-b300-dynamo-trt 的 Dynamo TensorRT-LLM 镜像更新至 1.5.1(按摘要固定)。

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@github-actions

Copy link
Copy Markdown
Contributor

Thanks for the contribution!

  • Review: If this PR changes files owned by someone other than a repository admin or @SemiAnalysisAI/core, ask one eligible CODEOWNER to complete the latest PR_REVIEW_CHECKLIST.md before contacting a core maintainer on Slack. Follow the template exactly, including As a PR reviewer and CODEOWNER, I have reviewed this and have, so sign-off verification triggers.
  • PR verification: Sweeps run only when the PR appends an inferencex-e2e/perf-changelog.yaml entry and carries exactly one primary label: full-sweep-fail-fast (strongly recommended; canary plus per-matrix fail-fast), full-sweep-enabled (canary; matrix jobs continue after a failure), or non-canary-full-sweep-enabled (no canary or fail-fast). The modifiers all-evals, evals-only, and agentx-fast require a primary label. On fork PRs, a maintainer applies the label. See sweep labels and reuse.
  • After merging: PR authors must ensure all GitHub Actions jobs pass. Transient failures often pass on rerun; see how to rerun failed jobs.
中文

感谢你的贡献!

  • **审阅:**如果 PR 修改的文件归属于仓库管理员及 @SemiAnalysisAI/core 之外的 CODEOWNER,请先联系一位有资格的 CODEOWNER 填写最新的 PR_REVIEW_CHECKLIST.md,再通过 Slack 联系核心维护者。必须严格遵循模板,并保留 As a PR reviewer and CODEOWNER, I have reviewed this and have,才能触发签核验证。
  • **PR 验证:**只有当 PR 在 inferencex-e2e/perf-changelog.yaml 末尾追加了条目,并且恰好带有一个主标签时,才会运行扫描:full-sweep-fail-fast(强烈推荐;canary 加逐矩阵 fail-fast)、full-sweep-enabled(有 canary;矩阵任务在失败后继续运行)或 non-canary-full-sweep-enabled(无 canary,也无 fail-fast)。修饰标签 all-evals、evals-only 和 agentx-fast 必须与主标签一起使用。fork PR 的标签由维护者添加。参见扫描标签与复用。
  • **合并后:**PR 作者必须确保所有 GitHub Actions 任务通过。临时性失败通常可以通过重新运行恢复;参见重新运行失败任务的说明。

保持路由器元数据不变,仅更新镜像。

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@Klaud-Cold

Copy link
Copy Markdown
Collaborator Author

Initial attempt · Cancelled · Run 38977286694 / attempt 1 · 2026-10-16 22:44 UTC
nvcr.io/nvidia/ai-dynamo/tensorrtllm-runtime:1.5.1@sha256:9a563fc8a59b9cc7b7281a52427077d6542ecd752d70af20dbb0508c0486d3ef · e31a56ebbf6d · Mean latency
Change: Measured ref 3f8ecd7 pinned the Dynamo TensorRT-LLM 1.5.1 image (TensorRT-LLM 1.2.0rc6.post2 to 1.3.0rc25, CUDA 13.0 to 13.2) and also relabeled router metadata.

Point Output tok/s/GPU ↑ TTFT ms ↓ TPOT ms ↓
8k/1k c40 PTP4x1/DTP8x2 c6a23f N/A N/A N/A
8k/1k c8 PTP4x1/DTP8x4 4f26dd N/A N/A N/A
8k/1k c20 PTP4x1/DTP8x4 b22644 N/A N/A N/A
8k/1k c72 PTP4x1/DTP8x1 d0b9dc N/A N/A N/A
8k/1k c144 PTP4x2/DTP8x1 7332cd N/A N/A N/A
8k/1k c512 PTP4x4/DTP8x1 511d7f N/A N/A N/A
8k/1k c64 PTP4x1/DTP8x4 d371e7 N/A N/A N/A
8k/1k c16 PTP4x1/DTP8x8 ba16d5 N/A N/A N/A
8k/1k c256 PTP4x2/DTP8x1 771f27 N/A N/A N/A
8k/1k c512 PTP4x3/DTP8x1 cc9596 N/A N/A N/A
8k/1k c256 PTP4x3/DTP8x5 6bba98 N/A N/A N/A
8k/1k c1075 PTP4x5/DTP8x1 1fe4d8 N/A N/A N/A
8k/1k c3072 PTP4x7/DTP8x1 939020 N/A N/A N/A

Note: All rows: cancelled.
Note: All rows: request errors unavailable.
Note: All rows: Δ N/A: point unavailable.

Next: Rerun the smoke on 573eeac, which keeps the image change only.

中文

初次尝试 · 已取消 · Run 38977286694 / attempt 1 · 2026-10-16 22:44 UTC
nvcr.io/nvidia/ai-dynamo/tensorrtllm-runtime:1.5.1@sha256:9a563fc8a59b9cc7b7281a52427077d6542ecd752d70af20dbb0508c0486d3ef · e31a56ebbf6d · 平均延迟
**变更:**测量 ref 3f8ecd7 按摘要固定 Dynamo TensorRT-LLM 1.5.1 镜像(TensorRT-LLM 1.2.0rc6.post2 升至 1.3.0rc25,CUDA 13.0 升至 13.2),并同时修改了路由器元数据标签。实测数值及异常说明见上表。
**下一步:**在仅保留镜像变更的 573eeac 上重新运行 smoke。

@Klaud-Cold

Klaud-Cold commented Oct 16, 2026 •

Copy link
Copy Markdown
Collaborator Author

Repair 1/5 · Failed · Run 38977531003 / attempt 1 · 2026-10-16 22:57 UTC
nvcr.io/nvidia/ai-dynamo/tensorrtllm-runtime:1.5.1@sha256:9a563fc8a59b9cc7b7281a52427077d6542ecd752d70af20dbb0508c0486d3ef · 62175beb470d · Mean latency
Change: Measured ref 573eeac keeps only the digest-pinned Dynamo 1.5.1 image change (TensorRT-LLM 1.3.0rc25); recipe flags were checked against source and left unchanged.

Point Output tok/s/GPU ↑ TTFT ms ↓ TPOT ms ↓
8k/1k c40 PTP4x1/DTP8x2 c6a23f N/A N/A N/A
8k/1k c8 PTP4x1/DTP8x4 4f26dd N/A N/A N/A
8k/1k c20 PTP4x1/DTP8x4 b22644 N/A N/A N/A
8k/1k c72 PTP4x1/DTP8x1 d0b9dc N/A N/A N/A
8k/1k c144 PTP4x2/DTP8x1 7332cd N/A N/A N/A
8k/1k c512 PTP4x4/DTP8x1 511d7f N/A N/A N/A
8k/1k c64 PTP4x1/DTP8x4 d371e7 N/A N/A N/A
8k/1k c16 PTP4x1/DTP8x8 ba16d5 N/A N/A N/A
8k/1k c256 PTP4x2/DTP8x1 771f27 N/A N/A N/A
8k/1k c512 PTP4x3/DTP8x1 cc9596 N/A N/A N/A
8k/1k c256 PTP4x3/DTP8x5 6bba98 N/A N/A N/A
8k/1k c1075 PTP4x5/DTP8x1 1fe4d8 N/A N/A N/A
8k/1k c3072 PTP4x7/DTP8x1 939020 N/A N/A N/A

Note: 8k/1k c40 PTP4x1/DTP8x2 c6a23f, 8k/1k c8 PTP4x1/DTP8x4 4f26dd, 8k/1k c20 PTP4x1/DTP8x4 b22644, 8k/1k c144 PTP4x2/DTP8x1 7332cd, 8k/1k c512 PTP4x4/DTP8x1 511d7f, 8k/1k c64 PTP4x1/DTP8x4 d371e7, 8k/1k c16 PTP4x1/DTP8x8 ba16d5, 8k/1k c256 PTP4x2/DTP8x1 771f27, 8k/1k c512 PTP4x3/DTP8x1 cc9596, 8k/1k c256 PTP4x3/DTP8x5 6bba98, 8k/1k c1075 PTP4x5/DTP8x1 1fe4d8, 8k/1k c3072 PTP4x7/DTP8x1 939020: cancelled.
Note: All rows: request errors unavailable.
Note: All rows: Δ N/A: point unavailable.
Note: 8k/1k c72 PTP4x1/DTP8x1 d0b9dc: failed.

Next: Readiness-blocked until DeepSeek-R1-0528 weights are staged on b300-dsxe scratch; closing this attempt without engine evidence.

中文

修复 1/5 · 失败 · Run 38977531003 / attempt 1 · 2026-10-16 22:57 UTC
nvcr.io/nvidia/ai-dynamo/tensorrtllm-runtime:1.5.1@sha256:9a563fc8a59b9cc7b7281a52427077d6542ecd752d70af20dbb0508c0486d3ef · 62175beb470d · 平均延迟
**变更:**测量 ref 573eeac 仅保留按摘要固定的 Dynamo 1.5.1 镜像变更(TensorRT-LLM 1.3.0rc25);配方参数已对照源码检查,保持不变。实测数值及异常说明见上表。
**下一步:**在 b300-dsxe 的 scratch 上预置 DeepSeek-R1-0528 权重之前处于就绪阻塞状态;本次尝试在没有引擎运行证据的情况下关闭。

@Klaud-Cold

Copy link
Copy Markdown
Collaborator Author

readiness-blocked · Cleanup pending. Stop and confirm owned runs before closing.

中文

readiness-blocked · 清理待完成。先停止并确认自有运行结束,再关闭 PR。

@Klaud-Cold Klaud-Cold closed this Oct 16, 2026
@Klaud-Cold
Klaud-Cold deleted the klaud/auto-c9ea66acb3a4b265-71fd8d13974a868c branch October 16, 2026 22:58
@Klaud-Cold

Copy link
Copy Markdown
Collaborator Author

readiness-blocked · Repairs: 1 · Runs: 38977286694, 38977531003
All owned runs ended. PR closed; branch deleted for retry.

中文

readiness-blocked · 修复次数:1 · 运行:38977286694, 38977531003
所有自有运行均已结束。PR 已关闭;分支已删除,可重新尝试。

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

Status: Done

Development

Successfully merging this pull request may close these issues.

1 participant