Skip to content

Commit b831e60

Browse files
Klaud-Coldclaude[bot]claudeadibarra
authored
[Klaud Cold] Update glm5.2-fp8-mi325x-sglang-agentic-mtp SGLang ROCm image to v0.5.20-rocm720-mi30x / 将 glm5.2-fp8-mi325x-sglang-agentic-mtp 的 SGLang ROCm 镜像更新至 v0.5.20-rocm720-mi30x (#3362)
* feat(glm5.2-mi325x): update SGLang ROCm image to v0.5.20-rocm720-mi30x Update glm5.2-fp8-mi325x-sglang-agentic-mtp from lmsysorg/sglang:v0.5.19-rocm720-mi30x to lmsysorg/sglang:v0.5.20-rocm720-mi30x and rename the retired --cuda-graph-max-bs alias to --cuda-graph-max-bs-decode in the recipe (sgl-project/sglang#38375). 将 glm5.2-fp8-mi325x-sglang-agentic-mtp 的 SGLang ROCm 镜像从 v0.5.19-rocm720-mi30x 更新至 v0.5.20-rocm720-mi30x,并将配方中已被上游移除的 --cuda-graph-max-bs 别名 改为 --cuda-graph-max-bs-decode(sgl-project/sglang#38375)。 Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> * chore(changelog): record glm5.2-fp8-mi325x-sglang-agentic-mtp v0.5.20 image refresh Append the perf-changelog entry for the SGLang ROCm image update to v0.5.20-rocm720-mi30x and the --cuda-graph-max-bs-decode rename. 为 SGLang ROCm 镜像更新至 v0.5.20-rocm720-mi30x 及 --cuda-graph-max-bs-decode 重命名追加 perf-changelog 条目。 Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> --------- Co-authored-by: claude[bot] <claude[bot]@users.noreply.github.com> Co-authored-by: Claude Fable 5.1 <noreply@anthropic.com> Co-authored-by: adibarra <93070681+adibarra@users.noreply.github.com>
1 parent 8490145 commit b831e60

3 files changed

Lines changed: 11 additions & 2 deletions

File tree

‎benchmarks/single_node/agentic/glm5.2_fp8_mi325x_mtp.sh‎

Lines changed: 3 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -91,7 +91,9 @@ SGLANG_CMD=(
9191
--chunked-prefill-size 131072
9292
--mem-fraction-static 0.85
9393
--max-running-requests "$MAX_RUNNING_REQUESTS"
94-
--cuda-graph-max-bs "$MAX_RUNNING_REQUESTS"
94+
# SGLang v0.5.20 retired the deprecated --cuda-graph-max-bs alias
95+
# (sgl-project/sglang#38375); the decode-phase flag is the same setting.
96+
--cuda-graph-max-bs-decode "$MAX_RUNNING_REQUESTS"
9597
--speculative-algorithm EAGLE
9698
--speculative-num-steps 3
9799
--speculative-eagle-topk 1

‎configs/amd-master.yaml‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -1050,7 +1050,7 @@ dsv41flash-fp4-mi300x-vllm-agentic-dspark:
10501050
# GPU-resident-KV c1/c2/c3/c4/c5/c6/c8 curve from Actions run 29657732517
10511051
# and enables EAGLE MTP with the committed thinking-on golden AL.
10521052
glm5.2-fp8-mi325x-sglang-agentic-mtp:
1053-
image: lmsysorg/sglang:v0.5.19-rocm720-mi30x
1053+
image: lmsysorg/sglang:v0.5.20-rocm720-mi30x
10541054
model: zai-org/GLM-5.2-FP8
10551055
model-prefix: glm5.2
10561056
runner: cluster:mi325x-amds

‎perf-changelog.yaml‎

Lines changed: 7 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -8931,3 +8931,10 @@
89318931
- "Use five-token DSpark with thinking_on golden AL 3.51 for throughput and real acceptance for eval, the shipped draft, and dsml_v41 tool parsing. Replay semianalysis_cc_traces_weka_062126 for 3600 seconds per point with five warmup requests per lane."
89328932
- "DSpark draft layers 37-39 keep their checkpoint precision: the recipe passes no --online_quant_config, so online_quant_config resolves to None and make_v4_quant_config builds the same per-layer quant spec for the drafter as for the target. The MTP weights (mtp.0/1/2) load inline from the same checkpoint with spec_decode=True, and config.get_layer_quant_config has no mtp.* special case, so layers 37-39 are neither re-quantized nor up/down-cast by ATOM and retain their on-disk dtype. --kv_cache_dtype bf16 and --index-cache-dtype fp8 touch cache storage only, not draft/MTP weights."
89338933
pr-link: https://github.com/SemiAnalysisAI/InferenceX/pull/3387
8934+
8935+
- config-keys:
8936+
- glm5.2-fp8-mi325x-sglang-agentic-mtp
8937+
description:
8938+
- "Update SGLang ROCm image to v0.5.20-rocm720-mi30x; replace retired --cuda-graph-max-bs with --cuda-graph-max-bs-decode."
8939+
- "将 SGLang ROCm 镜像更新至 v0.5.20-rocm720-mi30x,并以 --cuda-graph-max-bs-decode 替换已移除的 --cuda-graph-max-bs。"
8940+
pr-link: https://github.com/SemiAnalysisAI/InferenceX/pull/3362

0 commit comments

Comments
 (0)