Skip to content

Commit 3cf29c6

Browse files
committed
fix: disable GLM-5.2 GB300 draft MoE FP8 conversion
Set SGLANG_NVFP4_CKPT_FP8_NEXTN_MOE=0 so the NextN/MTP draft keeps its shipped precision, and use the 1800 s scheduler watchdog of the GLM-5.2 B200/GB200 recipes for disaggregated prefill and decode. 设置 SGLANG_NVFP4_CKPT_FP8_NEXTN_MOE=0,使 NextN/MTP draft 保持原始发布精度; 分离式 prefill 和 decode 采用 GLM-5.2 B200/GB200 配方的 1800 秒调度 watchdog。
1 parent 45aa3a2 commit 3cf29c6

3 files changed

Lines changed: 16 additions & 2 deletions

File tree

‎benchmarks/multi_node/srt-slurm-recipes/glm5.2/sglang/gb300-fp4/agentx/agg.yaml‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -48,7 +48,7 @@ roles:
4848
SGLANG_HICACHE_DEBUG_LOG: '1'
4949
SGLANG_HICACHE_DEBUG_SAMPLE_RATE: '16384'
5050
SGLANG_MOE_NVFP4_DISPATCH: '1'
51-
SGLANG_NVFP4_CKPT_FP8_NEXTN_MOE: '1'
51+
SGLANG_NVFP4_CKPT_FP8_NEXTN_MOE: '0'
5252
SGLANG_CLIP_MAX_NEW_TOKENS_ESTIMATION: '8'
5353
PIP_BREAK_SYSTEM_PACKAGES: '1'
5454
SGLANG_DG_CACHE_DIR: /deepgemm_cache

‎benchmarks/multi_node/srt-slurm-recipes/glm5.2/sglang/gb300-fp4/agentx/disagg-mtp-variants.yaml‎

Lines changed: 3 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -87,6 +87,7 @@ base:
8787
speculative-num-steps: 1
8888
speculative-eagle-topk: 1
8989
speculative-num-draft-tokens: 2
90+
watchdog-timeout: 1800
9091
enable-metrics: true
9192
enable-cache-report: true
9293
decode:
@@ -106,7 +107,7 @@ base:
106107
SGLANG_DISABLE_TP_MEMORY_INBALANCE_CHECK: '1'
107108
SGLANG_DEEPEP_NUM_MAX_DISPATCH_TOKENS_PER_RANK: '512'
108109
SGLANG_MOE_NVFP4_DISPATCH: '1'
109-
SGLANG_NVFP4_CKPT_FP8_NEXTN_MOE: '1'
110+
SGLANG_NVFP4_CKPT_FP8_NEXTN_MOE: '0'
110111
SGLANG_ENABLE_THINKING: '1'
111112
SGLANG_CLIP_MAX_NEW_TOKENS_ESTIMATION: '8'
112113
SGLANG_REASONING_EFFORT: max
@@ -147,6 +148,7 @@ base:
147148
model-loader-extra-config: '{"enable_multithread_load": true}'
148149
mem-fraction-static: 0.9
149150
disaggregation-decode-extra-slots: 0
151+
watchdog-timeout: 1800
150152
enable-metrics: true
151153
enable-cache-report: true
152154
health_check:

‎perf-changelog.yaml‎

Lines changed: 12 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -8986,3 +8986,15 @@
89868986
- "Hopper 使用 FA3,Blackwell FA4 fp8 descale 问题不适用,EAGLE3 draft 保持 attention_backend FLASH_ATTN。"
89878987
- "This change does not alter the EAGLE3 draft model data type. The draft loads unmodified from the published Inferact/MiniMax-M3-EAGLE3-GQA checkpoint via --speculative-config (method=eagle3). kv-cache-dtype fp8 sets KV-cache storage precision, not the draft weights, and no flag overrides or re-quantizes the draft weights."
89888988
pr-link: https://github.com/SemiAnalysisAI/InferenceX/pull/3503
8989+
8990+
- config-keys:
8991+
- glm5.2-fp4-gb300-dynamo-sglang-agentic-agg
8992+
- glm5.2-fp4-gb300-dynamo-sglang-agentic-disagg
8993+
scenario-type:
8994+
- agentic-coding
8995+
description:
8996+
- "Disable FP8 conversion of the GLM-5.2 NextN/MTP draft MoE on GB300 AgentX so the draft runs at its shipped precision; images, topology, workload and acceptance are unchanged."
8997+
- "Disaggregated prefill and decode use the 1800 s scheduler watchdog of the GLM-5.2 B200/GB200 recipes instead of the 300 s default."
8998+
- "关闭 GB300 GLM-5.2 AgentX 对 NextN/MTP draft MoE 的 FP8 转换,使 draft 保持原始发布精度;镜像、拓扑、工作负载和验收不变。"
8999+
- "分离式 prefill 和 decode 采用 GLM-5.2 B200/GB200 配方的 1800 秒调度 watchdog,替代默认 300 秒。"
9000+
pr-link: https://github.com/SemiAnalysisAI/InferenceX/pull/3402

0 commit comments

Comments
 (0)