Skip to content

Commit 1903073

Browse files
committed
fix: disable GLM-5.2 GB300 draft MoE FP8 conversion
Set SGLANG_NVFP4_CKPT_FP8_NEXTN_MOE=0 so the NextN/MTP draft keeps its shipped precision, and use the 1800 s scheduler watchdog of the GLM-5.2 B200/GB200 recipes for disaggregated prefill and decode. 设置 SGLANG_NVFP4_CKPT_FP8_NEXTN_MOE=0,使 NextN/MTP draft 保持原始发布精度; 分离式 prefill 和 decode 采用 GLM-5.2 B200/GB200 配方的 1800 秒调度 watchdog。 Co-authored-by: Wenyao Gao <edwingao28@users.noreply.github.com>
1 parent 5097c7d commit 1903073

3 files changed

Lines changed: 16 additions & 2 deletions

File tree

‎inferencex-e2e/benchmarks/multi_node/srt-slurm-recipes/glm5.2/sglang/gb300-fp4/agentx/agg.yaml‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -48,7 +48,7 @@ roles:
4848
SGLANG_HICACHE_DEBUG_LOG: '1'
4949
SGLANG_HICACHE_DEBUG_SAMPLE_RATE: '16384'
5050
SGLANG_MOE_NVFP4_DISPATCH: '1'
51-
SGLANG_NVFP4_CKPT_FP8_NEXTN_MOE: '1'
51+
SGLANG_NVFP4_CKPT_FP8_NEXTN_MOE: '0'
5252
SGLANG_CLIP_MAX_NEW_TOKENS_ESTIMATION: '8'
5353
PIP_BREAK_SYSTEM_PACKAGES: '1'
5454
SGLANG_DG_CACHE_DIR: /deepgemm_cache

‎inferencex-e2e/benchmarks/multi_node/srt-slurm-recipes/glm5.2/sglang/gb300-fp4/agentx/disagg-mtp-variants.yaml‎

Lines changed: 3 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -87,6 +87,7 @@ base:
8787
speculative-num-steps: 1
8888
speculative-eagle-topk: 1
8989
speculative-num-draft-tokens: 2
90+
watchdog-timeout: 1800
9091
enable-metrics: true
9192
enable-cache-report: true
9293
decode:
@@ -106,7 +107,7 @@ base:
106107
SGLANG_DISABLE_TP_MEMORY_INBALANCE_CHECK: '1'
107108
SGLANG_DEEPEP_NUM_MAX_DISPATCH_TOKENS_PER_RANK: '512'
108109
SGLANG_MOE_NVFP4_DISPATCH: '1'
109-
SGLANG_NVFP4_CKPT_FP8_NEXTN_MOE: '1'
110+
SGLANG_NVFP4_CKPT_FP8_NEXTN_MOE: '0'
110111
SGLANG_ENABLE_THINKING: '1'
111112
SGLANG_CLIP_MAX_NEW_TOKENS_ESTIMATION: '8'
112113
SGLANG_REASONING_EFFORT: max
@@ -147,6 +148,7 @@ base:
147148
model-loader-extra-config: '{"enable_multithread_load": true}'
148149
mem-fraction-static: 0.9
149150
disaggregation-decode-extra-slots: 0
151+
watchdog-timeout: 1800
150152
enable-metrics: true
151153
enable-cache-report: true
152154
health_check:

‎inferencex-e2e/perf-changelog.yaml‎

Lines changed: 12 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -8995,3 +8995,15 @@
89958995
- "Validate MiniMax-M3 AgentX on H200 with vLLM FP8 after moving the end-to-end project into inferencex-e2e/. Preserve the existing image, recipe, and benchmark settings."
89968996
- "将端到端项目迁入 inferencex-e2e/ 后,验证 H200 上的 vLLM FP8 MiniMax-M3 AgentX 运行路径,沿用现有镜像、配方和基准测试设置。"
89978997
pr-link: https://github.com/SemiAnalysisAI/InferenceX/pull/3525
8998+
8999+
- config-keys:
9000+
- glm5.2-fp4-gb300-dynamo-sglang-agentic-agg
9001+
- glm5.2-fp4-gb300-dynamo-sglang-agentic-disagg
9002+
scenario-type:
9003+
- agentic-coding
9004+
description:
9005+
- "Disable FP8 conversion of the GLM-5.2 NextN/MTP draft MoE on GB300 AgentX so the draft runs at its shipped precision; images, topology, workload and acceptance are unchanged."
9006+
- "Disaggregated prefill and decode use the 1800 s scheduler watchdog of the GLM-5.2 B200/GB200 recipes instead of the 300 s default."
9007+
- "关闭 GB300 GLM-5.2 AgentX 对 NextN/MTP draft MoE 的 FP8 转换,使 draft 保持原始发布精度;镜像、拓扑、工作负载和验收不变。"
9008+
- "分离式 prefill 和 decode 采用 GLM-5.2 B200/GB200 配方的 1800 秒调度 watchdog,替代默认 300 秒。"
9009+
pr-link: https://github.com/SemiAnalysisAI/InferenceX/pull/3402

0 commit comments

Comments
 (0)