Repository navigation
Commit 5e70328
[AgentX] GB300 DSV4.1 Flash SGLang: fold DSpark sampling, retune c64/c128 / [AgentX] GB300 DSV4.1 Flash SGLang:折叠 DSpark 采样并重调 c64/c128 (#3782)
* perf(agentx): fold DSpark sampling and retune c64/c128 for GB300 DSV4.1 Flash SGLang
Bump the image to nightly-dev-cu13-20260929-79cafec0 and set
SGLANG_DSPARK_FOLDED_SAMPLING=2 on every point: the AUTO default keeps
DSpark draft sampling eager for DeepSeek-V4.1 TP shards, which left
low-concurrency decode launch-bound. Run TP4 c64 with 8K chunks and a
16-step prefill-decode interval, and let TP2 c128 admit all 128 sessions
with a 4-step interval and mem-fraction-static 0.85.
中文:将镜像升级到 nightly-dev-cu13-20260929-79cafec0,并在所有点上设置
SGLANG_DSPARK_FOLDED_SAMPLING=2:AUTO 默认值对 DeepSeek-V4.1 的 TP 分片
保持 DSpark draft 采样为 eager 执行,导致低并发 decode 受 kernel 发射限制。
TP4 c64 改为 8K prefill chunk 与 16 步 prefill-decode interval;TP2 c128
接纳全部 128 个会话,使用 4 步 interval 与 mem-fraction-static 0.85。
* chore(changelog): link the GB300 DSV4.1 Flash SGLang entry to #3782
中文:将 GB300 DSV4.1 Flash SGLang 的 changelog 条目链接到 #3782。
* chore: refresh PR #3782 for sweep reuse [skip-sweep]
---------
Co-authored-by: weireweire <20922698+weireweire@users.noreply.github.com>
Co-authored-by: functionstackx <47992694+functionstackx@users.noreply.github.com>1 parent b9cb358 commit 5e70328
3 files changed
Lines changed: 31 additions & 10 deletions
File tree
- inferencex-e2e
- benchmarks/single_node/srt-slurm-recipes/dsv41flash/sglang/gb300-fp4-mtp
- configs
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
5 | 5 | | |
6 | 6 | | |
7 | 7 | | |
8 | | - | |
| 8 | + | |
9 | 9 | | |
10 | 10 | | |
11 | 11 | | |
| |||
47 | 47 | | |
48 | 48 | | |
49 | 49 | | |
| 50 | + | |
| 51 | + | |
| 52 | + | |
| 53 | + | |
50 | 54 | | |
51 | 55 | | |
52 | 56 | | |
| |||
63 | 67 | | |
64 | 68 | | |
65 | 69 | | |
66 | | - | |
67 | | - | |
68 | | - | |
69 | | - | |
70 | | - | |
| 70 | + | |
| 71 | + | |
| 72 | + | |
| 73 | + | |
| 74 | + | |
| 75 | + | |
71 | 76 | | |
72 | 77 | | |
73 | 78 | | |
| |||
202 | 207 | | |
203 | 208 | | |
204 | 209 | | |
205 | | - | |
| 210 | + | |
| 211 | + | |
| 212 | + | |
| 213 | + | |
| 214 | + | |
| 215 | + | |
206 | 216 | | |
207 | 217 | | |
208 | 218 | | |
| |||
337 | 347 | | |
338 | 348 | | |
339 | 349 | | |
340 | | - | |
341 | | - | |
| 350 | + | |
| 351 | + | |
342 | 352 | | |
343 | 353 | | |
344 | 354 | | |
| |||
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
9226 | 9226 | | |
9227 | 9227 | | |
9228 | 9228 | | |
9229 | | - | |
| 9229 | + | |
9230 | 9230 | | |
9231 | 9231 | | |
9232 | 9232 | | |
| |||
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
9517 | 9517 | | |
9518 | 9518 | | |
9519 | 9519 | | |
| 9520 | + | |
| 9521 | + | |
| 9522 | + | |
| 9523 | + | |
| 9524 | + | |
| 9525 | + | |
| 9526 | + | |
| 9527 | + | |
| 9528 | + | |
| 9529 | + | |
| 9530 | + | |
0 commit comments