Repository navigation
[TokenSpeed] DeepSeek-V4.1-Flash FP4 GB300 AgentX / [TokenSpeed] 新增 DeepSeek-V4.1-Flash FP4 GB300 AgentX - #5105
fanyang-real wants to merge 4 commits into
Conversation
|
Thanks for the contribution!
中文感谢你的贡献!
|
a7bd40d to
9ee32ea
Compare
Add dsv41flash-fp4-gb300-tokenspeed-agentic-dspark on the unmodified lightseekorg/tokenspeed:rc-377b78 image: one DEP4 TokenSpeed server per GB300 node behind SMG with sticky grouped DP, per-concurrency rows, and DSPARK support in the TokenSpeed golden-AL connector. Signed-off-by: Fan Yang <250624800+fanyang-real@users.noreply.github.com>
9ee32ea to
bd807a1
Compare
|
Generally most InferenceX submissions don't need to add changes to the infx package. On the off chance they do, generally the diff is small. Please prompt your agent to clean your PR to keep your diff to only necessary & minimal changes too. We have an increased code quality bar for changes to the infx package. 中文大多数 InferenceX 提交通常不需要修改 infx 包。即使确实需要,改动通常也很小。请让你的 agent 清理 PR,使改动只包含必要且最小的内容。infx 包的改动有更高的代码质量要求。 |
|
Claude finished @fanyang-real's task in 29s —— View job LGTM - no blocking issues found. |
…300-agentx Signed-off-by: Fan Yang <250624800+fanyang-real@users.noreply.github.com> # Conflicts: # inferencex-e2e/perf-changelog.yaml
…nimum Keep the speculative-config error text, revert the reflowed README lines and the existing test parameters, and reduce the DSPARK test to the curve lookup. Signed-off-by: Fan Yang <250624800+fanyang-real@users.noreply.github.com>
…300-agentx Signed-off-by: Fan Yang <250624800+fanyang-real@users.noreply.github.com> # Conflicts: # inferencex-e2e/perf-changelog.yaml
|
View unofficial run (performance): https://inferencex.semianalysis.com/inference?unofficialRun=44014585473 View unofficial run (accuracy): https://inferencex.semianalysis.com/evaluation?unofficialRun=44014585473 |
Summary
Adds
dsv41flash-fp4-gb300-tokenspeed-agentic-dspark: DeepSeek-V4.1-Flash FP4 AgentX on GB300 with the unmodified TokenSpeed imagelightseekorg/tokenspeed:rc-377b78, pinned by digest (sha256:c5e3c416…). The upstream nightly Docker workflow built it from lightseekorg/tokenspeed@377b782, and it bundles thetokenspeed-smg1.16.0.post20261115gateway. There is no setup script and nothing is installed at job start.ts serveapplies;--grouped-dp-sticky hashoverx-dynamo-parent-session-id, thenx-correlation-id, keeps a session tree on one attention-DP rank.agg-variants.yaml) at concurrency 128, 176, 216, 240, 256, 288, 320, 352, 368, 384, 448, 512, 768 and 896, each on the measured frontier. Each row sets the running-request cap, the decode CUDA graphs and the prefill gate intervals. Rows up to 352 add--prefill-single-round, and rows up to 256 capture every decode graph size up to the cap / 4.DSPARKand selects the DeepSeek-V4.1-Flash DSpark curve fromspeculative-num-steps.Pre-submission validation on our own GB300 nodes with the pinned image (Unofficial; details below): all 14 rows completed. Dashboard tiers vs the current GB300 best (#4844): 840.8k / 766.4k / 638.1k / 550.6k / 514.3k / 442.3k / 271.8k tok/s/GPU at P90 50 / 75 / 100 / 120 / 125 / 150 / 200 (+11.4% / +27.2% / +22.2% / +16.2% / +11.0% / +6.0% / +2.2%). GSM8K at concurrency 768: strict-match 0.975 / 0.973 in two runs (pass mark 0.90).
中文
新增
dsv41flash-fp4-gb300-tokenspeed-agentic-dspark:在 GB300 上用未经修改的 TokenSpeed 镜像lightseekorg/tokenspeed:rc-377b78运行 DeepSeek-V4.1-Flash FP4 AgentX,配置中按 digest 固定(sha256:c5e3c416…)。该镜像由上游 nightly Docker 工作流从 lightseekorg/tokenspeed@377b782 构建,内置tokenspeed-smg1.16.0.post20261115网关。配置没有 setup 脚本,任务启动时不安装任何东西。ts serve的网关设置;--grouped-dp-sticky hash先按x-dynamo-parent-session-id、再按x-correlation-id路由,使同一会话树固定在一个 attention-DP rank 上。agg-variants.yaml),并发度为 128、176、216、240、256、288、320、352、368、384、448、512、768 和 896,均在实测前沿上。每个测点设置运行请求上限、解码 CUDA graph 和预填充门控间隔。并发度 352 及以下加--prefill-single-round,256 及以下捕获从 1 到上限 / 4 的每个解码 graph 尺寸。DSPARK,并根据speculative-num-steps选择 DeepSeek-V4.1-Flash 的 DSpark 曲线。提交前验证(在我们自己的 GB300 节点上使用固定镜像,非官方结果;详见下方):14 个测点全部完成。相对当前 GB300 最佳结果(#4844)的看板分档:P90 50 / 75 / 100 / 120 / 125 / 150 / 200 分别为 840.8k / 766.4k / 638.1k / 550.6k / 514.3k / 442.3k / 271.8k tok/s/GPU(+11.4% / +27.2% / +22.2% / +16.2% / +11.0% / +6.0% / +2.2%)。并发度 768 的 GSM8K 评估两次 strict-match 为 0.975 / 0.973(及格线 0.90)。并发度 512 的 68,554 个请求中有 1 个失败,其余测点全部成功,所有测点均通过提交有效性检查。
Validation details
Pre-submission runs on our own GB300 nodes, not an InferenceX sweep (Unofficial). Each row ran the pinned image by digest with the recipe's engine args and env, SMG with the recipe's gateway args, and the AgentX client from InferenceX cf2af9a for 3600 s.
One request of 68,554 failed at concurrency 512; every other row completed all requests, and every row met the submission validity checks.
GSM8K at concurrency 768 with real DSpark verification and FlashInfer sampling: strict-match 0.9750 / 0.9727, flexible-extract 0.9742 / 0.9719 (two runs).
AI model disclosure
PR checklist and change type
Type of Change
Checklist
python -m infx.matrix.changed --base origin/mainlists affected config keys, I have appended one new entry covering them to the physical end ofinferencex-e2e/perf-changelog.yaml; if it lists none, I have added none. I have not edited historical entries