Skip to content

Commit 6a451cf

Browse files
perf(dsv41): pin B300 vLLM nightly ddd6fbca (#3458)
将 B300 DeepSeek-V4.1-Flash vLLM 配方固定至 nightly-ddd6fbca。 Co-authored-by: Xin Li <xinli@nvidia.com>
1 parent ee18d78 commit 6a451cf

3 files changed

Lines changed: 14 additions & 4 deletions

File tree

‎inferencex-e2e/benchmarks/single_node/srt-slurm-recipes/dsv41flash/vllm/b300-fp4-mtp/agentic.yaml‎

Lines changed: 7 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -6,7 +6,7 @@ base:
66
name: dsv41flash-fp4-b300-vllm-agentic
77
model:
88
path: hf:deepseek-ai/DeepSeek-V4.1-Flash
9-
container: vllm/vllm-openai:deepseekv41-flash-0909
9+
container: vllm/vllm-openai:nightly-ddd6fbca148a867aad1fcab7ec72f582b9977db4
1010
precision: fp4
1111
resources:
1212
gpu_type: b300
@@ -24,7 +24,7 @@ base:
2424
# The engine may take the full VLLM_ENGINE_READY_TIMEOUT_S to become ready.
2525
health_check:
2626
interval_seconds: 10
27-
max_attempts: 360
27+
max_attempts: 720
2828
roles:
2929
agg:
3030
nodes: 1
@@ -37,6 +37,10 @@ base:
3737
enable-auto-tool-choice: true
3838
reasoning-parser: deepseek_v41
3939
engram-config: '{"cpu_offload":true}'
40+
# FlashInfer sparse attention with the merged Blackwell sparse indexer
41+
# (MXFP4 indexer KV, sparse logits) at both TP sizes; fp8 KV cache.
42+
attention-config: '{"backend":"FLASHINFER_MLA_SPARSE_DSV41","indexer_kv_dtype":"mxfp4","indexer_sparse_logits":true}'
43+
kv-cache-dtype: fp8
4044
# Five-token DSpark with probabilistic drafting. Throughput runs replace
4145
# block rejection with the golden acceptance length.
4246
speculative-config: '{"method":"dspark","num_speculative_tokens":5,"draft_sample_method":"probabilistic","rejection_sample_method":"block","enable_adaptive_verification":false}'
@@ -46,7 +50,7 @@ base:
4650
env:
4751
VLLM_USE_RUST_FRONTEND: '1'
4852
PYTHONUNBUFFERED: '1'
49-
VLLM_ENGINE_READY_TIMEOUT_S: '3600'
53+
VLLM_ENGINE_READY_TIMEOUT_S: '7200'
5054
benchmark:
5155
type: custom
5256
command: bash /infmax-workspace/benchmarks/srt_agentic.sh

‎inferencex-e2e/configs/nvidia-master.yaml‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -8324,7 +8324,7 @@ dsv41flash-fp4-gb200-sglang-agentic-dspark:
83248324
- { tp: 2, ep: 1, kv-offloading: none, spec-decoding: mtp, conc-list: [1, 2, 4, 8, 16, 32, 64, 128], srt-recipe: benchmarks/single_node/srt-slurm-recipes/dsv41flash/sglang/gb200-fp4-mtp/agentic.yaml }
83258325

83268326
dsv41flash-fp4-b300-vllm-agentic-dspark:
8327-
image: vllm/vllm-openai:deepseekv41-flash-0909
8327+
image: vllm/vllm-openai:nightly-ddd6fbca148a867aad1fcab7ec72f582b9977db4
83288328
model: deepseek-ai/DeepSeek-V4.1-Flash
83298329
model-prefix: dsv41flash
83308330
runner: cluster:b300-dsxe

‎inferencex-e2e/perf-changelog.yaml‎

Lines changed: 6 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -9012,3 +9012,9 @@
90129012
- "Update the GLM-5.2 MXFP4 MI355X SGLang AgentX image from lmsysorg/sglang-rocm:v0.5.20-rocm720-mi35x-20260923 to lmsysorg/sglang-rocm:v0.5.20-rocm720-mi35x-20260924 (digest sha256:baadaba198e23c46c1dd651b5edefae8000bd5430943bd10b246b563905b3ecf); keep all serving flags, HiCache settings, and sweep points unchanged."
90139013
- "将 GLM-5.2 MXFP4 MI355X SGLang AgentX 镜像从 lmsysorg/sglang-rocm:v0.5.20-rocm720-mi35x-20260923 更新到 lmsysorg/sglang-rocm:v0.5.20-rocm720-mi35x-20260924(digest sha256:baadaba198e23c46c1dd651b5edefae8000bd5430943bd10b246b563905b3ecf);其余服务参数、HiCache 设置与 sweep 点保持不变。"
90149014
pr-link: https://github.com/SemiAnalysisAI/InferenceX/pull/3446
9015+
9016+
- config-keys:
9017+
- dsv41flash-fp4-b300-vllm-agentic-dspark
9018+
description:
9019+
- "Pin B300 DeepSeek-V4.1-Flash vLLM to nightly ddd6fbca with FlashInfer sparse attention, MXFP4 indexer KV, sparse logits, fp8 KV cache, and a 7200 s readiness timeout."
9020+
pr-link: https://github.com/SemiAnalysisAI/InferenceX/pull/3458

0 commit comments

Comments
 (0)