File tree Expand file tree Collapse file tree
benchmarks/single_node/srt-slurm-recipes/dsv4/vllm/mi355x-fp4-mtp Expand file tree Collapse file tree Original file line number Diff line number Diff line change 3939 data-parallel-size : 1
4040 gpu-memory-utilization : 0.86
4141 moe-backend : aiter
42+ monokernel : true
4243 compilation-config : ' {"mode":3,"cudagraph_mode":"FULL_AND_PIECEWISE"}'
4344 # Six-token DSpark with probabilistic drafting. Throughput runs replace
4445 # verification with the golden acceptance length.
@@ -167,6 +168,8 @@ override_dep8_c64:
167168 args :
168169 tensor-parallel-size : 1
169170 data-parallel-size : 8
171+ # Pro's monokernel requires TP8 without data parallelism.
172+ monokernel : false
170173 prefill-schedule-interval : 8
171174 long-prefill-token-threshold : 16384
172175 max-num-seqs : 64
@@ -189,6 +192,7 @@ override_dep8_c128:
189192 args :
190193 tensor-parallel-size : 1
191194 data-parallel-size : 8
195+ monokernel : false
192196 prefill-schedule-interval : 8
193197 long-prefill-token-threshold : 16384
194198 max-num-seqs : 128
@@ -211,6 +215,7 @@ override_dep8_c192:
211215 args :
212216 tensor-parallel-size : 1
213217 data-parallel-size : 8
218+ monokernel : false
214219 prefill-schedule-interval : 8
215220 long-prefill-token-threshold : 16384
216221 max-num-seqs : 192
Original file line number Diff line number Diff line change @@ -960,7 +960,7 @@ qwen3.5-fp8-mi325x-sglang-mtp:
960960# DeepSeek-V4-Pro-0813 FP4 AgentX on one MI355X node using vLLM DSpark K6.
961961# Keep the historical config key for changelog compatibility.
962962dsv4-fp4-mi355x-vllm-agentic-mtp :
963- image : vllm/vllm-openai-rocm:nightly-e9757321527ca1ecd514c07c1418dd2c53da3d19
963+ image : vllm/vllm-openai-rocm:nightly-e4a5b5537344b103ed10de7764378ef09604ef3d
964964 model : deepseek-ai/DeepSeek-V4-Pro-0813
965965 model-prefix : dsv4
966966 runner : cluster:mi355x-amds
Original file line number Diff line number Diff line change 1120411204 - "Add TP2 points with DRAM KV offload, and adjust the memory settings on the DRAM KV offload points."
1120511205 pr-link: https://github.com/SemiAnalysisAI/InferenceX/pull/XXX
1120611206 scenario-type: [agentic-coding]
11207+
11208+ - config-keys:
11209+ - dsv4-fp4-mi355x-vllm-agentic-mtp
11210+ description:
11211+ - "Update DeepSeek-V4-Pro-0813 on MI355X to vllm/vllm-openai-rocm:nightly-e4a5b5537344b103ed10de7764378ef09604ef3d and enable --monokernel for TP8 points; keep it disabled for data-parallel points."
11212+ pr-link: https://github.com/SemiAnalysisAI/InferenceX/pull/XXX
11213+ scenario-type: [agentic-coding]
You can’t perform that action at this time.
0 commit comments