Skip to content

Commit 310784e

Browse files
hjjqclaude
andcommitted
Merge origin/main into dsv41flash-gb300-vllm-agentx-agg-main [skip-sweep]
Keep this PR's GB300 image (vLLM main a2bcb565) over #4558's nightly-00b6c4c5, which is 862 commits older. Resolve the perf-changelog conflict with main's entries kept and this PR's entry last, and name nightly-00b6c4c5 as the image this entry replaces. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2 parents 41ba8f7 + f7e72d4 commit 310784e

3 files changed

Lines changed: 24 additions & 3 deletions

File tree

‎inferencex-e2e/configs/amd-master.yaml‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -723,7 +723,7 @@ glm5.2-fp8-mi325x-sglang-agentic-mtp:
723723
# golden AL 3.51 for throughput, real block verification for evals, adaptive
724724
# verification off on ROCm.
725725
dsv41flash-fp4-mi325x-vllm-agentic-dspark:
726-
image: vllm/vllm-openai-rocm:v0.32.0@sha256:14f6a575e1fb0a580b0f633c09e06f648709d60405b85f6a7ff3f80d49f39905
726+
image: vllm/vllm-openai-rocm:v0.33.0@sha256:4fcd39b758b660c4e9c345c021e2ea8f0ccdb31e3c32507d676cc3005ae83fba
727727
model: deepseek-ai/DeepSeek-V4.1-Flash
728728
model-prefix: dsv41flash
729729
runner: cluster:mi325x-amd

‎inferencex-e2e/configs/nvidia-master.yaml‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -5772,7 +5772,7 @@ glm5.3flash-fp4-gb300-sglang-agentic-mtp:
57725772
# caps batched tokens at 4096. Concurrency is bounded by the measured KV
57735773
# ceiling rather than a guess; see the search-space comment.
57745774
dsv41flash-fp4-h100-vllm-agentic-dspark:
5775-
image: vllm/vllm-openai:nightly-e4a5b5537344b103ed10de7764378ef09604ef3d@sha256:6b90105076216e861cd30fae8828efef9263a0399137b894289f3b6a685bf4ff
5775+
image: vllm/vllm-openai:nightly-00b6c4c565a8732c131e1f452d4a17803bb0842b@sha256:76a9614ed4ed20694914c87e82a0344bd73e1c6e081459a2467175ab903657e4
57765776
model: deepseek-ai/DeepSeek-V4.1-Flash
57775777
model-prefix: dsv41flash
57785778
runner: cluster:h100-dsxe

‎inferencex-e2e/perf-changelog.yaml‎

Lines changed: 22 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -12309,10 +12309,31 @@
1230912309
- "HBM-only KV at conc 1, 2, 8, 16, 24 and 32; the cookbook HiCache L1 + L2 tier (180 GB per rank, write-through) at conc 40 and 48 recording its 749.33 GB host allocation (187.33 GB per rank) as a 751 GB CPU DRAM share. ROCM_QUICK_REDUCE_QUANTIZATION=NONE per the all-reduce policy."
1231012310
pr-link: https://github.com/SemiAnalysisAI/InferenceX/pull/4720
1231112311

12312+
- config-keys:
12313+
- dsv41flash-fp4-mi325x-vllm-agentic-dspark
12314+
description:
12315+
- "Update the vLLM ROCm image from v0.32.0 to v0.33.0, pinned by digest."
12316+
pr-link: https://github.com/SemiAnalysisAI/InferenceX/pull/4794
12317+
scenario-type: [agentic-coding]
12318+
12319+
- config-keys:
12320+
- dsv41flash-fp4-h100-vllm-agentic-dspark
12321+
description:
12322+
- "Update the vLLM image from nightly-e4a5b5537344 to nightly-00b6c4c565a8, pinned by digest."
12323+
pr-link: https://github.com/SemiAnalysisAI/InferenceX/pull/4569
12324+
scenario-type: [agentic-coding]
12325+
12326+
- config-keys:
12327+
- dsv41flash-fp4-gb300-vllm-agentic-dspark
12328+
description:
12329+
- "Update the vLLM image from nightly-dev-arm64-cu130-ac9126e58aa7 to the digest-pinned nightly-00b6c4c5."
12330+
pr-link: https://github.com/SemiAnalysisAI/InferenceX/pull/4558
12331+
scenario-type: [agentic-coding]
12332+
1231212333
- config-keys:
1231312334
- dsv41flash-fp4-gb300-vllm-agentic-dspark
1231412335
description:
12315-
- "Update the GB300 DeepSeek-V4.1-Flash vLLM AgentX image from vllm/vllm-openai:nightly-dev-arm64-cu130-ac9126e58aa7 to the vLLM main nightly vllm/vllm-openai:cu134-nightly-a2bcb5657a881a002e79d0adab3c6b5e39436121 (main a2bcb565, pinned to index sha256:76a533475156ffdb3656ccc389d0c71f5cec16897c0f490ac43e4f379c4b4f0d; arm64 sha256:24541db891c590e3a4022c7206c3ebb00e4be35442f54c20f2edf12eddfef4bf) and replace the search space with seventeen single-node points. Every point sets gpu-memory-utilization 0.93 and runs five-token DSpark with adaptive verification off."
12336+
- "Update the GB300 DeepSeek-V4.1-Flash vLLM AgentX image from vllm/vllm-openai:nightly-00b6c4c565a8732c131e1f452d4a17803bb0842b to the vLLM main nightly vllm/vllm-openai:cu134-nightly-a2bcb5657a881a002e79d0adab3c6b5e39436121 (main a2bcb565, pinned to index sha256:76a533475156ffdb3656ccc389d0c71f5cec16897c0f490ac43e4f379c4b4f0d; arm64 sha256:24541db891c590e3a4022c7206c3ebb00e4be35442f54c20f2edf12eddfef4bf) and replace the search space with seventeen single-node points. Every point sets gpu-memory-utilization 0.93 and runs five-token DSpark with adaptive verification off."
1231612337
- "TP4 with the DeepSeek-V4.1 latency-specialized decode executor (additional-config decode_backend latency_specialized, FlashInfer sparse MLA, THP Engram host tables) behind vLLM's own frontend at concurrency 1 (max-num-seqs 1, prefill-schedule-interval 1), 4 (max-num-seqs 8, max-num-active-seqs 1, prefill-schedule-interval 16), 10 (max-num-seqs 16, max-num-active-seqs 2, prefill-schedule-interval 96) and 16, 24, 32 and 48 (max-num-seqs 64, max-num-active-seqs 3, 4, 6 and 8, prefill-schedule-interval 96)."
1231712338
- "DEP4 (TP1 x DP4 + EP4, DeepGEMM MegaMoE, FlashMLA mega-attention with sparse-MLA prefill workspace factor 4, deep_select top-k, Engram host tables shared across DP ranks, prefill-schedule-interval 96, NUMA binding) behind the consistent-hash vLLM Router at concurrency 136 (24 GiB KV pool), 170 (28 GiB), 200 (32 GiB), 256 (40 GiB), 320 (48 GiB), 384 (56 GiB), 512 (72 GiB, max-num-seqs 192), 640 (96 GiB, max-num-seqs 224), 768 (112 GiB, max-num-seqs 256) and 1024 (128 GiB, max-num-seqs 320), max-num-seqs 128 otherwise, each with a 65536-token prefill budget and long-prefill-token-threshold 8192. Removes TP4 at concurrency 8 and DEP2 at concurrency 8-192."
1231812339
pr-link: https://github.com/SemiAnalysisAI/InferenceX/pull/4844

0 commit comments

Comments
 (0)