diff --git a/inferencex-e2e/benchmarks/single_node/srt-slurm-recipes/h3/sglang/b200-bf16/serving-client.yaml b/inferencex-e2e/benchmarks/single_node/srt-slurm-recipes/h3/sglang/b200-bf16/serving-client.yaml index d3cbc3f117..83c575a2bb 100644 --- a/inferencex-e2e/benchmarks/single_node/srt-slurm-recipes/h3/sglang/b200-bf16/serving-client.yaml +++ b/inferencex-e2e/benchmarks/single_node/srt-slurm-recipes/h3/sglang/b200-bf16/serving-client.yaml @@ -32,7 +32,7 @@ services: - --num-gpus - "{gpus_per_node}" - --ulysses-degree - - "4" + - "{gpus_per_node}" - --tp-size - "1" - --encoder-parallel diff --git a/inferencex-e2e/configs/nvidia-master.yaml b/inferencex-e2e/configs/nvidia-master.yaml index 521117a5f3..40db965f44 100644 --- a/inferencex-e2e/configs/nvidia-master.yaml +++ b/inferencex-e2e/configs/nvidia-master.yaml @@ -7497,9 +7497,18 @@ h3-bf16-b200-sglang-video: - spec: h3-v2.yaml set: release-v1 search-space: + - gpus: 1 + srt-recipe: serving-client.yaml + conc-list: [1] + - gpus: 2 + srt-recipe: serving-client.yaml + conc-list: [1] - gpus: 4 srt-recipe: serving-client.yaml conc-list: [1] + - gpus: 8 + srt-recipe: serving-client.yaml + conc-list: [1] glm5.3-fp4-b300-sglang-agentic-mtp: image: lmsysorg/sglang:v0.5.20-cu130@sha256:b27fce60bc5494c118c4910702812bcfa8cee67abcdd1ff8b0902f21647552f4 diff --git a/inferencex-e2e/docs/MODELS.md b/inferencex-e2e/docs/MODELS.md index f67e268347..404327641b 100644 --- a/inferencex-e2e/docs/MODELS.md +++ b/inferencex-e2e/docs/MODELS.md @@ -36,7 +36,7 @@ For non-grandfathered models, `ROCM_QUICK_REDUCE_QUANTIZATION`, `VLLM_ROCM_QUICK |---|---|---| | Agentic coding | Long Context, Multi Turn Realistic traffic trace replay with sub agents | Active. This is the trace-replay agentic-coding benchmark (see the srt-slurm recipes under [`benchmarks/single_node/srt-slurm-recipes/`](../benchmarks/single_node/srt-slurm-recipes) and the shared client [`benchmarks/srt_agentic.sh`](../benchmarks/srt_agentic.sh)). Going forward, new models will likely be onboarded with agentic coding only. Speculative decoding may be enabled or disabled to produce the best Pareto points; a separate non-spec-decode A/B baseline is not required (see [Deprecation policy](#deprecation-policy)). | | Single-turn 8k1k | 8192 / 1024 | Active. This is the primary fixed-sequence-length scenario. | -| Video generation | H3: 124 frames at 24 fps, 1344x768, C1 | MiniMax-H3 uses native srt-slurm: H100/B200/MI355X SGLang select four-GPU, 20-clip `h3-v2.yaml` C1 points. The client and server live in [`video-generation`](../../video-generation); energy is reported per valid clip. Config presence does not qualify execution, power, or quality. See [video delivery](video-generation.md). | +| Video generation | H3: 124 frames at 24 fps, 1344x768, C1 | MiniMax-H3 uses native srt-slurm: SGLang selects 20-clip `h3-v2.yaml` C1 points on 1, 2, 4 and 8 B200 GPUs and 4 H100 or MI355X GPUs. The client and server live in [`video-generation`](../../video-generation); energy is reported per valid clip. Config presence does not qualify execution, power, or quality. See [video delivery](video-generation.md). | | Single-turn 1k1k | 1024 / 1024 | Deprecated since 2026-07-17 ([#2263](https://github.com/SemiAnalysisAI/InferenceX/pull/2263)), to save GPU cluster time for higher-priority real-world agentic-coding benchmarks and new frontier models. Archived configs lived in `configs/deprecated/` until it was deleted (#3463); see git history. The GLM-5.1 B200 TileRT point added later in [#2533](https://github.com/SemiAnalysisAI/InferenceX/pull/2533) was the last 1k1k config; it was retired on 2026-10-01 ([#3638](https://github.com/SemiAnalysisAI/InferenceX/pull/3638)). | | Single-turn 1k8k | 1024 / 8192 | **Deprecated for all models** since 2026-03-27 ([#911](https://github.com/SemiAnalysisAI/InferenceX/pull/911)), to save GPU cluster time for higher-priority real-world agentic-coding benchmarks and new frontier models. Configs were removed, not archived. | diff --git a/inferencex-e2e/docs/video-generation.md b/inferencex-e2e/docs/video-generation.md index 2263251293..090962dcb5 100644 --- a/inferencex-e2e/docs/video-generation.md +++ b/inferencex-e2e/docs/video-generation.md @@ -1,7 +1,8 @@ # Native video delivery -The `video-generation` scenario schedules one server job per deployment. H100, -B200 and MI355X SGLang select four-GPU C1 deployments using +The `video-generation` scenario schedules one server job per deployment. SGLang +selects C1 deployments on 1, 2, 4 and 8 B200 GPUs (Ulysses across all GPUs) and +on 4 H100 or MI355X GPUs, all using `h3-v2.yaml`: 20 distinct measured prompts, a separate warmup, 49 denoising updates, and 124 frames at 24 fps and 1344×768. A five-second request produces 124/24 seconds of video. Config presence is not GPU or quality qualification. diff --git a/inferencex-e2e/perf-changelog.yaml b/inferencex-e2e/perf-changelog.yaml index 1db54f77c1..9e141416db 100644 --- a/inferencex-e2e/perf-changelog.yaml +++ b/inferencex-e2e/perf-changelog.yaml @@ -12262,3 +12262,10 @@ - "Kimi-K3.1-Pro B300 vLLM: enable the recipe's low-latency decode (decode_backend=latency_specialized) at c1-c4, whose decode graphs stay within 64 rows (the image's wider MoE needs 152 resident CTAs, more than a B300's 148 SMs), and run c1-c2 at plain TP8 instead of TP8 x DCP8." - "Match the SGLang recipe's draft depths (DSpark 7 tokens through c8, 6 at c16, 5 at c24-c40, 4 from c48), size each decode graph cap to max-num-seqs x (1 + drafts), and raise the engine-ready timeout and health check to 7200 s." pr-link: https://github.com/SemiAnalysisAI/InferenceX/pull/4833 + +- config-keys: + - h3-bf16-b200-sglang-video + scenario-type: [video-generation] + description: + - "Sweep H3 C1 latency on 1, 2, 4 and 8 B200 GPUs with SGLang Ulysses across all GPUs; the B200 recipe now sets the Ulysses degree from the GPU count, which keeps the 4-GPU point unchanged." + pr-link: https://github.com/SemiAnalysisAI/InferenceX/pull/4852