Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Original file line number Diff line number Diff line change
Expand Up @@ -32,7 +32,7 @@ services:
- --num-gpus
- "{gpus_per_node}"
- --ulysses-degree
- "4"
- "{gpus_per_node}"
- --tp-size
- "1"
- --encoder-parallel
Expand Down
9 changes: 9 additions & 0 deletions inferencex-e2e/configs/nvidia-master.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -7497,9 +7497,18 @@ h3-bf16-b200-sglang-video:
- spec: h3-v2.yaml
set: release-v1
search-space:
- gpus: 1
srt-recipe: serving-client.yaml
conc-list: [1]
- gpus: 2
srt-recipe: serving-client.yaml
conc-list: [1]
- gpus: 4
srt-recipe: serving-client.yaml
conc-list: [1]
- gpus: 8
srt-recipe: serving-client.yaml
conc-list: [1]

glm5.3-fp4-b300-sglang-agentic-mtp:
image: lmsysorg/sglang:v0.5.20-cu130@sha256:b27fce60bc5494c118c4910702812bcfa8cee67abcdd1ff8b0902f21647552f4
Expand Down
2 changes: 1 addition & 1 deletion inferencex-e2e/docs/MODELS.md
Original file line number Diff line number Diff line change
Expand Up @@ -36,7 +36,7 @@ For non-grandfathered models, `ROCM_QUICK_REDUCE_QUANTIZATION`, `VLLM_ROCM_QUICK
|---|---|---|
| Agentic coding | Long Context, Multi Turn Realistic traffic trace replay with sub agents | Active. This is the trace-replay agentic-coding benchmark (see the srt-slurm recipes under [`benchmarks/single_node/srt-slurm-recipes/`](../benchmarks/single_node/srt-slurm-recipes) and the shared client [`benchmarks/srt_agentic.sh`](../benchmarks/srt_agentic.sh)). Going forward, new models will likely be onboarded with agentic coding only. Speculative decoding may be enabled or disabled to produce the best Pareto points; a separate non-spec-decode A/B baseline is not required (see [Deprecation policy](#deprecation-policy)). |
| Single-turn 8k1k | 8192 / 1024 | Active. This is the primary fixed-sequence-length scenario. |
| Video generation | H3: 124 frames at 24 fps, 1344x768, C1 | MiniMax-H3 uses native srt-slurm: H100/B200/MI355X SGLang select four-GPU, 20-clip `h3-v2.yaml` C1 points. The client and server live in [`video-generation`](../../video-generation); energy is reported per valid clip. Config presence does not qualify execution, power, or quality. See [video delivery](video-generation.md). |
| Video generation | H3: 124 frames at 24 fps, 1344x768, C1 | MiniMax-H3 uses native srt-slurm: SGLang selects 20-clip `h3-v2.yaml` C1 points on 1, 2, 4 and 8 B200 GPUs and 4 H100 or MI355X GPUs. The client and server live in [`video-generation`](../../video-generation); energy is reported per valid clip. Config presence does not qualify execution, power, or quality. See [video delivery](video-generation.md). |
| Single-turn 1k1k | 1024 / 1024 | Deprecated since 2026-07-17 ([#2263](https://github.com/SemiAnalysisAI/InferenceX/pull/2263)), to save GPU cluster time for higher-priority real-world agentic-coding benchmarks and new frontier models. Archived configs lived in `configs/deprecated/` until it was deleted (#3463); see git history. The GLM-5.1 B200 TileRT point added later in [#2533](https://github.com/SemiAnalysisAI/InferenceX/pull/2533) was the last 1k1k config; it was retired on 2026-10-01 ([#3638](https://github.com/SemiAnalysisAI/InferenceX/pull/3638)). |
| Single-turn 1k8k | 1024 / 8192 | **Deprecated for all models** since 2026-03-27 ([#911](https://github.com/SemiAnalysisAI/InferenceX/pull/911)), to save GPU cluster time for higher-priority real-world agentic-coding benchmarks and new frontier models. Configs were removed, not archived. |

Expand Down
5 changes: 3 additions & 2 deletions inferencex-e2e/docs/video-generation.md
Original file line number Diff line number Diff line change
@@ -1,7 +1,8 @@
# Native video delivery

The `video-generation` scenario schedules one server job per deployment. H100,
B200 and MI355X SGLang select four-GPU C1 deployments using
The `video-generation` scenario schedules one server job per deployment. SGLang
selects C1 deployments on 1, 2, 4 and 8 B200 GPUs (Ulysses across all GPUs) and
on 4 H100 or MI355X GPUs, all using
`h3-v2.yaml`: 20 distinct measured prompts, a separate warmup, 49 denoising
updates, and 124 frames at 24 fps and 1344×768. A five-second request produces
124/24 seconds of video. Config presence is not GPU or quality qualification.
Expand Down
7 changes: 7 additions & 0 deletions inferencex-e2e/perf-changelog.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -12262,3 +12262,10 @@
- "Kimi-K3.1-Pro B300 vLLM: enable the recipe's low-latency decode (decode_backend=latency_specialized) at c1-c4, whose decode graphs stay within 64 rows (the image's wider MoE needs 152 resident CTAs, more than a B300's 148 SMs), and run c1-c2 at plain TP8 instead of TP8 x DCP8."
- "Match the SGLang recipe's draft depths (DSpark 7 tokens through c8, 6 at c16, 5 at c24-c40, 4 from c48), size each decode graph cap to max-num-seqs x (1 + drafts), and raise the engine-ready timeout and health check to 7200 s."
pr-link: https://github.com/SemiAnalysisAI/InferenceX/pull/4833

- config-keys:
- h3-bf16-b200-sglang-video
scenario-type: [video-generation]
description:
- "Sweep H3 C1 latency on 1, 2, 4 and 8 B200 GPUs with SGLang Ulysses across all GPUs; the B200 recipe now sets the Ulysses degree from the GPU count, which keeps the 4-GPU point unchanged."
pr-link: https://github.com/SemiAnalysisAI/InferenceX/pull/4852
Loading