Repository navigation
[VideoGenX] add SGLang Wan2.2 on H100 with 4 and 8 GPUs / [VideoGenX] 在 H100 上新增 SGLang Wan2.2(4、8 卡) - #5114
Open
edwingao28 wants to merge 10 commits into
Open
edwingao28 wants to merge 10 commits into
edwingao28 wants to merge 10 commits into
Conversation
edwingao28
requested review from
Ankur-singh,
RohitNagraj,
chunfangamd,
functionstackx,
kedarpotdar-nv,
nvpohanh and
wzhao18
as code owners
November 17, 2026 07:16
Contributor
|
Thanks for the contribution!
中文感谢你的贡献!
|
edwingao28
force-pushed
the
feat/wan22-h100-sglang-video
branch
from
November 17, 2026 07:17
f28acb3 to
88f5855
Compare
Contributor
|
View unofficial run (performance): https://inferencex.semianalysis.com/inference?unofficialRun=44198185716 View unofficial run (accuracy): https://inferencex.semianalysis.com/evaluation?unofficialRun=44198185716 |
The default 17-frame warmup skips sharded VAE decode on 8 ranks. NCCL then allocates the decode group's buffers mid-request on nearly full 80 GB cards and fails.
This branch has not been deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Wan2.2-T2V-A14B has no H100 curve. The dashboard shows it on B200 and B300 with vLLM-Omni only, so an H100 result has no baseline on either engine. This PR adds the H100 SGLang row on 4 and 8 GPUs. #5113 adds the same row on H200.
Changes:
video-generation/is byte-identical to [VideoGenX] add SGLang Wan2.2 on H200 with 1, 2, 4 and 8 GPUs / [VideoGenX] 在 H200 上新增 SGLang Wan2.2(1、2、4、8 卡) #5113 at918dd98d9. Commitsfca8236f1,bc21e9a2eand0ef28e01ccarry it here. [VideoGenX] add SGLang Wan2.2 on H200 with 1, 2, 4 and 8 GPUs / [VideoGenX] 在 H200 上新增 SGLang Wan2.2(1、2、4、8 卡) #5113 describes and tests it.wan22/sglang/h100-bf16/are the [VideoGenX] add SGLang Wan2.2 on H200 with 1, 2, 4 and 8 GPUs / [VideoGenX] 在 H200 上新增 SGLang Wan2.2(1、2、4、8 卡) #5113 H200 recipes withgpu_type: h100. The 8-GPU recipe adds one warmup flag. They use the image digest (SGLang v0.5.23) andVIDEO_RUNTIME_REVISIONof the H3 H100 SGLang row, TP 1,--performance-mode speed, no torch.compile, no DiT offload,--flow-shift 12.0and no conditioning cache.serving-client-g4.yamlserving-client.yaml--warmup-num-frames 21)--warmup-num-frames 21on 8 GPUs. Without it the first 81-frame decode ran out of GPU memory, and the 8-GPU point returned 0 / 20 clips.base.py#L118).warmup_request_builder.py#L56). At 832x480 that is 5 x 60 x 104 latent elements, or 3900 per rank on the 8 decode ranks of CFG 2 x Ulysses 4.ncclCuMemAllocfailed.wan22-bf16-h100-sglang-videooncluster:h100-dsxerunswan22.yamlrelease-v1at C1 on 4 and 8 GPUs.perf-changelog.yamlblock lists only the new key.MODELS.mdanddocs/video-generation.mdadd the H100 SGLang row.Results:
PR sweeps on
cluster:h100-dsxe:wan22.yamlrelease-v1at C1 with 1 warmup and 20 measured clips per point, SGLang v0.5.23 (297e54fd), every component resident on the GPU. Latency, throughput and energy are the values the VideoGenX preview API returns for each run.242d44fef242d44fef0ef28e01c0ef28e01c654359e9388f5855d3validate-video-resultspasses, with 20 / 20 valid clips and valid power at both points.Warmup requests: 1/1 server warmup req (832x480x21f, 2/40 steps). It has 21 decodes of 0.95 to 0.99 s and noout of memoryline.(ncclCuMemAlloc) NCCL WARN Cuda failure 2 'out of memory'. SGLang kept the request in progress, so the client waited out its 900 s limit.03e996b141a5. Its p50 differs by 0.87 s (1.6 %) between the two runs.torch.OutOfMemoryError: CUDA out of memory. Tried to allocate 160.00 MiB. GPU 0 has a total capacity of 79.18 GiB of which 58.12 MiB is free.transformer_2took 26.62 GB andtransformertook 26.62 GB. Thetext_encoderload then started with 19.47 GB free. SGLang loads the UMT5 encoder in FP32, which is 21.2 GiB for its 5.68 billion parameters.CUDA out of memory. Tried to allocate 320.00 MiB. GPU 0 has a total capacity of 79.18 GiB of which 162.00 MiB is free.SGLang split the text encoder across the two GPUs, 10.58 GB each. After loading, each GPU had 6.80 GB free. At the failure PyTorch had 68.18 GiB allocated and 5.07 GiB reserved but unallocated.Direct reproduction of the 8-GPU failure: Slurm job 1867 on one
cluster:h100-dsxenode with 8 H100 80 GB GPUs. It ran outside Actions, so its rows have no run link. It used the image digest, SGLang revision, model snapshot and server flags of the 8-GPU recipe, withNCCL_DEBUG=INFO. Each row is one server start followed by 81-frame, 40-step requests.nvidia-smisampled GPU memory every 250 ms.ncclCuMemAllocfailed on ranks 1 to 6. The decode had logged 496 channel connects and 1099 P2P buffer allocations by then.Testing:
infx generate --config-key wan22-bf16-h100-sglang-videorenders 2 recipes. Only the 8-GPU recipe carries--warmup-num-frames 21.python -m infx.matrix.changedagainst merge base7e2b3622flists onlywan22-bf16-h100-sglang-video, with 2 added points and 0 removed. Their fingerprints,03e996b141a5and3061acbb60c7, are the two points of run 44198185716.video-generationsuite passes, 141 tests.Limits:
297e54fd. A different SGLang version needs the frame count rechecked.Related: Shares
video-generation/with #5113 (H200 SGLang). Whichever of the two merges second needs a main sync, because both edit the same places innvidia-master.yaml,MODELS.md,docs/video-generation.mdandperf-changelog.yaml. #5111 adds the vLLM-Omni row on the same H100 cluster. Follows #5082 (B300) and #5084 (B200 GPU-count sweep).PR checklist and change type
Type of Change
Checklist
python -m infx.matrix.changed --base origin/mainlists affected config keys, I have appended one new entry covering them to the physical end ofinferencex-e2e/perf-changelog.yaml; if it lists none, I have added none. I have not edited historical entriesinferencex-e2e/perf-changelog.yamlentry, it carries exactly one primary sweep label (a maintainer applies it on fork PRs):full-sweep-fail-fast(recommended),full-sweep-enabled, ornon-canary-full-sweep-enabled. Optional modifiersall-evals,evals-only, andagentx-fastrequire a primary label; the last two block reuse while applied./use <run_id>(or the legacy/reuse-sweep-run) on this PR. Do this only once there is a final full sweep that is all green with evals passing, since after this comment the primary sweep label will no longer automatically kick off new sweeps. Remove and re-add the primary sweep label to force a new sweep.