Repository navigation
[VideoGenX] add SGLang Wan2.2 on H200 with 1, 2, 4 and 8 GPUs / [VideoGenX] 在 H200 上新增 SGLang Wan2.2(1、2、4、8 卡) - #5113
Open
edwingao28 wants to merge 6 commits into
Open
edwingao28 wants to merge 6 commits into
edwingao28 wants to merge 6 commits into
Conversation
edwingao28
requested review from
Ankur-singh,
RohitNagraj,
chunfangamd,
kedarpotdar-nv,
nvpohanh and
wzhao18
as code owners
November 17, 2026 07:13
Contributor
|
Thanks for the contribution!
中文感谢你的贡献!
|
6 of 11 tasks
Contributor
|
View unofficial run (performance): https://inferencex.semianalysis.com/inference?unofficialRun=44067008117 View unofficial run (accuracy): https://inferencex.semianalysis.com/evaluation?unofficialRun=44067008117 |
This branch has not been deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Wan2.2-T2V-A14B has dashboard curves on vLLM-Omni only.
_wan22_payloadinvideo-generation/evaluator/runner.pyraisesWan2.2 plans have a vllm-omni request contract onlyfor any other runtime, so no SGLang Wan2.2 run can start and the vLLM-Omni rows have no SGLang baseline beside them. This PR adds the SGLang request contract and an H200 row that uses it.Changes:
_wan22_payloadnow builds an SGLang request. The vLLM-Omni request is unchanged.flow_shiftandboundary_ratio, because SGLang v0.5.23 accepts both as request fields and its Wan pipeline reads neither.x264_preset: "ultrafast"andoutput_compression: 63(CRF 18), so SGLang delivers the libx264 encoding that vLLM-Omni v0.32.0 delivers by default instead of its ownfastat CRF 25.serveinevaluator/srt_selected.pybinds the two omitted controls at launch for Wan2.2 on SGLang. It refuses to start unless exactly one flag sets--flow-shiftto the plan's value and the snapshot'smodel_index.jsonboundary_ratioequals the plan.--flow_shiftand the--flow-shift=form count, because SGLang's parser accepts both. Without this check a mismatch would run different work with no error.run_planrefuses an SGLang Wan2.2 run unless its sealed launch identity sets the plan's flow shift. The requests omit the shift, so the recorded launch is the only record of which shift was served.wan22.yamlcontrol297e54fdwidth,height,frame_count,fps,num_inference_steps, seedwidth,height,num_frames,fps,num_inference_steps,seedvideo_api.py#L254-L275guidance_scale,guidance_scale_2denoising.py#L1935-L1944negative_prompt(empty)Nonefalls back to the model default, so the empty string is encoded as the CFG negative.utils.py#L296-L297flow_shift--flow-shift. The Wan pipeline builds its UniPC scheduler from the pipeline config, and the request field only reaches a log line.wan_pipeline.py#L42-L46,schedule_batch.py#L433-L437boundary_ratiomodel_index.json. The Wan2.2 sampling class declares no extra request fields, so a request value is dropped.composed_pipeline_base.py#L459-L503,denoising.py#L1187-L1213wan22.yamlcontrol)x264_presetandoutput_compression. SGLang encodes atcrf = int((1 - output_compression / 100) * 51), so 63 gives CRF 18.protocol.py#L165-L166,utils.py#L452-L455--disable-conditioning-cachepipeline_executor.py#L65-L69wan22/sglang/h200-bf16/follow the H3 SGLang H200 recipe: the same image digest (SGLang v0.5.23) andVIDEO_RUNTIME_REVISION, TP 1,--performance-mode speed, no torch.compile, no DiT offload. They drop--model-variant, warm up at 832x480 and add--flow-shift 12.0. The parallel layout is the one the vLLM-Omni Wan2.2 rows use:serving-client-g1.yamlserving-client-g2.yamlserving-client-g4.yamlserving-client.yamlwan22-bf16-h200-sglang-videooncluster:h200-dgxcrunswan22.yamlrelease-v1at C1 on 1, 2, 4 and 8 GPUs.perf-changelog.yamlblock lists only the new key.specs/README.mdanddocs/video-generation.mdstate which controls SGLang takes from the server and which encoding its requests ask for.MODELS.mdanddocs/video-generation.mdadd the H200 SGLang row.Results:
Run Sweep 44067008117, attempt 1, head
9363f771, labelnon-canary-full-sweep-enabled.cluster:h200-dgxc, SGLang v0.5.23 (297e54fd),wan22.yamlrelease-v1at C1 with 1 warmup and 20 measured clips per point, pland1f6d1d429e1. Each row readsgpu/c1/aggregate.jsonandgpu/c1/baseline/run.jsonfrom that point'sh3-results-44067008117-1-<recipe fingerprint>artifact.bicycle-pass-s11motion (passes above 0.00098)crf/subme/bframesvalidate-video-resultspasses, with 20 / 20 valid clips and valid power at every point.crf18,subme0,bframes0,media,mbtree0 andref1. The vLLM-Omni H200 run 44034985283 records the same six values.bicycle-pass-s11is the lowest-motion clip for both engines. It scores lowest of the 20 clips at all four points here. It also scores lowest at all four vLLM-Omni H200 points of run 44034985283 (0.0035 to 0.0045) and at the vLLM-Omni H100 1-GPU point of run 44035678622 (0.0047).7bc36aeddeliveredfastat CRF 25. The clip scored 0.00080, 0.00077, 0.00120 and 0.00101 at 1, 2, 4 and 8 GPUs. The 1 and 2 GPU points had 19 / 20 valid clips, sovalidate-video-resultsfailed.Testing:
test_wan22_sglang_request_omits_only_the_controls_its_server_bindschecks the exact SGLang request, including the two encoding fields. It fails on main with thevllm-omni request contract onlyerror.test_sglang_wan22_launch_must_serve_the_controls_its_requests_omitcovers 7 launches.--flow-shift 12.0and--flow_shift=12.0both start. A wrong shift, a missing shift, a shift set twice and a wrong checkpoint boundary each refuse to start. A vLLM-Omni launch is not checked. The 4 refusal cases fail on main, where the server starts.test_wan22_sglang_run_requires_a_launch_that_sets_the_plan_flow_shiftcovers a run with no sealed launch and a run whose launch sets--flow-shift 5.0. Both are refused before any output is written. Both cases fail on main.video-generationsuite passes, 141 tests.infx generate --config-key wan22-bf16-h200-sglang-videorenders 4 recipes.python -m infx.matrix.changed --base origin/mainlists onlywan22-bf16-h200-sglang-video, with 4 added points and 0 removed.Limits:
--flow-shift 12.0and--disable-conditioning-cache.Setting boundary ratio to 0.875.guidance_scale_2and the expert switch are shown by source only.bicycle-pass-s11on its still part. In the 1-GPU clip the bike stops at 0.56 s. The 71 frame intervals after that average 2.6 times the threshold atultrafastCRF 18. In run 44052229897 the same 71 intervals average 0.55 times the threshold atfastCRF 25. 1-GPU output is deterministic, so both files encode the same generated frames. The vLLM-Omni H200 clips show the same floor: after 0.56 s their lowest single interval is 2.2 to 3.8 times the threshold.--ulysses-degree 2so the layout equals the vLLM-Omni row. The 4-GPU server log showsulysses_degree2 andkv_gather_degree1.vgbench run --executeagainst an operator-managed SGLang endpoint needs--numerical-identitywith a launch that sets the plan's--flow-shift. The operator declares that identity, and the client cannot check the checkpoint'sboundary_ratiothere.--flow-shiftand--flow_shiftby full name. It does not count an argparse prefix abbreviation or a value set through--config.Related: Follows #5082 (B300) and #5084 (B200 GPU-count sweep). #5110 adds the vLLM-Omni row on the same H200 cluster.
PR checklist and change type
Type of Change
Checklist
python -m infx.matrix.changed --base origin/mainlists affected config keys, I have appended one new entry covering them to the physical end ofinferencex-e2e/perf-changelog.yaml; if it lists none, I have added none. I have not edited historical entriesinferencex-e2e/perf-changelog.yamlentry, it carries exactly one primary sweep label (a maintainer applies it on fork PRs):full-sweep-fail-fast(recommended),full-sweep-enabled, ornon-canary-full-sweep-enabled. Optional modifiersall-evals,evals-only, andagentx-fastrequire a primary label; the last two block reuse while applied./use <run_id>(or the legacy/reuse-sweep-run) on this PR. Do this only once there is a final full sweep that is all green with evals passing, since after this comment the primary sweep label will no longer automatically kick off new sweeps. Remove and re-add the primary sweep label to force a new sweep.