Skip to content

[VideoGenX] add SGLang Wan2.2 on H200 with 1, 2, 4 and 8 GPUs / [VideoGenX] 在 H200 上新增 SGLang Wan2.2(1、2、4、8 卡) - #5113

Open
edwingao28 wants to merge 6 commits into
mainfrom
feat/wan22-h200-sglang-video
Open

edwingao28 wants to merge 6 commits into
mainfrom
feat/wan22-h200-sglang-video

Conversation

@edwingao28

@edwingao28 edwingao28 commented Nov 17, 2026 •

Copy link
Copy Markdown
Collaborator

Summary

Wan2.2-T2V-A14B has dashboard curves on vLLM-Omni only. _wan22_payload in video-generation/evaluator/runner.py raises Wan2.2 plans have a vllm-omni request contract only for any other runtime, so no SGLang Wan2.2 run can start and the vLLM-Omni rows have no SGLang baseline beside them. This PR adds the SGLang request contract and an H200 row that uses it.

Changes:

  • _wan22_payload now builds an SGLang request. The vLLM-Omni request is unchanged.
    • It leaves out flow_shift and boundary_ratio, because SGLang v0.5.23 accepts both as request fields and its Wan pipeline reads neither.
    • It adds x264_preset: "ultrafast" and output_compression: 63 (CRF 18), so SGLang delivers the libx264 encoding that vLLM-Omni v0.32.0 delivers by default instead of its own fast at CRF 25.
  • serve in evaluator/srt_selected.py binds the two omitted controls at launch for Wan2.2 on SGLang. It refuses to start unless exactly one flag sets --flow-shift to the plan's value and the snapshot's model_index.json boundary_ratio equals the plan. --flow_shift and the --flow-shift= form count, because SGLang's parser accepts both. Without this check a mismatch would run different work with no error.
  • run_plan refuses an SGLang Wan2.2 run unless its sealed launch identity sets the plan's flow shift. The requests omit the shift, so the recorded launch is the only record of which shift was served.
wan22.yaml control How SGLang gets it SGLang source at 297e54fd
width, height, frame_count, fps, num_inference_steps, seed Request fields width, height, num_frames, fps, num_inference_steps, seed video_api.py#L254-L275
guidance_scale, guidance_scale_2 Request fields. The high-noise expert uses the first and the low-noise expert the second. denoising.py#L1935-L1944
negative_prompt (empty) Request field. Only None falls back to the model default, so the empty string is encoded as the CFG negative. utils.py#L296-L297
flow_shift Server flag --flow-shift. The Wan pipeline builds its UniPC scheduler from the pipeline config, and the request field only reaches a log line. wan_pipeline.py#L42-L46, schedule_batch.py#L433-L437
boundary_ratio The checkpoint's model_index.json. The Wan2.2 sampling class declares no extra request fields, so a request value is dropped. composed_pipeline_base.py#L459-L503, denoising.py#L1187-L1213
Delivered encoding (not a wan22.yaml control) Request fields x264_preset and output_compression. SGLang encodes at crf = int((1 - output_compression / 100) * 51), so 63 gives CRF 18. protocol.py#L165-L166, utils.py#L452-L455
No reused text embeddings Server flag --disable-conditioning-cache pipeline_executor.py#L65-L69
  • Recipes under wan22/sglang/h200-bf16/ follow the H3 SGLang H200 recipe: the same image digest (SGLang v0.5.23) and VIDEO_RUNTIME_REVISION, TP 1, --performance-mode speed, no torch.compile, no DiT offload. They drop --model-variant, warm up at 832x480 and add --flow-shift 12.0. The parallel layout is the one the vLLM-Omni Wan2.2 rows use:
GPUs Recipe CFG parallel Ulysses
1 serving-client-g1.yaml 1 1
2 serving-client-g2.yaml 2 1
4 serving-client-g4.yaml 2 2
8 serving-client.yaml 2 4
  • Row wan22-bf16-h200-sglang-video on cluster:h200-dgxc runs wan22.yaml release-v1 at C1 on 1, 2, 4 and 8 GPUs.
  • Changelog. One perf-changelog.yaml block lists only the new key.
  • Docs. specs/README.md and docs/video-generation.md state which controls SGLang takes from the server and which encoding its requests ask for. MODELS.md and docs/video-generation.md add the H200 SGLang row.
  • H3 requests and every vLLM-Omni request are unchanged.

Results:

Run Sweep 44067008117, attempt 1, head 9363f771, label non-canary-full-sweep-enabled. cluster:h200-dgxc, SGLang v0.5.23 (297e54fd), wan22.yaml release-v1 at C1 with 1 warmup and 20 measured clips per point, plan d1f6d1d429e1. Each row reads gpu/c1/aggregate.json and gpu/c1/baseline/run.json from that point's h3-results-44067008117-1-<recipe fingerprint> artifact.

GPUs CFG parallel x Ulysses p50 s per clip Clips per hour kJ per clip Valid clips Power valid bicycle-pass-s11 motion (passes above 0.00098) x264 crf / subme / bframes Job
1 1 x 1 204.6 17.60 141.2 20 / 20 yes 0.00266 18 / 0 / 0 passed
2 2 x 1 102.0 35.30 140.5 20 / 20 yes 0.00262 18 / 0 / 0 passed
4 2 x 2 53.3 67.53 147.1 20 / 20 yes 0.00284 18 / 0 / 0 passed
8 2 x 4 28.3 127.17 155.1 20 / 20 yes 0.00273 18 / 0 / 0 passed
  • The run qualifies. validate-video-results passes, with 20 / 20 valid clips and valid power at every point.
  • The delivered encoding matches vLLM-Omni. All 21 clips of every point record x264 crf 18, subme 0, bframes 0, me dia, mbtree 0 and ref 1. The vLLM-Omni H200 run 44034985283 records the same six values.
  • bicycle-pass-s11 is the lowest-motion clip for both engines. It scores lowest of the 20 clips at all four points here. It also scores lowest at all four vLLM-Omni H200 points of run 44034985283 (0.0035 to 0.0045) and at the vLLM-Omni H100 1-GPU point of run 44035678622 (0.0047).
  • SGLang's bike stops early. At 1 GPU the bike rolls in during the first 0.6 s and then stays almost still. vLLM-Omni starts this seed from the same scene on H200, and its bike keeps creeping.
  • At SGLang's default encoding this clip decided validity. Run 44052229897 on head 7bc36aed delivered fast at CRF 25. The clip scored 0.00080, 0.00077, 0.00120 and 0.00101 at 1, 2, 4 and 8 GPUs. The 1 and 2 GPU points had 19 / 20 valid clips, so validate-video-results failed.
  • The other 19 measured clips score at least 0.0055 at every point.
  • Clip time is steady within a run. The latency standard deviation is at most 0.17 s at every point.
  • Clip time varies more between runs. The 1-GPU p50 was 199.8 s, 201.6 s and 204.6 s in runs 44037016164, 44052229897 and 44067008117, each on a different GPU.

Testing:

  • test_wan22_sglang_request_omits_only_the_controls_its_server_binds checks the exact SGLang request, including the two encoding fields. It fails on main with the vllm-omni request contract only error.
  • test_sglang_wan22_launch_must_serve_the_controls_its_requests_omit covers 7 launches. --flow-shift 12.0 and --flow_shift=12.0 both start. A wrong shift, a missing shift, a shift set twice and a wrong checkpoint boundary each refuse to start. A vLLM-Omni launch is not checked. The 4 refusal cases fail on main, where the server starts.
  • test_wan22_sglang_run_requires_a_launch_that_sets_the_plan_flow_shift covers a run with no sealed launch and a run whose launch sets --flow-shift 5.0. Both are refused before any output is written. Both cases fail on main.
  • The video-generation suite passes, 141 tests.
  • infx generate --config-key wan22-bf16-h200-sglang-video renders 4 recipes.
  • python -m infx.matrix.changed --base origin/main lists only wan22-bf16-h200-sglang-video, with 4 added points and 0 removed.

Limits:

  • The contract table is read from SGLang source. The run confirms three parts of it at all four points.
    • Each recorded request carries exactly the 14 per-request fields.
    • Each sealed launch carries --flow-shift 12.0 and --disable-conditioning-cache.
    • Each server log prints Setting boundary ratio to 0.875.
  • The use of guidance_scale_2 and the expert switch are shown by source only.
  • Why SGLang's bike stops while vLLM-Omni's keeps creeping is unexplained.
  • The motion check passes bicycle-pass-s11 on its still part. In the 1-GPU clip the bike stops at 0.56 s. The 71 frame intervals after that average 2.6 times the threshold at ultrafast CRF 18. In run 44052229897 the same 71 intervals average 0.55 times the threshold at fast CRF 25. 1-GPU output is deterministic, so both files encode the same generated frames. The vLLM-Omni H200 clips show the same floor: after 0.56 s their lowest single interval is 2.2 to 3.8 times the threshold.
  • At 4 GPUs SGLang would pick its K/V-gather exchange for the sequence split when no degree is set. The recipe sets --ulysses-degree 2 so the layout equals the vLLM-Omni row. The 4-GPU server log shows ulysses_degree 2 and kv_gather_degree 1.
  • vgbench run --execute against an operator-managed SGLang endpoint needs --numerical-identity with a launch that sets the plan's --flow-shift. The operator declares that identity, and the client cannot check the checkpoint's boundary_ratio there.
  • The launch check matches --flow-shift and --flow_shift by full name. It does not count an argparse prefix abbreviation or a value set through --config.

Related: Follows #5082 (B300) and #5084 (B200 GPU-count sweep). #5110 adds the vLLM-Omni row on the same H200 cluster.

PR checklist and change type

Type of Change

  • Bug fix
  • New feature
  • Configuration change
  • Documentation update
  • Other (please describe)

Checklist

  • I have completed the AI model disclosure and kept it current
  • I have tested my changes locally
  • I have updated documentation if necessary
  • If python -m infx.matrix.changed --base origin/main lists affected config keys, I have appended one new entry covering them to the physical end of inferencex-e2e/perf-changelog.yaml; if it lists none, I have added none. I have not edited historical entries
  • If this PR appends an inferencex-e2e/perf-changelog.yaml entry, it carries exactly one primary sweep label (a maintainer applies it on fork PRs): full-sweep-fail-fast (recommended), full-sweep-enabled, or non-canary-full-sweep-enabled. Optional modifiers all-evals, evals-only, and agentx-fast require a primary label; the last two block reuse while applied.
  • Before merging via reuse, a maintainer with write access to the repository has commented /use <run_id> (or the legacy /reuse-sweep-run) on this PR. Do this only once there is a final full sweep that is all green with evals passing, since after this comment the primary sweep label will no longer automatically kick off new sweeps. Remove and re-add the primary sweep label to force a new sweep.

@github-actions

Copy link
Copy Markdown
Contributor

Thanks for the contribution!

  • Review: If this PR changes files owned by someone other than a repository admin or @SemiAnalysisAI/core, ask one eligible CODEOWNER to complete the latest PR_REVIEW_CHECKLIST.md before contacting a core maintainer on Slack. Follow the template exactly, including As a PR reviewer and CODEOWNER, I have reviewed this and have, so sign-off verification triggers.
  • PR verification: Sweeps run only when the PR appends an inferencex-e2e/perf-changelog.yaml entry and carries exactly one primary label: full-sweep-fail-fast (strongly recommended; canary plus per-matrix fail-fast), full-sweep-enabled (canary; matrix jobs continue after a failure), or non-canary-full-sweep-enabled (no canary or fail-fast). The modifiers all-evals, evals-only, and agentx-fast require a primary label. On fork PRs, a maintainer applies the label. See sweep labels and reuse.
  • After merging: PR authors must ensure all GitHub Actions jobs pass. Transient failures often pass on rerun; see how to rerun failed jobs.
中文

感谢你的贡献!

  • **审阅:**如果 PR 修改的文件归属于仓库管理员及 @SemiAnalysisAI/core 之外的 CODEOWNER,请先联系一位有资格的 CODEOWNER 填写最新的 PR_REVIEW_CHECKLIST.md,再通过 Slack 联系核心维护者。必须严格遵循模板,并保留 As a PR reviewer and CODEOWNER, I have reviewed this and have,才能触发签核验证。
  • **PR 验证:**只有当 PR 在 inferencex-e2e/perf-changelog.yaml 末尾追加了条目,并且恰好带有一个主标签时,才会运行扫描:full-sweep-fail-fast(强烈推荐;canary 加逐矩阵 fail-fast)、full-sweep-enabled(有 canary;矩阵任务在失败后继续运行)或 non-canary-full-sweep-enabled(无 canary,也无 fail-fast)。修饰标签 all-evals、evals-only 和 agentx-fast 必须与主标签一起使用。fork PR 的标签由维护者添加。参见扫描标签与复用。
  • **合并后:**PR 作者必须确保所有 GitHub Actions 任务通过。临时性失败通常可以通过重新运行恢复;参见重新运行失败任务的说明。

@edwingao28 edwingao28 added the full-sweep-fail-fast Full sweep with canary gate; first failure cancels the rest of that matrix (recommended) label Nov 17, 2026

@claude claude Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nothing blocking. The comments below are optional suggestions. There is no need to push a fix for them before merging.

Comment thread video-generation/evaluator/runner.py Resolved
@github-actions

github-actions Bot commented Nov 17, 2026 •

Copy link
Copy Markdown
Contributor

@edwingao28 edwingao28 added non-canary-full-sweep-enabled Run the full sweep without the canary gate (full search space, no trim) and removed full-sweep-fail-fast Full sweep with canary gate; first failure cancels the rest of that matrix (recommended) labels Nov 17, 2026

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

non-canary-full-sweep-enabled Run the full sweep without the canary gate (full search space, no trim)

Projects

Status: No status

Development

Successfully merging this pull request may close these issues.

1 participant