Skip to content

[VideoGenX] add vLLM-Omni Wan2.2 on H100 with 1, 2, 4 and 8 GPUs / [VideoGenX] 在 H100 上新增 vLLM-Omni Wan2.2(1、2、4、8 卡) - #5111

Open
edwingao28 wants to merge 2 commits into
mainfrom
feat/wan22-h100-vllm-omni-video
Open

edwingao28 wants to merge 2 commits into
mainfrom
feat/wan22-h100-vllm-omni-video

Conversation

@edwingao28

@edwingao28 edwingao28 commented Nov 17, 2026 •

Copy link
Copy Markdown
Collaborator

Summary

Wan2.2-T2V-A14B has vLLM-Omni rows only on B200 and B300 (wan22-bf16-b200-vllm-omni-video, wan22-bf16-b300-vllm-omni-video). No Hopper GPU runs it, so VideoGenX has no H100 reference for this model.

Changes:

  • Recipes under wan22/vllm-omni/h100-bf16/. Each is the B200 recipe with gpu_type: h100. The image digest (vLLM-Omni v0.32.0), engine flags and env are the same, including OMNI_DIFFUSION_PROMPT_EMBED_CACHE=0.
GPUs Recipe CFG parallel Ulysses VAE patch parallel
1 serving-client-g1.yaml 1 1 off
2 serving-client-g2.yaml 2 1 2
4 serving-client-g4.yaml 2 2 4
8 serving-client.yaml 2 4 8
  • Row wan22-bf16-h100-vllm-omni-video on cluster:h100-dsxe: wan22.yaml release-v1, C1.
  • perf-changelog.yaml: one appended block for the new key.
  • Docs: MODELS.md, docs/video-generation.md and SUBMITTING.md add H100 to the hardware that runs Wan2.2 on vLLM-Omni.
  • No Python change. The recipes do not set VIDEO_PREFETCH_MODEL, offload or quantization.

Results: H100 80 GB on cluster:h100-dsxe, vLLM-Omni v0.32.0, wan22.yaml release-v1 at C1, PR sweep 44035678622 attempt 2 at 9e2868e8a. validate-video-results passed.

GPUs CFG x Ulysses p50 latency (s/clip) Throughput (clips/h) Energy (kJ/clip) Valid clips Power valid Job
1 1 x 1 213.4 16.87 147.6 20 / 20 yes g1 canary
2 2 x 1 108.7 33.11 148.7 20 / 20 yes g2
4 2 x 2 63.1 56.96 170.8 20 / 20 yes g4
8 2 x 4 33.8 106.36 178.5 20 / 20 yes g8
  • Every point's slowest clip is within 0.9% of its p50.
  • One 80 GB H100 holds the model at 1 GPU. The g1 server reports 64.85 GiB of GPU memory after loading.
  • Each server became ready inside the 3600 s readiness window: 39.5 min at 1 GPU, then 9.0, 11.7 and 15.0 min at 2, 4 and 8 GPUs. No server log shows a Hugging Face download. The 1-GPU point read the snapshot first, and its engine reported Loading weights took 2158.29 seconds. The recipes therefore leave VIDEO_PREFETCH_MODEL unset.
  • The slowest job (1 GPU) took 118 minutes of the 165-minute video job limit.
  • Attempt 1 was cancelled before any point ran. Its scheduler job was held after a node prolog error, so attempt 2 reran the unchanged head.

Testing:

  • infx generate --config-key wan22-bf16-h100-vllm-omni-video renders 4 recipes. The model path is the pinned snapshot 5be7df9619b54f4e2667b2755bc6a756675b5cd7 under the job's HF hub cache mount.
  • infx.matrix.changed --base origin/main lists only this key: 4 added points, 0 removed.

Limits:

  • One sweep per point; run-to-run variance is not measured.
  • The 1-GPU server needed 39.5 min of the 60-min readiness window on its first read of the snapshot. A slower cache read on another node could miss that window.
  • Video Quality run 44082324105 scored none of the 80 clips (no_reference), because its reference set covers H3 only.

Related: Follows #5082 (B300) and #5084 (B200 GPU-count sweep).

PR checklist and change type

Type of Change

  • Bug fix
  • New feature
  • Configuration change
  • Documentation update
  • Other (please describe)

Checklist

  • I have completed the AI model disclosure and kept it current
  • I have tested my changes locally
  • I have updated documentation if necessary
  • If python -m infx.matrix.changed --base origin/main lists affected config keys, I have appended one new entry covering them to the physical end of inferencex-e2e/perf-changelog.yaml; if it lists none, I have added none. I have not edited historical entries
  • If this PR appends an inferencex-e2e/perf-changelog.yaml entry, it carries exactly one primary sweep label (a maintainer applies it on fork PRs): full-sweep-fail-fast (recommended), full-sweep-enabled, or non-canary-full-sweep-enabled. Optional modifiers all-evals, evals-only, and agentx-fast require a primary label; the last two block reuse while applied.
  • Before merging via reuse, a maintainer with write access to the repository has commented /use <run_id> (or the legacy /reuse-sweep-run) on this PR. Do this only once there is a final full sweep that is all green with evals passing, since after this comment the primary sweep label will no longer automatically kick off new sweeps. Remove and re-add the primary sweep label to force a new sweep.

@github-actions

Copy link
Copy Markdown
Contributor

Thanks for the contribution!

  • Review: If this PR changes files owned by someone other than a repository admin or @SemiAnalysisAI/core, ask one eligible CODEOWNER to complete the latest PR_REVIEW_CHECKLIST.md before contacting a core maintainer on Slack. Follow the template exactly, including As a PR reviewer and CODEOWNER, I have reviewed this and have, so sign-off verification triggers.
  • PR verification: Sweeps run only when the PR appends an inferencex-e2e/perf-changelog.yaml entry and carries exactly one primary label: full-sweep-fail-fast (strongly recommended; canary plus per-matrix fail-fast), full-sweep-enabled (canary; matrix jobs continue after a failure), or non-canary-full-sweep-enabled (no canary or fail-fast). The modifiers all-evals, evals-only, and agentx-fast require a primary label. On fork PRs, a maintainer applies the label. See sweep labels and reuse.
  • After merging: PR authors must ensure all GitHub Actions jobs pass. Transient failures often pass on rerun; see how to rerun failed jobs.
中文

感谢你的贡献!

  • **审阅:**如果 PR 修改的文件归属于仓库管理员及 @SemiAnalysisAI/core 之外的 CODEOWNER,请先联系一位有资格的 CODEOWNER 填写最新的 PR_REVIEW_CHECKLIST.md,再通过 Slack 联系核心维护者。必须严格遵循模板,并保留 As a PR reviewer and CODEOWNER, I have reviewed this and have,才能触发签核验证。
  • **PR 验证:**只有当 PR 在 inferencex-e2e/perf-changelog.yaml 末尾追加了条目,并且恰好带有一个主标签时,才会运行扫描:full-sweep-fail-fast(强烈推荐;canary 加逐矩阵 fail-fast)、full-sweep-enabled(有 canary;矩阵任务在失败后继续运行)或 non-canary-full-sweep-enabled(无 canary,也无 fail-fast)。修饰标签 all-evals、evals-only 和 agentx-fast 必须与主标签一起使用。fork PR 的标签由维护者添加。参见扫描标签与复用。
  • **合并后:**PR 作者必须确保所有 GitHub Actions 任务通过。临时性失败通常可以通过重新运行恢复;参见重新运行失败任务的说明。

@edwingao28
edwingao28 force-pushed the feat/wan22-h100-vllm-omni-video branch from da1b03b to 9e2868e Compare November 17, 2026 07:02
@edwingao28 edwingao28 added the full-sweep-fail-fast Full sweep with canary gate; first failure cancels the rest of that matrix (recommended) label Nov 17, 2026
Comment on lines +47 to +57
PYTHONPATH: /video-package:/infmax-workspace
PYTHONUNBUFFERED: "1"
VIDEO_SERVER_PORT: "30283"
VIDEO_RUNTIME_REVISION: 5917730a4b02aedb44d374173cdc1b10484ccd52
HF_HUB_OFFLINE: "1"
TRANSFORMERS_OFFLINE: "1"
VLLM_WORKER_MULTIPROC_METHOD: spawn
# The release set reuses each prompt across five seeds; cached text embeddings would skip work.
OMNI_DIFFUSION_PROMPT_EMBED_CACHE: "0"
VLLM_OMNI_VIDEO_SYNC_TIMEOUT: "14400"
VLLM_OMNI_SERVER_STORAGE__PATH: /logs/video/server/storage

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🔴 The H100 Wan2.2 servers may never become ready, because the shared HF cache on h100-dsxe is not prefetched. All four sweep points could then time out at readiness. The B300 recipes set VIDEO_PREFETCH_MODEL: "1". That fix exists because b300-dsxe serves Wan from a runner-home network-filesystem HF cache, where the shard loader read at ~10 MB/s and missed the 3600 s window. h100-dsxe mounts its cache from the same /data/home/sa-gha-runner tree. Fix: make the H100 recipes match the B300 ones by adding VIDEO_PREFETCH_MODEL: "1" to all four, unless the h100-dsxe cache is shown to be fast enough.

Why this was flagged

This depends on a condition that cannot be checked from this checkout: whether /data/home/sa-gha-runner/gharunners/hf-hub-cache (h100-dsxe hf-hub-cache volume, configs/runners.yaml:374) is on the same kind of network filesystem as b300-dsxe's runner-home cache. The B300 recipe comment (b300-bf16/serving-client.yaml:54-57) says the engine's multi-thread mmap shard loader reads that cache at ~10 MB/s. At that rate it missed the one-hour readiness window. The H100 recipes are the B300 recipes byte for byte except gpu_type, with the VIDEO_PREFETCH_MODEL block removed (h100-bf16/serving-client.yaml:47-57). readiness.timeout_seconds: 3600 and --init-timeout 3600 stay unchanged. If h100-dsxe's /data/home behaves like b300-dsxe's, each of the 1, 2, 4 and 8 GPU points loads a Wan2.2 A14B snapshot of over 100 GB at ~10 MB/s. Every point then fails readiness and the full-sweep-fail-fast run aborts.

Verification: Trigger: h100-dsxe's runner-home cache at /data/home/sa-gha-runner/gharunners/hf-hub-cache is on the same kind of slow network filesystem that b300-dsxe uses. All four new H100 recipes leave out VIDEO_PREFETCH_MODEL. B300 sets VIDEO_PREFETCH_MODEL: "1" in all four recipes (b300-bf16/serving-client.yaml:57, g1:55, g2:57, g4:57).

@github-actions

Copy link
Copy Markdown
Contributor

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

full-sweep-fail-fast Full sweep with canary gate; first failure cancels the rest of that matrix (recommended)

Projects

Status: No status

Development

Successfully merging this pull request may close these issues.

1 participant