Repository navigation
[VideoGenX] add vLLM-Omni Wan2.2 on H100 with 1, 2, 4 and 8 GPUs / [VideoGenX] 在 H100 上新增 vLLM-Omni Wan2.2(1、2、4、8 卡) - #5111
edwingao28 wants to merge 2 commits into
Conversation
|
Thanks for the contribution!
中文感谢你的贡献!
|
da1b03b to
9e2868e
Compare
| PYTHONPATH: /video-package:/infmax-workspace | ||
| PYTHONUNBUFFERED: "1" | ||
| VIDEO_SERVER_PORT: "30283" | ||
| VIDEO_RUNTIME_REVISION: 5917730a4b02aedb44d374173cdc1b10484ccd52 | ||
| HF_HUB_OFFLINE: "1" | ||
| TRANSFORMERS_OFFLINE: "1" | ||
| VLLM_WORKER_MULTIPROC_METHOD: spawn | ||
| # The release set reuses each prompt across five seeds; cached text embeddings would skip work. | ||
| OMNI_DIFFUSION_PROMPT_EMBED_CACHE: "0" | ||
| VLLM_OMNI_VIDEO_SYNC_TIMEOUT: "14400" | ||
| VLLM_OMNI_SERVER_STORAGE__PATH: /logs/video/server/storage |
There was a problem hiding this comment.
🔴 The H100 Wan2.2 servers may never become ready, because the shared HF cache on h100-dsxe is not prefetched. All four sweep points could then time out at readiness. The B300 recipes set VIDEO_PREFETCH_MODEL: "1". That fix exists because b300-dsxe serves Wan from a runner-home network-filesystem HF cache, where the shard loader read at ~10 MB/s and missed the 3600 s window. h100-dsxe mounts its cache from the same /data/home/sa-gha-runner tree. Fix: make the H100 recipes match the B300 ones by adding VIDEO_PREFETCH_MODEL: "1" to all four, unless the h100-dsxe cache is shown to be fast enough.
Why this was flagged
This depends on a condition that cannot be checked from this checkout: whether /data/home/sa-gha-runner/gharunners/hf-hub-cache (h100-dsxe hf-hub-cache volume, configs/runners.yaml:374) is on the same kind of network filesystem as b300-dsxe's runner-home cache. The B300 recipe comment (b300-bf16/serving-client.yaml:54-57) says the engine's multi-thread mmap shard loader reads that cache at ~10 MB/s. At that rate it missed the one-hour readiness window. The H100 recipes are the B300 recipes byte for byte except gpu_type, with the VIDEO_PREFETCH_MODEL block removed (h100-bf16/serving-client.yaml:47-57). readiness.timeout_seconds: 3600 and --init-timeout 3600 stay unchanged. If h100-dsxe's /data/home behaves like b300-dsxe's, each of the 1, 2, 4 and 8 GPU points loads a Wan2.2 A14B snapshot of over 100 GB at ~10 MB/s. Every point then fails readiness and the full-sweep-fail-fast run aborts.
Verification: Trigger: h100-dsxe's runner-home cache at /data/home/sa-gha-runner/gharunners/hf-hub-cache is on the same kind of slow network filesystem that b300-dsxe uses. All four new H100 recipes leave out VIDEO_PREFETCH_MODEL. B300 sets VIDEO_PREFETCH_MODEL: "1" in all four recipes (b300-bf16/serving-client.yaml:57, g1:55, g2:57, g4:57).
|
View unofficial run (performance): https://inferencex.semianalysis.com/inference?unofficialRun=44035678622 View unofficial run (accuracy): https://inferencex.semianalysis.com/evaluation?unofficialRun=44035678622 |
Summary
Wan2.2-T2V-A14B has vLLM-Omni rows only on B200 and B300 (
wan22-bf16-b200-vllm-omni-video,wan22-bf16-b300-vllm-omni-video). No Hopper GPU runs it, so VideoGenX has no H100 reference for this model.Changes:
wan22/vllm-omni/h100-bf16/. Each is the B200 recipe withgpu_type: h100. The image digest (vLLM-Omni v0.32.0), engine flags and env are the same, includingOMNI_DIFFUSION_PROMPT_EMBED_CACHE=0.serving-client-g1.yamlserving-client-g2.yamlserving-client-g4.yamlserving-client.yamlwan22-bf16-h100-vllm-omni-videooncluster:h100-dsxe:wan22.yamlrelease-v1, C1.perf-changelog.yaml: one appended block for the new key.MODELS.md,docs/video-generation.mdandSUBMITTING.mdadd H100 to the hardware that runs Wan2.2 on vLLM-Omni.VIDEO_PREFETCH_MODEL, offload or quantization.Results: H100 80 GB on
cluster:h100-dsxe, vLLM-Omni v0.32.0,wan22.yamlrelease-v1at C1, PR sweep 44035678622 attempt 2 at9e2868e8a.validate-video-resultspassed.Loading weights took 2158.29 seconds. The recipes therefore leaveVIDEO_PREFETCH_MODELunset.Testing:
infx generate --config-key wan22-bf16-h100-vllm-omni-videorenders 4 recipes. The model path is the pinned snapshot5be7df9619b54f4e2667b2755bc6a756675b5cd7under the job's HF hub cache mount.infx.matrix.changed --base origin/mainlists only this key: 4 added points, 0 removed.Limits:
no_reference), because its reference set covers H3 only.Related: Follows #5082 (B300) and #5084 (B200 GPU-count sweep).
PR checklist and change type
Type of Change
Checklist
python -m infx.matrix.changed --base origin/mainlists affected config keys, I have appended one new entry covering them to the physical end ofinferencex-e2e/perf-changelog.yaml; if it lists none, I have added none. I have not edited historical entriesinferencex-e2e/perf-changelog.yamlentry, it carries exactly one primary sweep label (a maintainer applies it on fork PRs):full-sweep-fail-fast(recommended),full-sweep-enabled, ornon-canary-full-sweep-enabled. Optional modifiersall-evals,evals-only, andagentx-fastrequire a primary label; the last two block reuse while applied./use <run_id>(or the legacy/reuse-sweep-run) on this PR. Do this only once there is a final full sweep that is all green with evals passing, since after this comment the primary sweep label will no longer automatically kick off new sweeps. Remove and re-add the primary sweep label to force a new sweep.