Skip to content

[VideoGenX] add vLLM-Omni Wan2.2 on B300 with 1, 2, 4 and 8 GPUs / [VideoGenX] 在 B300 上新增 vLLM-Omni Wan2.2(1、2、4、8 卡) - #5082

Merged
edwingao28 merged 6 commits into
mainfrom
feat/wan22-b300-vllm-omni-video
Nov 16, 2026
Merged

edwingao28 merged 6 commits into
mainfrom
feat/wan22-b300-vllm-omni-video

Conversation

@edwingao28

@edwingao28 edwingao28 commented Nov 14, 2026 •

Copy link
Copy Markdown
Collaborator

Summary

Adds a B300 Wan2.2-T2V-A14B baseline with the workload from the B200 Wan2.2 baseline, swept on 1, 2, 4 and 8 GPUs like the B200 GPU sweep. Rebased onto main now that #5081 is merged.

Changes:

  • Recipes under wan22/vllm-omni/b300-bf16/ (gpu_type: b300, same vLLM-Omni v0.32.0 digest, prompt-embedding cache off):
GPUs Recipe CFG parallel Ulysses VAE patch parallel
1 serving-client-g1.yaml 1 1 off
2 serving-client-g2.yaml 2 1 2
4 serving-client-g4.yaml 2 2 4
8 serving-client.yaml 2 4 8
  • Row wan22-bf16-b300-vllm-omni-video on cluster:b300-dsxe: wan22.yaml release-v1, C1.
  • One perf-changelog.yaml block; MODELS.md, the video docs and SUBMITTING.md describe the B200 and B300 GPU curves.

Verification: python -m infx.matrix.plan against main yields the 4 B300 video jobs above. GPU sweep pending.

  • Snapshot prefetch: the first B300 canary missed the server's one-hour readiness window. b300-dsxe serves the model from a network-filesystem HF cache, where vLLM-Omni's multi-thread mmap shard loader read the transformer at about 10 MB/s (the text encoder streamed from the same cache at about 240 MB/s). With VIDEO_PREFETCH_MODEL=1, set in the B300 recipes, evaluator/srt_selected.py now reads each snapshot file sequentially once before it starts the engine, so the engine's load hits the page cache. The engine command is unchanged. New tests check that every file is read through snapshot symlinks and that the prefetch runs only when the recipe asks for it.
中文

概要

为 B200 Wan2.2 基线 的 Wan2.2-T2V-A14B 工作负载新增 B300 基线,并与 B200 多卡扫描 一样在 1、2、4、8 卡上扫描。#5081 已合并,本 PR 已变基到 main。

  • 1 卡:CFG 1 × Ulysses 1,无 VAE patch 并行;2 卡:CFG 2 × Ulysses 1;4 卡:CFG 2 × Ulysses 2;8 卡:CFG 2 × Ulysses 4。

  • cluster:b300-dsxe 上的 wan22-bf16-b300-vllm-omni-video:wan22.yaml release-v1,C1。

  • 快照预读: 首次 B300 canary 未能在一小时内就绪。b300-dsxe 的 HF 缓存位于网络文件系统上,vLLM-Omni 多线程 mmap 分片加载仅约 10 MB/s(同一缓存中的文本编码器约 240 MB/s)。B300 配方设置 VIDEO_PREFETCH_MODEL=1 后,evaluator/srt_selected.py 在启动引擎前顺序读取一遍快照文件,引擎加载直接命中页缓存;引擎命令不变。

edwingao28 added a commit that referenced this pull request Nov 14, 2026
Co-authored-by: Perplexity <noreply@perplexity.ai>
@github-actions

Copy link
Copy Markdown
Contributor

Thanks for the contribution!

  • Review: If this PR changes files owned by someone other than a repository admin or @SemiAnalysisAI/core, ask one eligible CODEOWNER to complete the latest PR_REVIEW_CHECKLIST.md before contacting a core maintainer on Slack. Follow the template exactly, including As a PR reviewer and CODEOWNER, I have reviewed this and have, so sign-off verification triggers.
  • PR verification: Sweeps run only when the PR appends an inferencex-e2e/perf-changelog.yaml entry and carries exactly one primary label: full-sweep-fail-fast (strongly recommended; canary plus per-matrix fail-fast), full-sweep-enabled (canary; matrix jobs continue after a failure), or non-canary-full-sweep-enabled (no canary or fail-fast). The modifiers all-evals, evals-only, and agentx-fast require a primary label. On fork PRs, a maintainer applies the label. See sweep labels and reuse.
  • After merging: PR authors must ensure all GitHub Actions jobs pass. Transient failures often pass on rerun; see how to rerun failed jobs.
中文

感谢你的贡献!

  • **审阅:**如果 PR 修改的文件归属于仓库管理员及 @SemiAnalysisAI/core 之外的 CODEOWNER,请先联系一位有资格的 CODEOWNER 填写最新的 PR_REVIEW_CHECKLIST.md,再通过 Slack 联系核心维护者。必须严格遵循模板,并保留 As a PR reviewer and CODEOWNER, I have reviewed this and have,才能触发签核验证。
  • **PR 验证:**只有当 PR 在 inferencex-e2e/perf-changelog.yaml 末尾追加了条目,并且恰好带有一个主标签时,才会运行扫描:full-sweep-fail-fast(强烈推荐;canary 加逐矩阵 fail-fast)、full-sweep-enabled(有 canary;矩阵任务在失败后继续运行)或 non-canary-full-sweep-enabled(无 canary,也无 fail-fast)。修饰标签 all-evals、evals-only 和 agentx-fast 必须与主标签一起使用。fork PR 的标签由维护者添加。参见扫描标签与复用。
  • **合并后:**PR 作者必须确保所有 GitHub Actions 任务通过。临时性失败通常可以通过重新运行恢复;参见重新运行失败任务的说明。

Run the B200 Wan2.2-T2V-A14B deployment unchanged on 8 B300 GPUs: same
image digest, wan22.yaml release-v1 workload, CFG parallel 2 x Ulysses 4
and server environment.
Add 1, 2 and 4 GPU entries to wan22-bf16-b300-vllm-omni-video with the
same per-count deployments as B200, and describe both GPU curves in the
docs.
edwingao28 added a commit that referenced this pull request Nov 15, 2026
Co-authored-by: Perplexity <noreply@perplexity.ai>
@edwingao28
edwingao28 force-pushed the feat/wan22-b300-vllm-omni-video branch from 1b7bd2e to 6db07ac Compare November 15, 2026 03:04
@edwingao28 edwingao28 added the full-sweep-fail-fast Full sweep with canary gate; first failure cancels the rest of that matrix (recommended) label Nov 15, 2026
@edwingao28 edwingao28 changed the title [VideoGenX] add vLLM-Omni Wan2.2 video config on B300 / [VideoGenX] 在 B300 上新增 vLLM-Omni Wan2.2 视频配置 [VideoGenX] add vLLM-Omni Wan2.2 on B300 with 1, 2, 4 and 8 GPUs / [VideoGenX] 在 B300 上新增 vLLM-Omni Wan2.2(1、2、4、8 卡) Nov 15, 2026
@edwingao28
edwingao28 marked this pull request as ready for review November 15, 2026 03:04
@edwingao28
edwingao28 requested a review from a team November 15, 2026 03:04
@claude

claude Bot commented Nov 15, 2026 •

Copy link
Copy Markdown
Contributor

Claude finished @edwingao28's task in 48s —— View job


Review of PR #5082

  • Gather context (diff, existing reviews)
  • Validate recipes, master config, and perf-changelog
  • Post review

LGTM - no blocking issues found.

The B300 8-GPU recipe matches the B200 Wan2.2 recipe on main except for gpu_type. The 1, 2 and 4-GPU recipes differ only in their CFG, Ulysses and VAE patch parallel settings. The image is a public digest-pinned vllm/vllm-omni image. The perf-changelog.yaml entry is appended at the end of the file. Video points route to SRT_VIDEO, which does not depend on a cluster lane, and they read the model from the b300-dsxe hf-hub-cache volume (~/.cache/huggingface/hub). As the PR description says, the Wan2.2 snapshot has to be staged there before the sweep.

One small note, not a blocker: on main the B200 Wan2.2 row still only has the 8-GPU point because #5084 is not merged. Until it merges, the new MODELS.md, video-generation.md and SUBMITTING.md text that says Wan2.2 runs on "1, 2, 4 and 8 B200 or B300 GPUs" overstates B200. Merge #5084 first or word the docs for B300 only.

I could not run infx.matrix.changed because the command needed approval. That doesn't affect the review: the PR already adds a changelog entry for wan22-bf16-b300-vllm-omni-video.

@claude claude Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code review found no issues

No high-confidence issues detected in this change.

@edwingao28
edwingao28 force-pushed the feat/wan22-b300-vllm-omni-video branch from 6db07ac to 736e2e1 Compare November 15, 2026 04:36
@github-actions

github-actions Bot commented Nov 15, 2026 •

Copy link
Copy Markdown
Contributor

… B300

The b300-dsxe HF cache sits on a network filesystem. vLLM-Omni's
multi-thread mmap shard loader read the Wan transformer there at about
10 MB/s, so the server missed its one-hour readiness window, while the
text encoder streamed from the same cache at about 240 MB/s.

The video server now reads every snapshot file sequentially once when
the recipe sets VIDEO_PREFETCH_MODEL=1, so the engine's load hits the
page cache. The B300 Wan recipes opt in.
# Conflicts:
#	inferencex-e2e/perf-changelog.yaml
@edwingao28

Copy link
Copy Markdown
Collaborator Author

/use 43747718442

@github-actions

github-actions Bot commented Nov 16, 2026 •

Copy link
Copy Markdown
Contributor

@edwingao28 staged run 43747718442.

This run has no staged performance curves.

This run remains available across future /stage-results requests. Staging the same run ID again updates its staged data. Staging workflow

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

full-sweep-fail-fast Full sweep with canary gate; first failure cancels the rest of that matrix (recommended)

Projects

Status: Done

Development

Successfully merging this pull request may close these issues.

1 participant