Repository navigation
[VideoGenX] add vLLM-Omni Wan2.2 on B300 with 1, 2, 4 and 8 GPUs / [VideoGenX] 在 B300 上新增 vLLM-Omni Wan2.2(1、2、4、8 卡) - #5082
Conversation
Co-authored-by: Perplexity <noreply@perplexity.ai>
|
Thanks for the contribution!
中文感谢你的贡献!
|
Run the B200 Wan2.2-T2V-A14B deployment unchanged on 8 B300 GPUs: same image digest, wan22.yaml release-v1 workload, CFG parallel 2 x Ulysses 4 and server environment.
Add 1, 2 and 4 GPU entries to wan22-bf16-b300-vllm-omni-video with the same per-count deployments as B200, and describe both GPU curves in the docs.
Co-authored-by: Perplexity <noreply@perplexity.ai>
1b7bd2e to
6db07ac
Compare
|
Claude finished @edwingao28's task in 48s —— View job Review of PR #5082
LGTM - no blocking issues found. The B300 8-GPU recipe matches the B200 Wan2.2 recipe on main except for One small note, not a blocker: on main the B200 Wan2.2 row still only has the 8-GPU point because #5084 is not merged. Until it merges, the new MODELS.md, I could not run |
6db07ac to
736e2e1
Compare
|
View unofficial run (performance): https://inferencex.semianalysis.com/inference?unofficialRun=43747718442 View unofficial run (accuracy): https://inferencex.semianalysis.com/evaluation?unofficialRun=43747718442 |
… B300 The b300-dsxe HF cache sits on a network filesystem. vLLM-Omni's multi-thread mmap shard loader read the Wan transformer there at about 10 MB/s, so the server missed its one-hour readiness window, while the text encoder streamed from the same cache at about 240 MB/s. The video server now reads every snapshot file sequentially once when the recipe sets VIDEO_PREFETCH_MODEL=1, so the engine's load hits the page cache. The B300 Wan recipes opt in.
# Conflicts: # inferencex-e2e/perf-changelog.yaml
|
/use 43747718442 |
|
@edwingao28 staged run 43747718442. This run has no staged performance curves. This run remains available across future |
Summary
Adds a B300 Wan2.2-T2V-A14B baseline with the workload from the B200 Wan2.2 baseline, swept on 1, 2, 4 and 8 GPUs like the B200 GPU sweep. Rebased onto main now that #5081 is merged.
Changes:
wan22/vllm-omni/b300-bf16/(gpu_type: b300, same vLLM-Omni v0.32.0 digest, prompt-embedding cache off):serving-client-g1.yamlserving-client-g2.yamlserving-client-g4.yamlserving-client.yamlwan22-bf16-b300-vllm-omni-videooncluster:b300-dsxe:wan22.yamlrelease-v1, C1.perf-changelog.yamlblock; MODELS.md, the video docs and SUBMITTING.md describe the B200 and B300 GPU curves.Verification:
python -m infx.matrix.planagainst main yields the 4 B300 video jobs above. GPU sweep pending.VIDEO_PREFETCH_MODEL=1, set in the B300 recipes,evaluator/srt_selected.pynow reads each snapshot file sequentially once before it starts the engine, so the engine's load hits the page cache. The engine command is unchanged. New tests check that every file is read through snapshot symlinks and that the prefetch runs only when the recipe asks for it.中文
概要
为 B200 Wan2.2 基线 的 Wan2.2-T2V-A14B 工作负载新增 B300 基线,并与 B200 多卡扫描 一样在 1、2、4、8 卡上扫描。#5081 已合并,本 PR 已变基到 main。
1 卡:CFG 1 × Ulysses 1,无 VAE patch 并行;2 卡:CFG 2 × Ulysses 1;4 卡:CFG 2 × Ulysses 2;8 卡:CFG 2 × Ulysses 4。
cluster:b300-dsxe上的wan22-bf16-b300-vllm-omni-video:wan22.yamlrelease-v1,C1。快照预读: 首次 B300 canary 未能在一小时内就绪。b300-dsxe 的 HF 缓存位于网络文件系统上,vLLM-Omni 多线程 mmap 分片加载仅约 10 MB/s(同一缓存中的文本编码器约 240 MB/s)。B300 配方设置
VIDEO_PREFETCH_MODEL=1后,evaluator/srt_selected.py在启动引擎前顺序读取一遍快照文件,引擎加载直接命中页缓存;引擎命令不变。