Repository navigation
feat: update FastVideo diffusers worker to new typed API - #9591
Conversation
WalkthroughThis PR refactors the FastVideo worker to support a typed request/response API with expanded configuration for torch.compile, FP4 quantization, and attention backend selection. The core worker implementation is rewritten, documentation is updated to reflect new defaults, and local deployment infrastructure is modernized to use utility script sourcing and updated environment variables. ChangesFastVideo Worker Typed API & Configuration Update
🎯 4 (Complex) | ⏱️ ~45 minutes 🚥 Pre-merge checks | ✅ 4 | ❌ 1❌ Failed checks (1 warning)
✅ Passed checks (4 passed)
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
9f6f384 to
0db3ada
Compare
0db3ada to
5be627c
Compare
5be627c to
566bc49
Compare
|
@SolitaryThinker @vishruthb I did a quick check using Docker Compose and have following error: After removing those lines:
I was able to run it but got another error on RTX A6000: Attached the bash script used for reproduction: run_docker.sh Could you please verify if the examples is functional using local and Docker setup? Thanks. |
|
Hi @vishruthb @SolitaryThinker just following up on Jakub's feedback above: #9591 (comment) |
Signed-off-by: Vishruth Bharath <vishruthbharath@gmail.com>
Signed-off-by: Vishruth Bharath <vishruthbharath@gmail.com>
Signed-off-by: Vishruth Bharath <vishruthbharath@gmail.com>
Signed-off-by: Vishruth Bharath <vishruthbharath@gmail.com>
Signed-off-by: Vishruth Bharath <vishruthbharath@gmail.com>
Signed-off-by: Vishruth Bharath <vishruthbharath@gmail.com>
Signed-off-by: Vishruth Bharath <vishruthbharath@gmail.com>
The ai-dynamo==1.2.1 wheel installed by the example Dockerfile does not export dynamo.llm.WorkerType, and its register_llm() has no worker_type parameter, so importing/passing it crashed the worker at startup. Also gate VIDEO_SPARSE_ATTN on compute capability >= 9.0 before VideoGenerator.from_config: fastvideo-kernel 0.2.6 compiles its VSA kernels for sm90a only and its Triton fallback fails at runtime on pre-Hopper GPUs (e.g. sm86), previously surfacing as an opaque background-process error after weights had already loaded. Signed-off-by: Vishruth Bharath <vishruthbharath@gmail.com>
- Move the torch import to module level; torch is a hard dependency of fastvideo, so the try/except ImportError path was unreachable. - Catch ValueError from request validation specifically in create_video; keep a deliberate broad catch for FastVideo internals that yields the documented VideoCreateResponse(status="failed") instead of re-raising, so /v1/videos clients receive structured errors rather than transport failures. - Default --vsa-sparsity to None so users can opt out of setting VSA_sparsity in PipelineSelection.experimental; apply the documented 0.8 default only when the flag is unset and the backend is VIDEO_SPARSE_ATTN. Signed-off-by: Vishruth Bharath <vishruthbharath@gmail.com>
Document the compute-capability requirements surfaced by the new startup gate (FastWan2.1+VSA needs >= 9.0, LTX2+TORCH_SDPA as the pre-Hopper path, NVFP4 needs >= 10.0) and replace the permanently redirected tree/ Dockerfile link with its blob/ form for the lychee check. Signed-off-by: Vishruth Bharath <vishruthbharath@gmail.com>
eb535d9 to
ed1d4dc
Compare
Signed-off-by: Ryan McCormick <rmccormick@nvidia.com>
|
/ok to test 91ddc82 |
|
/ok to test 5b923b9 |
Overview:
Update the FastVideo example worker in
examples/diffusers/to use FastVideo's new typed API (fastvideo.api), replacing the legacygenerate_video()interface. Full native backend promotion is tracked separately in #9535.This version also wires the optimization flag to FastVideo's real NVFP4 transformer quantization path and pins a FastVideo commit that implements that typed carrier.
Details:
examples/diffusers/worker.pyVideoGenerator.from_config(GeneratorConfig(...))andgenerator.generate(GenerationRequest(...))with structured config objects (EngineConfig,OffloadConfig,CompileConfig,QuantizationConfig,PipelineSelection)--model,--num-gpus,--attention-backend, offload toggles,--torch-compile,--fp4-quantization,--vsa-sparsity--fp4-quantizationnow requestsQuantizationConfig(transformer_quant="NVFP4"), matching FastVideo's typed NVFP4 transformer quantization APIdata[0].b64_json;urlis rejected because this example worker does not host generated filesFastVideo/FastWan2.1-T2V-1.3B-DiffusersSupporting files
Dockerfile: pinned FastVideo install toe6022c20b27c94ef546143a7796bb73e1c77cdb4for typed API and NVFP4 transformer quantization supportDockerfile/local/docker-compose.yml: defaultTORCH_CUDA_ARCH_LISTnow includes12.0for RTX 50-series GPUs, alongside Blackwell data center targetslocal/run_local.sh: updated for new CLI flags, uses shared launch utilitieslocal/docker-compose.yml: updated worker commanddocs/features/diffusion/fastvideo.md: updated for new API, NVFP4 behavior, RTX 50-series build defaults, and current response-format behaviorTesting done
Static/local verification:
nvidia-smi python3 -c "import torch; print(torch.cuda.get_device_name(0)); print(torch.cuda.get_device_capability(0))" python3 -m py_compile examples/diffusers/worker.py git diff --check origin/main...HEADVerified host GPU:
NVIDIA GeForce RTX 5090, compute capability(12, 0).Verified the pinned FastVideo revision exposes the required typed NVFP4 path:
get_quantization_config("NVFP4")generator_config_to_fastvideo_args(...)resolvesengine.quantization.transformer_quantFastVideoArgs._apply_transformer_quant()pins the quant config ontopipeline_config.dit_config.quant_configSmoke test from this branch before the latest cleanup/NVFP4 pin update (RTX 5090, CUDA 12.8):
Response:
"status": "completed","object": "video",b64_jsonpresent.Known limitations
FASTVIDEO_ATTENTION_BACKENDfrom the--attention-backendCLI flag. When Flash Attention packages are unavailable, FastVideo falls back to Torch SDPA automatically. Explicit FA4/FA3 import path alignment with upstream FastVideo is tracked separately.Where should the reviewer start?
examples/diffusers/worker.py- the core change. Everything else follows from it.Related Issues:
Closes DIS-2247