[lgeiger]

Purpose

#23207 Broke Qwen3VL since it relies on Qwen2_5_VisionAttention as well but still used the old signature:

RuntimeError: vllm::flash_attn_maxseqlen_wrapper() Expected a value of type 'Tensor' for argument 'max_seqlen' but instead found type 'int'

This PR updates Qwen3VL and Qwen3Omni to fix this.

Test Plan

vllm serve Qwen/Qwen3-VL-2B-Instruct-FP8

Note: I didn't test Qwen3OmniMoE since I didn't have a large enough GPU on hand today, but it's the same code changes as for Qwen3VL so I believe it should be fine.

/cc @Lucaskabela @ywang96

I'm happy to make a followup PR to add torch.compile support to the other modules in Qwen3VL tomorrow once I get a chance to run some benchmarks.

[lgeiger]

max_seqlen is actually a scalar value so this updates the dummy values to match this.

[gemini-code-assist]

Code Review

This pull request addresses a runtime error with torch.compile in Qwen3VL and Qwen3Omni models. The error was caused by passing a Python int instead of a torch.Tensor for the max_seqlen argument in the vision attention mechanism.

The changes correctly modify the compute_attn_mask_seqlen methods in qwen2_5_vl.py, qwen3_omni_moe_thinker.py, and qwen3_vl.py to return torch.Tensor objects for max_seqlen and seqlens, instead of Python scalars or lists. This is achieved by removing .item() and .tolist() calls. The type hints for the forward methods are also updated accordingly.

These changes are consistent, targeted, and directly resolve the reported torch.compile issue. The code quality is good, and I have no further suggestions.

[Lucaskabela]

Thanks for the fix @lgeiger! I was planning on making may way through to Qwen3, but didn't realize this PR had (indirectly) impacted those files

Given that but no test failed, do we have CI coverage for Qwen3 models? (cc @ywang96 @ProExpertProg @zou3519 )

If not it might be worth adding it to the basic initialization suite of tests

[lgeiger]

Thanks for the fix @lgeiger! I was planning on making may way through to Qwen3, but didn't realize this PR had (indirectly) impacted those files

No worries! The addition of torch.compile to the ViT part is amazing. Looking forward to your Qwen3 PRs 🎉

[Lucaskabela]

Also I would definitely welcome adding torch.compile to the vision component of Qwen3, so let me know if you need review/support on that or otherwise want to own it :) It should be relatively straightforward

The only "gotcha" is that right now the torch.compile integration compiles each instantiation of the VisionBlock separately - I am planning on putting out a fix for that shortly but in the meanwhile don't be surprised to see 30+ compile logs if applying torch.compile to the VisionBlock (it shouldn't effect correctness or runtime, but adds slightly to compile time and is not needed hence why I plan to fix this soon in followup)

[ywang96]

Thanks for fixing this! Stamping it now and I will test Qwen3-Omni on my end.

Given that but no test failed, do we have CI coverage for Qwen3 models?

Yea we didn't add them onto CI back then because the model support PR was merged before the model release, but we should definitely add one now that the smaller version of the models are available. I will take that as an action item

[ywang96]

@lgeiger @Lucaskabela I added an fix for the missing attribute of self.vllm_config for Qwen3Omni since this model's encoder execution in fact inherits those from Qwen2.5Omni - something to also keep in mind when we add the support for Qwen3-VL, and I wonder if we should just make this an interface requirement for multimodal models.