[FixBug]online serving fails for high-resolution videos

[princepride]

Purpose

Fix #128

This PR resolves a critical bug in the Qwen3-Omni model's deepstack feature that caused crashes when processing high-resolution videos. The root cause was incorrect tensor dimension indexing when retrieving the sequence length from input_ids, which led to shape mismatches between visual embeddings and hidden states during the deepstack processing.

Root Cause:

• input_ids has shape [batch_size, seq_len], e.g., [1, 8192]
• The code incorrectly used input_ids.size(0) to get sequence length, which returned the batch size (1) instead of the actual sequence length
• This caused only 1 token's worth of deepstack embeddings to be retrieved from the buffer, while the model expected the full sequence length (e.g., 8192 or 3701 tokens)

Fix:
Changed input_ids.size(0) to input_ids.size(1) in two locations to correctly retrieve the sequence length (second dimension) instead of batch size (first dimension).
BTW, I used two H200s for testing, and an OOM error occurred. Therefore, I adjusted the qwen3 omni deployment file.

Test Plan

1. Setup: Start vLLM-Omni server with Qwen3-Omni-30B-A3B-Instruct model

   vllm serve /path/to/Qwen3-Omni-30B-A3B-Instruct --omni --port 8091

2. Test with high-resolution video: Run the multimodal generation client with a large video file

   python examples/online_serving/qwen3_omni/openai_chat_completion_client_for_multimodal_generation.py \
       --query-type use_video \
       --video-path sample_demo_2.mp4 \
       --prompt "explain this video" \
       --model /path/to/Qwen3-Omni-30B-A3B-Instruct

sample_demo_2.mp4

Test Result

audio_0.wav

Essential Elements of an Effective PR Description Checklist

• The purpose of the PR, such as "Fix some issue (link existing issues this PR will resolve)".
• The test plan, such as providing test command.
• The test results, such as pasting the results comparison before and after, or e2e results
• (Optional) The necessary documentation update, such as updating supported_models.md and examples for a new model.
• (Optional) Release notes update. If your change is user facing, please update the release notes draft.

[chatgpt-codex-connector]

The account who enabled Codex for this repo no longer has access to Codex. Please contact the admins of this repo to enable Codex again.

[SamitHuang]

nice bugfix!

[princepride]

@SamitHuang I already revert the yaml change, can you help merge it, thank you!😊

[Gaohan123]

LGTM. Great!

[david6666666]

nice catch !