[NPUW] Fix logic to padding VLM 3D Position Ids #31174
Open
+39
−4
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
github-actions
bot
added
category: NPU
GuoliangShiIntel
changed the title
|
@AsyaPronina @dmatveev Please take a look. |
|
@TolyaTalamanov Please take a look. |
GuoliangShiIntel
force-pushed
the
sgl/fix_3d_position_id
branch
from
0a83da7 to
36b0e45
Compare
AlexanderKalistratov
approved these changes
Jul 4, 2025
|
build_jenkins |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Details:
Background:
Regular LLM uses 2D position_ids
[BATCH, SEQ_LEN], Qwen2.5 VL/Omni uses 3D position_ids[3, BATCH, SEQ_LEN]The first dimension (3) represents the three components of position encoding:
time, height, and widthenabling alignment across multimodal inputs like text, audio, and video
Issue:
Currently, the
position_idsdata is always placed as a continuous block at the end of theposition_ids_paddedbuffer. This approach works fine for padding shapes like [1, 500] to [1, 1024]. However, it causes issues for shapes like [3, 1, 500] to [3, 1, 1024].Tickets: