Fix temporal padding in Qwen2VLImageProcessor when the number of frames is not divisible by temporal_patch_size #38076
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Contributor
|
Hi 👋, thank you for opening this pull request! The pull request is converted to draft by default. The CI will be paused while the PR is in draft mode. When it is ready for review, please click the |
Contributor
Author
|
@zucchini-nlp Could you please review this PR? |
…e not divisible by temporal_patch_size
ritwickchaudhry
force-pushed
the
fix-qwen2vl-temporal-padding
branch
from
f876f69 to
9bd3e16
Compare
zucchini-nlp
approved these changes
May 12, 2025
|
|
||
| # Check the shape after padding | ||
| expected_output_video_shape = (102900, 1176) # Adjusted based on padding | ||
| self.assertEqual(tuple(encoded_video.shape), expected_output_video_shape) |
Member
There was a problem hiding this comment.
ultra nit: asserting ListEqual can give more informative error output when tests fail :)
|
The docs for this PR live here. All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update. |
4 tasks
zucchini-nlp
pushed a commit
to zucchini-nlp/transformers
that referenced
this pull request
May 14, 2025
…es is not divisible by temporal_patch_size (huggingface#38076) Qwen2VL: Fix temporal padding in Qwen2VLImageProcessor when frames are not divisible by temporal_patch_size
Contributor
|
qwen3vl_video_processor has same problem.... |
Member
|
@yaogang2060 can you submit a PR and tag me pls? |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
4 participants
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
This PR fixes an issue in Qwen2VLImageProcessor where the current implementation does not correctly handle cases when the number of video frames is not divisible by
temporal_patch_size.Problem:
The existing logic repeats the last frame
temporal_patch_size- 1 times. This works correctly whentemporal_patch_sizeequals 2 but fails when the size is greater.Solution:
The fix replaces:
repeats = np.repeat(patches[-1][np.newaxis], temporal_patch_size - 1, axis=0)with:
repeats = np.repeat(patches[-1][np.newaxis], temporal_patch_size - (patches.shape[0] % temporal_patch_size), axis=0)This ensures that the correct number of padding frames are added when the frame count is not divisible by the
temporal_patch_size.Additional Changes:
Added a unit test to verify the padding logic for edge cases where the number of frames is not divisible by the patch size.
Issue Reference:
Fixes #38003