Add Swin2SR ImageProcessorFast #37169

thisisiron · 2025-04-01T11:46:15Z

What does this PR do?

#36978
Add fast image processor for Swin2SR

github-actions · 2025-04-01T11:46:27Z

Hi 👋, thank you for opening this pull request! The pull request is converted to draft by default. The CI will be paused while the PR is in draft mode. When it is ready for review, please click the Ready for review button (at the bottom of the PR page). This will assign reviewers and trigger CI.

yonigozlan

Hi @thisisiron , thanks for working on this! A few things to fix, otherwise LGTM

src/transformers/models/swin2sr/image_processing_swin2sr_fast.py

yonigozlan

Missed a few things in my first review. Also since there is very little processing happening here, the fast processors doesn't look faster than the slow on cpu, so we might need to modify or skip test_fast_is_faster_than_slow

yonigozlan · 2025-04-02T14:16:31Z

tests/models/swin2sr/test_image_processing_swin2sr.py

+    def test_slow_fast_equivalence(self):
+        image_inputs = self.image_processor_tester.prepare_image_inputs(equal_resolution=False, torchify=True)[0]
+
+        image_processor_slow = self.image_processing_class(**self.image_processor_dict, crop_to_patches=True)
+        image_processor_fast = self.fast_image_processing_class(**self.image_processor_dict, crop_to_patches=True)
+
+        encoded_slow = image_processor_slow(image_inputs, return_tensors="pt").pixel_values
+        encoded_fast = image_processor_fast(image_inputs, return_tensors="pt").pixel_values
+
+        self.assertTrue(torch.allclose(encoded_slow, encoded_fast, atol=1e-1))
+
+    def test_slow_fast_equivalence_batched(self):
+        image_inputs = self.image_processor_tester.prepare_image_inputs(equal_resolution=True, torchify=True)
+
+        image_processor_slow = self.image_processing_class(**self.image_processor_dict, crop_to_patches=True)
+        image_processor_fast = self.fast_image_processing_class(**self.image_processor_dict, crop_to_patches=True)
+
+        encoded_slow = image_processor_slow(image_inputs, return_tensors="pt").pixel_values
+        encoded_fast = image_processor_fast(image_inputs, return_tensors="pt").pixel_values
+
+        self.assertTrue(torch.allclose(encoded_slow, encoded_fast, atol=1e-1))


Any reason why this is overidden?

test_slow_fast_equivalence is unnecessary, but I added test_slow_fast_equivalence_batched because it can handle cases in batch when the image sizes are the same. And crop_to_patches=True will be removed.

yonigozlan · 2025-04-02T14:16:47Z

tests/models/swin2sr/test_image_processing_swin2sr.py

+        for image_processing_class in self.image_processor_list:
+            image_processing = image_processing_class(**self.image_processor_dict)


This should be done for all tests

I ran the following command:
RUN_SLOW=1 python -m pytest tests/models/swin2sr/test_image_processing_swin2sr.py

The log below shows the result of executing the above command.

configfile: pyproject.toml plugins: timeout-2.3.1, hypothesis-6.124.2, anyio-4.8.0, asyncio-0.23.8, rich-0.2.0, xdist-3.6.1 asyncio: mode=strict collected 18 items tests/models/swin2sr/test_image_processing_swin2sr.py::Swin2SRImageProcessingTest::test_call_numpy PASSED [ 5%] tests/models/swin2sr/test_image_processing_swin2sr.py::Swin2SRImageProcessingTest::test_call_numpy_4_channels PASSED [ 11%] tests/models/swin2sr/test_image_processing_swin2sr.py::Swin2SRImageProcessingTest::test_call_pil PASSED [ 16%] tests/models/swin2sr/test_image_processing_swin2sr.py::Swin2SRImageProcessingTest::test_call_pytorch PASSED [ 22%] tests/models/swin2sr/test_image_processing_swin2sr.py::Swin2SRImageProcessingTest::test_can_compile_fast_image_processor PASSED [ 27%] tests/models/swin2sr/test_image_processing_swin2sr.py::Swin2SRImageProcessingTest::test_cast_dtype_device PASSED [ 33%] tests/models/swin2sr/test_image_processing_swin2sr.py::Swin2SRImageProcessingTest::test_fast_is_faster_than_slow PASSED [ 38%] tests/models/swin2sr/test_image_processing_swin2sr.py::Swin2SRImageProcessingTest::test_image_processor_from_and_save_pretrained PASSED [ 44%] tests/models/swin2sr/test_image_processing_swin2sr.py::Swin2SRImageProcessingTest::test_image_processor_preprocess_arguments PASSED [ 50%] tests/models/swin2sr/test_image_processing_swin2sr.py::Swin2SRImageProcessingTest::test_image_processor_properties PASSED [ 55%] tests/models/swin2sr/test_image_processing_swin2sr.py::Swin2SRImageProcessingTest::test_image_processor_save_load_with_autoimageprocessor PASSED [ 61%] tests/models/swin2sr/test_image_processing_swin2sr.py::Swin2SRImageProcessingTest::test_image_processor_to_json_file PASSED [ 66%] tests/models/swin2sr/test_image_processing_swin2sr.py::Swin2SRImageProcessingTest::test_image_processor_to_json_string PASSED [ 72%] tests/models/swin2sr/test_image_processing_swin2sr.py::Swin2SRImageProcessingTest::test_init_without_params PASSED [ 77%] tests/models/swin2sr/test_image_processing_swin2sr.py::Swin2SRImageProcessingTest::test_save_load_fast_slow PASSED [ 83%] tests/models/swin2sr/test_image_processing_swin2sr.py::Swin2SRImageProcessingTest::test_save_load_fast_slow_auto PASSED [ 88%] tests/models/swin2sr/test_image_processing_swin2sr.py::Swin2SRImageProcessingTest::test_slow_fast_equivalence PASSED [ 94%] tests/models/swin2sr/test_image_processing_swin2sr.py::Swin2SRImageProcessingTest::test_slow_fast_equivalence_batched PASSED [100%] ============================================================================================================ 18 passed in 6.78s ============================================================================================================

thisisiron · 2025-04-02T15:25:53Z

Missed a few things in my first review. Also since there is very little processing happening here, the fast processors doesn't look faster than the slow on cpu, so we might need to modify or skip test_fast_is_faster_than_slow

@yonigozlan, Then would it be better to skip it?

yonigozlan · 2025-04-09T18:52:10Z

@yonigozlan, Then would it be better to skip it?
In this case yes I think that's the best thing to do

thisisiron · 2025-04-10T04:53:26Z

@yonigozlan, I’ve added the skip to the test.

yonigozlan

Great thanks for contributing! LGTM :)

HuggingFaceDocBuilderDev · 2025-05-07T16:16:07Z

The docs for this PR live here. All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.

* Add fast image processor support for Swin2SR * Add Swin2SR tests of fast image processing * Update docs and remove unnecessary test func * Fix docstring formatting * Skip fast vs slow processing test --------- Co-authored-by: Yoni Gozlan <[email protected]>

github-actions bot marked this pull request as draft April 1, 2025 11:46

thisisiron changed the title ~~Add swin2sr imageprocessorfast~~ Add Swin2SR ImageProcessorFast Apr 1, 2025

thisisiron marked this pull request as ready for review April 1, 2025 11:46

github-actions bot requested review from ydshieh and yonigozlan April 1, 2025 11:47

yonigozlan mentioned this pull request Apr 2, 2025

[Contributions Welcome] Add Fast Image Processors #36978

Closed

81 tasks

yonigozlan reviewed Apr 2, 2025

View reviewed changes

thisisiron added 4 commits April 5, 2025 03:39

Add fast image processor support for Swin2SR

3c50682

Add Swin2SR tests of fast image processing

65c088f

Update docs and remove unnecessary test func

6e6ec2e

Fix docstring formatting

664fb0c

thisisiron force-pushed the add-swin2sr-imageprocessorfast branch from c853267 to 664fb0c Compare April 4, 2025 18:41

Skip fast vs slow processing test

860dc2c

thisisiron requested a review from yonigozlan May 6, 2025 16:51

Merge branch 'main' into add-swin2sr-imageprocessorfast

42fab51

yonigozlan approved these changes May 7, 2025

View reviewed changes

yonigozlan merged commit 5c47d08 into huggingface:main May 7, 2025
20 checks passed

thisisiron deleted the add-swin2sr-imageprocessorfast branch July 23, 2025 17:40

		for image_processing_class in self.image_processor_list:
		image_processing = image_processing_class(**self.image_processor_dict)

Add Swin2SR ImageProcessorFast #37169

Add Swin2SR ImageProcessorFast #37169

Conversation

thisisiron commented Apr 1, 2025

What does this PR do?

Uh oh!

github-actions bot commented Apr 1, 2025

Uh oh!

yonigozlan left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

yonigozlan left a comment

Choose a reason for hiding this comment

Uh oh!

yonigozlan Apr 2, 2025

Choose a reason for hiding this comment

Uh oh!

thisisiron Apr 2, 2025

Choose a reason for hiding this comment

Uh oh!

yonigozlan Apr 2, 2025

Choose a reason for hiding this comment

Uh oh!

thisisiron Apr 2, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

thisisiron commented Apr 2, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

yonigozlan commented Apr 9, 2025

Uh oh!

thisisiron commented Apr 10, 2025

Uh oh!

yonigozlan left a comment

Choose a reason for hiding this comment

Uh oh!

HuggingFaceDocBuilderDev commented May 7, 2025

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

thisisiron Apr 2, 2025 •

edited

Loading

thisisiron commented Apr 2, 2025 •

edited

Loading