[proto] Ported all transforms to the new API #6305

vfdev-5 · 2022-07-22T17:57:29Z

Description:

Ported all transforms to the new API
Added tests

~~Please merge #6272 before this PR~~

What remains

Polish sharp edges:
- Search and fix for TODOs in torchvision/prototype/transforms and test/test_prototype_*. Mostly those to implement a missing functionality.
Better testing:
- check if we test all ops and transforms
- functional ops have to be tested on result correctness, jit/eager, devices etc. Philip was proposing to provide a simplified way to test that.
- transforms have to be tested on parameter generation and checking if functional counterpart is correctly called via mocking (this is already done for majority of transforms in test_prototype_transforms.py)
- we need to test specific Feature.op for example BoundingBox.rotate as they change metadata like image_size.

CI Failures to take into account:

FAILED test/test_prototype_transforms_functional.py::test_eager_vs_scripted[gaussian_blur_image_tensor-26]

____________ test_eager_vs_scripted[gaussian_blur_image_tensor-26] _____________
Traceback (most recent call last):
  File "/home/runner/work/vision/vision/test/test_prototype_transforms_functional.py", line 740, in test_eager_vs_scripted
    torch.testing.assert_close(eager, scripted)
  File "/opt/hostedtoolcache/Python/3.7.13/x64/lib/python3.7/site-packages/torch/testing/_comparison.py", line 1359, in assert_close
    msg=msg,
  File "/opt/hostedtoolcache/Python/3.7.13/x64/lib/python3.7/site-packages/torch/testing/_comparison.py", line 1093, in assert_equal
    raise error_metas[0].to_error(msg)
AssertionError: Tensor-likes are not equal!

* Added supported/unsupported data checks in the tests for cutmix/mixup * Added RandomRotation, RandomAffine transforms tests * Added tests for RandomZoomOut, Pad * Update test_prototype_transforms.py

* Added GaussianBlur transform and tests * Fixing code format * Copied correctness test

…s-api

* Added random color transforms and tests * Disable smoke test for RandomSolarize, RandomAdjustSharpness

- replaced real image creation by mocks for other tests

…s-api

* WIP [proto] Added functional elastic transform with tests * Added more functional tests * WIP on elastic op * Added elastic transform and tests * Added tests * Added tests for ElasticTransform

…s-api

Added tests

* Added image_size computation for BoundingBox.rotate if expand * Added tests

datumbox

Thanks @vfdev-5, looks good overall. I've added a few comments, let me know what you think. I'm not going to check the tests because as you indicate more discussions will be required over the testing strategy.

torchvision/prototype/transforms/_color.py

torchvision/prototype/transforms/_geometry.py

torchvision/prototype/transforms/_utils.py

torchvision/prototype/transforms/functional/_geometry.py

…s-api

datumbox

LGTM, I haven't checked the testing strategy but I know @pmeier's plan is to revise it thoroughly so I'm happy to go with whatever you prefer for now.

datumbox · 2022-07-28T13:23:49Z

torchvision/prototype/transforms/_utils.py



 def has_all(sample: Any, *types: Type) -> bool:
-    return not bool(set(types) - set(_extract_types(sample)))
+    flat_sample, _ = tree_flatten(sample)
+    return not bool(set(types) - set([type(obj) for obj in flat_sample]))


This will fail if we subclass one of the types (say we subclass Feature.Image) but since that's not the intention now, we are OK to keep it as-is.

datumbox · 2022-07-28T13:25:15Z

torchvision/transforms/functional_pil.py

@@ -260,7 +260,7 @@ def _parse_fill(
 ) -> Dict[str, Optional[Union[float, List[float], Tuple[float, ...]]]]:

    # Process fill color for affine transforms
-    num_bands = len(img.getbands())
+    num_bands = get_image_num_channels(img)


Nit: Instead of num_bands use num_channels?

Summary: * [proto] Added few transforms tests, part 1 (#6262) * Added supported/unsupported data checks in the tests for cutmix/mixup * Added RandomRotation, RandomAffine transforms tests * Added tests for RandomZoomOut, Pad * Update test_prototype_transforms.py * Added RandomCrop transform and tests (#6271) * [proto] Added GaussianBlur transform and tests (#6273) * Added GaussianBlur transform and tests * Fixing code format * Copied correctness test * [proto] Added random color transforms and tests (#6275) * Added random color transforms and tests * Disable smoke test for RandomSolarize, RandomAdjustSharpness * Added RandomPerspective and tests (#6284) - replaced real image creation by mocks for other tests * Added more functional tests (#6285) * [proto] Added elastic transform and tests (#6295) * WIP [proto] Added functional elastic transform with tests * Added more functional tests * WIP on elastic op * Added elastic transform and tests * Added tests * Added tests for ElasticTransform * Try to format code as in #5106 * Fixed bug in affine get_params test * Implemented RandomErase on PIL input as fallback to tensors (#6309) Added tests * Added image_size computation for BoundingBox.rotate if expand (#6319) * Added image_size computation for BoundingBox.rotate if expand * Added tests * Added erase_image_pil and eager/jit erase_image_tensor test (#6320) * Updates according to the review Reviewed By: NicolasHug Differential Revision: D38351755 fbshipit-source-id: 4b52b530f93c3dfa92326e01803665cb44003a63 Co-authored-by: Vasilis Vryniotis <[email protected]>

vfdev-5 added 11 commits July 12, 2022 22:50

[proto] Added few transforms tests, part 1 (#6262)

df6918c

* Added supported/unsupported data checks in the tests for cutmix/mixup * Added RandomRotation, RandomAffine transforms tests * Added tests for RandomZoomOut, Pad * Update test_prototype_transforms.py

Added RandomCrop transform and tests (#6271)

615b175

[proto] Added GaussianBlur transform and tests (#6273)

bb2f4e1

* Added GaussianBlur transform and tests * Fixing code format * Copied correctness test

Merge branch 'main' of github.com:pytorch/vision into proto-transform…

5bb8178

…s-api

[proto] Added random color transforms and tests (#6275)

378f3c3

* Added random color transforms and tests * Disable smoke test for RandomSolarize, RandomAdjustSharpness

Added RandomPerspective and tests (#6284)

3a9aca1

- replaced real image creation by mocks for other tests

Added more functional tests (#6285)

794443d

Merge branch 'main' of github.com:pytorch/vision into proto-transform…

247b4e2

…s-api

[proto] Added elastic transform and tests (#6295)

94c7dde

* WIP [proto] Added functional elastic transform with tests * Added more functional tests * WIP on elastic op * Added elastic transform and tests * Added tests * Added tests for ElasticTransform

Try to format code as in #5106

b7ed683

Merge branch 'main' of github.com:pytorch/vision into proto-transform…

bdc9e7c

…s-api

facebook-github-bot added the cla signed label Jul 22, 2022

vfdev-5 requested a review from datumbox July 22, 2022 17:57

vfdev-5 added module: transforms prototype labels Jul 22, 2022

vfdev-5 added 2 commits July 25, 2022 14:09

Fixed bug in affine get_params test

14d221d

Implemented RandomErase on PIL input as fallback to tensors (#6309)

2ee8dca

Added tests

This was referenced Jul 25, 2022

[proto] Draft for Transforms API v2 #6205

Closed

Added RandomAffine to prototype Transforms API #6130

Closed

vfdev-5 and others added 4 commits July 25, 2022 20:19

Merge branch 'main' into proto-transforms-api

a8f970e

Added image_size computation for BoundingBox.rotate if expand (#6319)

23112f8

* Added image_size computation for BoundingBox.rotate if expand * Added tests

Added erase_image_pil and eager/jit erase_image_tensor test (#6320)

2586de6

Merge branch 'main' into proto-transforms-api

1118c85

datumbox reviewed Jul 27, 2022

View reviewed changes

datumbox and others added 3 commits July 27, 2022 15:24

Merge branch 'main' into proto-transforms-api

c05169f

Merge branch 'main' of github.com:pytorch/vision into proto-transform…

72b5fd1

…s-api

Updates according to the review

d226e16

vfdev-5 force-pushed the proto-transforms-api branch from a5a66d7 to d226e16 Compare July 28, 2022 10:51

vfdev-5 requested a review from datumbox July 28, 2022 11:42

datumbox approved these changes Jul 28, 2022

View reviewed changes

vfdev-5 merged commit 77c8c91 into main Jul 28, 2022

vfdev-5 deleted the proto-transforms-api branch July 28, 2022 13:58

datumbox mentioned this pull request Jul 28, 2022

Add CarlaStereo, Kitti2012Stereo, and Kitti2015Stereo datasets for Stereo(#6269) #6311

Merged

This was referenced Aug 19, 2022

Port transforms.RandomCrop to prototype.transforms #5522

Closed

Port transforms.RandomAffine to prototype.transforms #5530

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[proto] Ported all transforms to the new API #6305

[proto] Ported all transforms to the new API #6305

vfdev-5 commented Jul 22, 2022 •

edited

Loading

datumbox left a comment •

edited

Loading

datumbox left a comment

datumbox Jul 28, 2022

datumbox Jul 28, 2022

[proto] Ported all transforms to the new API #6305

[proto] Ported all transforms to the new API #6305

Conversation

vfdev-5 commented Jul 22, 2022 • edited Loading

Description:

What remains

datumbox left a comment • edited Loading

Choose a reason for hiding this comment

datumbox left a comment

Choose a reason for hiding this comment

datumbox Jul 28, 2022

Choose a reason for hiding this comment

datumbox Jul 28, 2022

Choose a reason for hiding this comment

vfdev-5 commented Jul 22, 2022 •

edited

Loading

datumbox left a comment •

edited

Loading