[proto] Speed up adjust color ops

[vfdev-5]

Description:

• Rewritten _blend op in prototype space
• Simplified adjust_brightness
• Optimized _rgb_to_gray
• Removed redundant checks
• Follow-up PR to [NOMERGE] Add optimized _blend #6765

Results:

[------------------ Adjust_brightness cpu torch.uint8 ------------------]
                     |  adjust_brightness stable  |  adjust_brightness v2
1 threads: --------------------------------------------------------------
      (3, 400, 400)  |            1080            |          483         

Times are in microseconds (us).

[----------------- Adjust_contrast cpu torch.uint8 -----------------]
                     |  adjust_contrast stable  |  adjust_contrast v2
1 threads: ----------------------------------------------------------
      (3, 400, 400)  |           1150           |         926        

Times are in microseconds (us).

[------------------ Adjust_saturation cpu torch.uint8 ------------------]
                     |  adjust_saturation stable  |  adjust_saturation v2
1 threads: --------------------------------------------------------------
      (3, 400, 400)  |            1200            |          935         

Times are in microseconds (us).

[------------------ Adjust_sharpness cpu torch.uint8 -----------------]
                     |  adjust_sharpness stable  |  adjust_sharpness v2
1 threads: ------------------------------------------------------------
      (3, 400, 400)  |            4.06           |          3.64       

Times are in milliseconds (ms).

[----------------- Adjust_brightness cpu torch.float32 -----------------]
                     |  adjust_brightness stable  |  adjust_brightness v2
1 threads: --------------------------------------------------------------
      (3, 400, 400)  |            665             |          206         
6 threads: --------------------------------------------------------------
      (3, 400, 400)  |            167             |           57         

Times are in microseconds (us).

[---------------- Adjust_contrast cpu torch.float32 ----------------]
                     |  adjust_contrast stable  |  adjust_contrast v2
1 threads: ----------------------------------------------------------
      (3, 400, 400)  |           691            |         455        
6 threads: ----------------------------------------------------------
      (3, 400, 400)  |           248            |         162        

Times are in microseconds (us).

[----------------- Adjust_saturation cpu torch.float32 -----------------]
                     |  adjust_saturation stable  |  adjust_saturation v2
1 threads: --------------------------------------------------------------
      (3, 400, 400)  |            728             |          466         

Times are in microseconds (us).

[----------------- Adjust_sharpness cpu torch.float32 ----------------]
                     |  adjust_sharpness stable  |  adjust_sharpness v2
1 threads: ------------------------------------------------------------
      (3, 400, 400)  |            3100           |          2860       

Times are in microseconds (us).

More results in logs, code

Logs with #6765 code + simplified adjust brightness : https://github.com/vfdev-5/tvapiv2_benchmarks/blob/ba35a27f14fd77123ea96f0c54ac9e2fb22f0f15/output/20221018-115341-output-adjust-color-ops_datumbox.log

[pmeier]

One more perf improvement. Otherwise LGTM if CI is green. Thanks Victor!

[pmeier]

Suggested change

	grayscale_image = _rgb_to_gray(image) if c == 3 else image
	mean = torch.mean(grayscale_image.to(dtype), dim=(-3, -2, -1), keepdim=True)
	grayscale_image = _rgb_to_gray(image.to(dtype)) if c == 3 else image.to(dtype)
	mean = torch.mean(grayscale_image, dim=(-3, -2, -1), keepdim=True)

This saves one conversion in _rgb_to_gray in case the input is uint8: _rgb_to_gray would convert the output of its floating point computation back to uint8 just for the result being converted back to floating point in before the torch.mean call.

[vfdev-5]

@pmeier actually, doing so the output of mean is not the same for uint8 image.
As we originally applied uint8 cast in the end of _rgb_to_gray we get rid of all floating point values. This is not the case if image is casted to float before _rgb_to_gray.
Here is an example of difference for mean:

tensor([[[125.9521]]])   # original implementation
# vs
tensor([[[126.4607]]])  # cast to float before `_rgb_to_gray`.

So, finally consistency tests report for example:

Mismatched elements: 256 / 2772 (9.2%)                                                                                                                                      
Greatest absolute difference: 3 at index (1, 2, 4, 21) (up to 1e-05 allowed)   
Greatest relative difference: 0.25 at index (2, 0, 6, 21) (up to 1e-05 allowed)

and this is a real failure, IMO.

[pmeier]

I agree that the behavior changes, but IMO repeatedly converting to uint8 in the computation and thus eliminating intermediate values sounds more like a missed opportunity in the original kernel than a bug now. Thus, I would consider this more like a "bug fix" rather than a BC breaking change. On the other hand, that is not a strong opinion. Not going to block over this.

[datumbox]

LGTM, looks great!

[datumbox]

I like this! Supersedes the work at #6765

[vfdev-5]

This implementation is not consistent with previous behaviour. Valid time measurements are reported in the description.

With the latest commit we reduce runtime for adjust_contrast:

[MiChatz]

Issue:
This clamp_ implementation is enforcing the histogram to take values between 0 and 1 in my case is not working the way I will expect since my images are normalized to values around 0 (ex: -0.2 to 0.8) so the multiplication with the factor of 1 will not return an identical image.

Suggestion:
output = image.mul(brightness_factor).clamp_(image.min(), image.max())

[vfdev-5]

@MiChatz thanks for the feedback. There is (yet unwritten) assumption for color transformations on float images that image range is between [0, 1].