add segmentation reference consistency tests #6591
Conversation
| @@ -873,7 +877,7 @@ def make_datapoints(self, with_mask=True): | |||
|
|
|||
| yield (pil_image, target) | |||
|
|
|||
| tensor_image = torch.randint(0, 256, size=(3, *size), dtype=torch.uint8) | |||
| tensor_image = torch.Tensor(make_image(size=size, color_space=features.ColorSpace.RGB)) | |||
There was a problem hiding this comment.
I missed this before. Let's use the utilities everywhere.
|
|
||
| self.check(t, t_ref) | ||
|
|
||
| def test_random_resize_eval(self): |
There was a problem hiding this comment.
Both this and the test above currently fail. The resized mask is off by quite a bit and so far I don't have an idea why or if that is expected. For the references the resize is performed as PIL image while the prototype transforms uses a features.Mask which internally is treated like a tensor image.
There was a problem hiding this comment.
This happens because PIL and PyTorch don't agree on how exactly the values should be picked during nearest interpolation in case the upsampling is not an integer multiple. For example
from torchvision.prototype.transforms import functional as F
import torch
num_categories = 21
size = [520]
torch.manual_seed(123)
segmentation_mask = torch.randint(0, num_categories, (384, 640), dtype=torch.uint8)
print(segmentation_mask[:3, :3])
a = F.resize_mask(segmentation_mask, size)
print(a[:5, :5])
b = F.to_image_tensor(
F.resize_image_pil(
F.to_image_pil(segmentation_mask),
size,
interpolation=F.InterpolationMode.NEAREST,
)
).squeeze(0)
print(b[:5, :5])
assert a.shape == b.shapetensor([[16, 1, 0, 12],
[ 3, 7, 20, 15],
[12, 19, 11, 12],
[ 2, 20, 6, 9]], dtype=torch.uint8)
tensor([[16, 16, 1, 0, 0],
[16, 16, 1, 0, 0],
[ 3, 3, 7, 20, 20],
[12, 12, 19, 11, 11],
[12, 12, 19, 11, 11]], dtype=torch.uint8)
tensor([[16, 1, 1, 0, 12],
[ 3, 7, 7, 20, 15],
[ 3, 7, 7, 20, 15],
[12, 19, 19, 11, 12],
[ 2, 20, 20, 6, 9]], dtype=torch.uint8)
This might have an effect on the accuracy. So far I don't see a way to change this other than aligning torch.nn.functional.interpolate with what PIL does for nearest interpolation.
| # 1. transforms.RandomCrop uses a different scheme to pad images and masks of insufficient size than its name | ||
| # counterpart in the detection references. Thus, we cannot use it with `pad_if_needed=True` | ||
| # 2. transforms.Pad only supports a fixed padding, but the segmentation datasets don't have a fixed image size. | ||
| class PadIfSmaller(prototype_transforms.Transform): |
There was a problem hiding this comment.
This implements the option suggested in #6433 (comment).
I've also played around with my proposal from #6433 (comment), but it was a lot more complex.
This PR demonstrates that this transform in combination with our transforms.RandomCrop can be used to mimic the behavior of the RandomCrop from the segmentation references. Thus, if we go through with this PR, I will port this transform to #6433.
|
Test failure is unrelated. |
Summary: * add segmentation reference consistency tests * fall back to smoke tests for resize * add test for RandomCrop Reviewed By: YosuaMichael Differential Revision: D39885419 fbshipit-source-id: 12692442af350ca9b97e8764ed429d2d56e0682a Co-authored-by: Vasilis Vryniotis <[email protected]>
Follow-up to #6566 for segmentation references.