Skip to content

Use torch.testing.assert_close in test_models.py #3879

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 34 commits into from
May 24, 2021
Merged
Changes from all commits
Commits
Show all changes
34 commits
Select commit Hold shift + click to select a range
c4fc01b
adopt `torch.testing.assert_close` in test suite
pmeier May 20, 2021
bfbe19b
revert some changes
pmeier May 20, 2021
09f86f4
add todo
pmeier May 20, 2021
86402f0
flake8
pmeier May 20, 2021
48d32e6
Hopefully fixed test_functional_tensor
NicolasHug May 20, 2021
15b50e3
hopefully fixed test_ops
NicolasHug May 20, 2021
a54880f
Merge branch 'master' of github.com:pytorch/vision into assert-close
NicolasHug May 20, 2021
61874ac
Fix test_utils
NicolasHug May 20, 2021
30f20a3
revert unwanted changes to test_image
NicolasHug May 20, 2021
3a29ae3
maybe fixed test_transforms
NicolasHug May 20, 2021
d6d73d0
Merge branch 'master' into assert-close
NicolasHug May 20, 2021
863f144
fix test_datasets_video_utils
pmeier May 21, 2021
c8a5afa
fix test_transforms
pmeier May 21, 2021
e697e88
Merge branch 'master' into assert-close
pmeier May 21, 2021
93614f0
flake8
pmeier May 21, 2021
11caf01
Merge branch 'master' of github.com:pytorch/vision into assert-close
NicolasHug May 21, 2021
d7fde8c
Merge branch 'assert-close' of github.com:pmeier/vision into assert-c…
NicolasHug May 21, 2021
0b237c7
use cu102 see if the nightlies are actual nightlies?
NicolasHug May 21, 2021
c2ace86
obviously forgot to call regenerate.py
NicolasHug May 21, 2021
d78226a
not as obvious, reverting
NicolasHug May 21, 2021
bb543a7
Merge branch 'master' into assert-close
NicolasHug May 21, 2021
7507a0c
Merge branch 'master' into assert-close
NicolasHug May 21, 2021
4f862cf
revert everything but models
NicolasHug May 21, 2021
60b11ac
Merge branch 'master' into assert_close_models
NicolasHug May 21, 2021
18d77de
Merge branch 'master' into assert_close_models
NicolasHug May 21, 2021
f84f48d
Merge branch 'master' into assert_close_models
NicolasHug May 22, 2021
cc7e200
Merge branch 'master' of github.com:pytorch/vision into assert_close_…
NicolasHug May 24, 2021
c5aa55c
don't check device
NicolasHug May 24, 2021
190b947
custom check for detection models?
NicolasHug May 24, 2021
d0ea60e
restore test_models???
NicolasHug May 24, 2021
e7fa11f
Merge branch 'master' of github.com:pytorch/vision into assert_close_…
NicolasHug May 24, 2021
ac5b72d
restore custom check??
NicolasHug May 24, 2021
f89a7d3
put some stuff back, I can't reproduce failures
NicolasHug May 24, 2021
d4a51c8
remove newline
NicolasHug May 24, 2021
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
9 changes: 4 additions & 5 deletions test/test_models.py
Original file line number Diff line number Diff line change
Expand Up @@ -120,7 +120,7 @@ def check_out(out):
# predictions match.
expected_file = self._get_expected_file(name)
expected = torch.load(expected_file)
self.assertEqual(out.argmax(dim=1), expected.argmax(dim=1), prec=prec)
torch.testing.assert_close(out.argmax(dim=1), expected.argmax(dim=1), rtol=prec, atol=prec)
return False # Partial validation performed

return True # Full validation performed
Expand Down Expand Up @@ -205,7 +205,8 @@ def compute_mean_std(tensor):
# scores.
expected_file = self._get_expected_file(name)
expected = torch.load(expected_file)
self.assertEqual(output[0]["scores"], expected[0]["scores"], prec=prec)
torch.testing.assert_close(output[0]["scores"], expected[0]["scores"], rtol=prec, atol=prec,
check_device=False, check_dtype=False)

# Note: Fmassa proposed turning off NMS by adapting the threshold
# and then using the Hungarian algorithm as in DETR to find the
Expand Down Expand Up @@ -301,10 +302,8 @@ def test_memory_efficient_densenet(self):
model2.eval()
out2 = model2(x)

max_diff = (out1 - out2).abs().max()

self.assertTrue(num_params == num_grad)
self.assertTrue(max_diff < 1e-5)
torch.testing.assert_close(out1, out2, rtol=0.0, atol=1e-5)

def test_resnet_dilation(self):
# TODO improve tests to also check that each layer has the right dimensionality
Expand Down