Skip to content

Fix failing tests with PyTorch nightly | fix(torchlib) #1185

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 5 commits into from
Nov 27, 2023

Conversation

justinchuby
Copy link
Collaborator

@justinchuby justinchuby commented Nov 27, 2023

Follow up to fix #1061. All pipelines should now be green.

  • Adjusted tolerance for float16 tests that are recently added in PyTorch
  • Fix some dtype issues in function implementations

Copy link

codecov bot commented Nov 27, 2023

Codecov Report

All modified and coverable lines are covered by tests ✅

Comparison is base (72b771c) 78.50% compared to head (c10e88f) 78.63%.

Additional details and impacted files
@@            Coverage Diff             @@
##             main    #1185      +/-   ##
==========================================
+ Coverage   78.50%   78.63%   +0.13%     
==========================================
  Files         118      118              
  Lines       15320    15323       +3     
  Branches     2388     2388              
==========================================
+ Hits        12027    12050      +23     
+ Misses       2903     2878      -25     
- Partials      390      395       +5     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

Copy link

github-actions bot commented Nov 27, 2023

Test Results

         24 files  ±         0         24 suites  ±0   1h 42m 52s ⏱️ + 19m 8s
  11 212 tests +       79    8 347 ✔️ +       38      2 860 💤 +       37  5 +5 
270 080 runs  +71 411  60 699 ✔️ +11 800  209 376 💤 +59 618  5 +5 

For more details on these failures, see this check.

Results for commit c10e88f. ± Comparison against base commit 72b771c.

This pull request removes 5 and adds 84 tests. Note that renamed tests count towards both.
onnxscript.tests.function_libs.torch_lib.ops_test
onnxscript.tests.function_libs.torch_lib.ops_test.TestOutputConsistencyEagerCPU ‑ test_output_match_opinfo__nn_functional_conv3d_cpu_float32
onnxscript.tests.function_libs.torch_lib.ops_test.TestOutputConsistencyEagerCPU ‑ test_output_match_opinfo__nn_functional_conv3d_cpu_int64
onnxscript.tests.function_libs.torch_lib.ops_test.TestOutputConsistencyFullGraphCPU ‑ test_output_match_opinfo__nn_functional_conv3d_cpu_float32
onnxscript.tests.function_libs.torch_lib.ops_test.TestOutputConsistencyFullGraphCPU ‑ test_output_match_opinfo__nn_functional_conv3d_cpu_int64
onnxscript.tests.function_libs.torch_lib.ops_test.TestOutputConsistencyEagerCPU ‑ test_output_match_opinfo__addbmm_cpu_float16
onnxscript.tests.function_libs.torch_lib.ops_test.TestOutputConsistencyEagerCPU ‑ test_output_match_opinfo__addcdiv_cpu_float16
onnxscript.tests.function_libs.torch_lib.ops_test.TestOutputConsistencyEagerCPU ‑ test_output_match_opinfo__addmm_cpu_float16
onnxscript.tests.function_libs.torch_lib.ops_test.TestOutputConsistencyEagerCPU ‑ test_output_match_opinfo__addmm_decomposed_cpu_float16
onnxscript.tests.function_libs.torch_lib.ops_test.TestOutputConsistencyEagerCPU ‑ test_output_match_opinfo__addmv_cpu_float16
onnxscript.tests.function_libs.torch_lib.ops_test.TestOutputConsistencyEagerCPU ‑ test_output_match_opinfo__atan2_cpu_float16
onnxscript.tests.function_libs.torch_lib.ops_test.TestOutputConsistencyEagerCPU ‑ test_output_match_opinfo__baddbmm_cpu_float16
onnxscript.tests.function_libs.torch_lib.ops_test.TestOutputConsistencyEagerCPU ‑ test_output_match_opinfo__bernoulli_cpu_float16
onnxscript.tests.function_libs.torch_lib.ops_test.TestOutputConsistencyEagerCPU ‑ test_output_match_opinfo__bmm_cpu_float16
onnxscript.tests.function_libs.torch_lib.ops_test.TestOutputConsistencyEagerCPU ‑ test_output_match_opinfo__cross_cpu_float16
…
This pull request removes 2 skipped tests and adds 43 skipped tests. Note that renamed tests count towards both.
onnxscript.tests.function_libs.torch_lib.ops_test.TestOutputConsistencyEagerCPU ‑ test_output_match_opinfo__nn_functional_conv3d_cpu_int64
onnxscript.tests.function_libs.torch_lib.ops_test.TestOutputConsistencyFullGraphCPU ‑ test_output_match_opinfo__nn_functional_conv3d_cpu_int64
onnxscript.tests.function_libs.torch_lib.ops_test.TestOutputConsistencyEagerCPU ‑ test_output_match_opinfo__addmm_decomposed_cpu_float16
onnxscript.tests.function_libs.torch_lib.ops_test.TestOutputConsistencyEagerCPU ‑ test_output_match_opinfo__einsum_cpu_float16
onnxscript.tests.function_libs.torch_lib.ops_test.TestOutputConsistencyEagerCPU ‑ test_output_match_opinfo__linspace_tensor_overload_cpu_float16
onnxscript.tests.function_libs.torch_lib.ops_test.TestOutputConsistencyEagerCPU ‑ test_output_match_opinfo__linspace_tensor_overload_cpu_float32
onnxscript.tests.function_libs.torch_lib.ops_test.TestOutputConsistencyEagerCPU ‑ test_output_match_opinfo__linspace_tensor_overload_cpu_int32
onnxscript.tests.function_libs.torch_lib.ops_test.TestOutputConsistencyEagerCPU ‑ test_output_match_opinfo__linspace_tensor_overload_cpu_int64
onnxscript.tests.function_libs.torch_lib.ops_test.TestOutputConsistencyEagerCPU ‑ test_output_match_opinfo__log_softmax_cpu_float16
onnxscript.tests.function_libs.torch_lib.ops_test.TestOutputConsistencyEagerCPU ‑ test_output_match_opinfo__matmul_cpu_float16
onnxscript.tests.function_libs.torch_lib.ops_test.TestOutputConsistencyEagerCPU ‑ test_output_match_opinfo__nn_functional_adaptive_avg_pool1d_cpu_float16
onnxscript.tests.function_libs.torch_lib.ops_test.TestOutputConsistencyEagerCPU ‑ test_output_match_opinfo__nn_functional_adaptive_avg_pool2d_cpu_float16
…

♻️ This comment has been updated with latest results.

Copy link
Contributor

@BowenBao BowenBao left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Changes LG, CI not green yet tho

@justinchuby
Copy link
Collaborator Author

Yes there are still a few but my vpn isn't working :(

@justinchuby justinchuby merged commit 57fa921 into main Nov 27, 2023
@justinchuby justinchuby deleted the justinchu/fix-for-nightly branch November 27, 2023 22:02
@justinchuby justinchuby added the module: torchlib Related to the torch/aten function lib in development label Nov 27, 2023
@titaiwangms
Copy link
Contributor

win pipelines are also expected to be green?

@justinchuby
Copy link
Collaborator Author

Yes. There are some other issues happening

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
module: torchlib Related to the torch/aten function lib in development
Projects
None yet
Development

Successfully merging this pull request may close these issues.

[Broken CI] linspace_tensor tests failing with the latest torch-nightly
3 participants