Create env to test with TORCHLIB_EXPERIMENTAL_PREFER_TRACING on | test(torchlib) by justinchuby · Pull Request #1180 · microsoft/onnxscript

justinchuby · 2023-11-22T19:55:46Z

Stack from ghstack (oldest at bottom):

Implement the experimental evaluator for folding branches and castlikes | feat(torchlib) #1178
Mark some functions as traceable | feat(torchlib) #1177
Define the EXPERIMENTAL_PREFER_TRACING flag and the traceable option | feat(torchlib) #1176
-> Create env to test with TORCHLIB_EXPERIMENTAL_PREFER_TRACING on | test(torchlib) #1180

Changes

Create env to test with TORCHLIB_EXPERIMENTAL_PREFER_TRACING on
Test with Python 3.11 as well
Fixes [Broken CI] linspace_tensor tests failing with the latest torch-nightly #1061
Fix aten::any.dims and aten::all.dims

Signed-off-by: Justin Chu <justinchu@microsoft.com> [ghstack-poisoned]

…t(torchlib) [ghstack-poisoned]

…NG on | test(torchlib)" ### Changes - Create env to test with TORCHLIB_EXPERIMENTAL_PREFER_TRACING on - Test with Python 3.11 as well [ghstack-poisoned]

codecov · 2023-11-22T20:02:33Z

Codecov Report

All modified and coverable lines are covered by tests ✅

Comparison is base (77ef131) 78.61% compared to head (55bdb27) 78.64%.

Additional details and impacted files

@@            Coverage Diff             @@
##             main    #1180      +/-   ##
==========================================
+ Coverage   78.61%   78.64%   +0.03%     
==========================================
  Files         118      118              
  Lines       15312    15319       +7     
  Branches     1660     2388     +728     
==========================================
+ Hits        12037    12048      +11     
+ Misses       2889     2877      -12     
- Partials      386      394       +8

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

…NG on | test(torchlib)" ### Changes - Create env to test with TORCHLIB_EXPERIMENTAL_PREFER_TRACING on - Test with Python 3.11 as well [ghstack-poisoned]

…TAL_PREFER_TRACING on | test(torchlib)" ### Changes - Create env to test with TORCHLIB_EXPERIMENTAL_PREFER_TRACING on - Test with Python 3.11 as well [ghstack-poisoned]

…NG on | test(torchlib)" ### Changes - Create env to test with TORCHLIB_EXPERIMENTAL_PREFER_TRACING on - Test with Python 3.11 as well [ghstack-poisoned]

github-actions · 2023-11-22T20:55:51Z

Test Results

        24 files +        6       24 suites +6 1h 32m 51s ⏱️ + 31m 26s
  11 212 tests +      18   8 344 ✔️ +        1     2 855 💤 +      59   13 ❌ -     42
253 800 runs +93 624 56 645 ✔️ +19 902 197 013 💤 +75 632 142 ❌ - 1 910

For more details on these failures, see this check.

Results for commit 55bdb27. ± Comparison against base commit 77ef131.

This pull request removes 469 and adds 487 tests. Note that renamed tests count towards both.

onnxscript.function_libs.tools.torch_lib.deduce_type_constraints_test.TestDeduceTypeConstraints ‑ test_deduce_type_constraints_does_not_crash_for_onnx_function_aten_softmax
onnxscript.function_libs.tools.torch_lib.deduce_type_constraints_test.TestDeduceTypeConstraints ‑ test_deduce_type_constraints_does_not_crash_for_onnx_function_aten_special_log_softmax
onnxscript.tests.function_libs.torch_lib.ops_test.TestFunctionValidity ‑ test_function_has_op_schema_007_aten_allclose
onnxscript.tests.function_libs.torch_lib.ops_test.TestFunctionValidity ‑ test_function_has_op_schema_008_aten_all
onnxscript.tests.function_libs.torch_lib.ops_test.TestFunctionValidity ‑ test_function_has_op_schema_009_aten_abs
onnxscript.tests.function_libs.torch_lib.ops_test.TestFunctionValidity ‑ test_function_has_op_schema_010_aten_abs_complex
onnxscript.tests.function_libs.torch_lib.ops_test.TestFunctionValidity ‑ test_function_has_op_schema_011_aten_acos
onnxscript.tests.function_libs.torch_lib.ops_test.TestFunctionValidity ‑ test_function_has_op_schema_012_aten_acosh
onnxscript.tests.function_libs.torch_lib.ops_test.TestFunctionValidity ‑ test_function_has_op_schema_013_aten_add
onnxscript.tests.function_libs.torch_lib.ops_test.TestFunctionValidity ‑ test_function_has_op_schema_014_aten_add_complex
…

onnxscript.tests.function_libs.torch_lib.ops_test.TestFunctionValidity ‑ test_function_has_op_schema_007_aten_all_dims
onnxscript.tests.function_libs.torch_lib.ops_test.TestFunctionValidity ‑ test_function_has_op_schema_008_aten_allclose
onnxscript.tests.function_libs.torch_lib.ops_test.TestFunctionValidity ‑ test_function_has_op_schema_009_aten_all
onnxscript.tests.function_libs.torch_lib.ops_test.TestFunctionValidity ‑ test_function_has_op_schema_010_aten_abs
onnxscript.tests.function_libs.torch_lib.ops_test.TestFunctionValidity ‑ test_function_has_op_schema_011_aten_abs_complex
onnxscript.tests.function_libs.torch_lib.ops_test.TestFunctionValidity ‑ test_function_has_op_schema_012_aten_acos
onnxscript.tests.function_libs.torch_lib.ops_test.TestFunctionValidity ‑ test_function_has_op_schema_013_aten_acosh
onnxscript.tests.function_libs.torch_lib.ops_test.TestFunctionValidity ‑ test_function_has_op_schema_014_aten_add
onnxscript.tests.function_libs.torch_lib.ops_test.TestFunctionValidity ‑ test_function_has_op_schema_015_aten_add_complex
onnxscript.tests.function_libs.torch_lib.ops_test.TestFunctionValidity ‑ test_function_has_op_schema_016_aten_addbmm
…

This pull request removes 2 skipped tests and adds 20 skipped tests. Note that renamed tests count towards both.

onnxscript.function_libs.tools.torch_lib.deduce_type_constraints_test.TestDeduceTypeConstraints ‑ test_deduce_type_constraints_does_not_crash_for_onnx_function_aten_softmax
onnxscript.function_libs.tools.torch_lib.deduce_type_constraints_test.TestDeduceTypeConstraints ‑ test_deduce_type_constraints_does_not_crash_for_onnx_function_aten_special_log_softmax

onnxscript.tests.function_libs.torch_lib.ops_test.TestOutputConsistencyEagerCPU ‑ test_output_match_opinfo__all_dims_cpu_bool
onnxscript.tests.function_libs.torch_lib.ops_test.TestOutputConsistencyEagerCPU ‑ test_output_match_opinfo__all_dims_cpu_float16
onnxscript.tests.function_libs.torch_lib.ops_test.TestOutputConsistencyEagerCPU ‑ test_output_match_opinfo__all_dims_cpu_float32
onnxscript.tests.function_libs.torch_lib.ops_test.TestOutputConsistencyEagerCPU ‑ test_output_match_opinfo__all_dims_cpu_int32
onnxscript.tests.function_libs.torch_lib.ops_test.TestOutputConsistencyEagerCPU ‑ test_output_match_opinfo__all_dims_cpu_int64
onnxscript.tests.function_libs.torch_lib.ops_test.TestOutputConsistencyEagerCPU ‑ test_output_match_opinfo__any_dims_cpu_bool
onnxscript.tests.function_libs.torch_lib.ops_test.TestOutputConsistencyEagerCPU ‑ test_output_match_opinfo__any_dims_cpu_float16
onnxscript.tests.function_libs.torch_lib.ops_test.TestOutputConsistencyEagerCPU ‑ test_output_match_opinfo__any_dims_cpu_float32
onnxscript.tests.function_libs.torch_lib.ops_test.TestOutputConsistencyEagerCPU ‑ test_output_match_opinfo__any_dims_cpu_int32
onnxscript.tests.function_libs.torch_lib.ops_test.TestOutputConsistencyEagerCPU ‑ test_output_match_opinfo__any_dims_cpu_int64
…

This pull request skips 43 and un-skips 2 tests.

onnxscript.tests.function_libs.torch_lib.ops_test.TestOutputConsistencyEagerCPU ‑ test_output_match_opinfo__all_dim_cpu_bool
onnxscript.tests.function_libs.torch_lib.ops_test.TestOutputConsistencyEagerCPU ‑ test_output_match_opinfo__all_dim_cpu_float16
onnxscript.tests.function_libs.torch_lib.ops_test.TestOutputConsistencyEagerCPU ‑ test_output_match_opinfo__all_dim_cpu_float32
onnxscript.tests.function_libs.torch_lib.ops_test.TestOutputConsistencyEagerCPU ‑ test_output_match_opinfo__all_dim_cpu_int32
onnxscript.tests.function_libs.torch_lib.ops_test.TestOutputConsistencyEagerCPU ‑ test_output_match_opinfo__all_dim_cpu_int64
onnxscript.tests.function_libs.torch_lib.ops_test.TestOutputConsistencyEagerCPU ‑ test_output_match_opinfo__any_dim_cpu_bool
onnxscript.tests.function_libs.torch_lib.ops_test.TestOutputConsistencyEagerCPU ‑ test_output_match_opinfo__any_dim_cpu_float16
onnxscript.tests.function_libs.torch_lib.ops_test.TestOutputConsistencyEagerCPU ‑ test_output_match_opinfo__any_dim_cpu_float32
onnxscript.tests.function_libs.torch_lib.ops_test.TestOutputConsistencyEagerCPU ‑ test_output_match_opinfo__any_dim_cpu_int32
onnxscript.tests.function_libs.torch_lib.ops_test.TestOutputConsistencyEagerCPU ‑ test_output_match_opinfo__any_dim_cpu_int64
…

onnxscript.tests.function_libs.torch_lib.ops_test.TestOutputConsistencyEagerCPU ‑ test_output_match_opinfo__ops_aten__log_softmax_half_cpu_float16
onnxscript.tests.function_libs.torch_lib.ops_test.TestOutputConsistencyEagerCPU ‑ test_output_match_opinfo__ops_aten_native_group_norm_cpu_float16

♻️ This comment has been updated with latest results.

…TAL_PREFER_TRACING on | test(torchlib)" ### Changes - Create env to test with TORCHLIB_EXPERIMENTAL_PREFER_TRACING on - Test with Python 3.11 as well [ghstack-poisoned]

…NG on | test(torchlib)" ### Changes - Create env to test with TORCHLIB_EXPERIMENTAL_PREFER_TRACING on - Test with Python 3.11 as well [ghstack-poisoned]

…TAL_PREFER_TRACING on | test(torchlib)" ### Changes - Create env to test with TORCHLIB_EXPERIMENTAL_PREFER_TRACING on - Test with Python 3.11 as well - Fixes #1061 [ghstack-poisoned]

…NG on | test(torchlib)" ### Changes - Create env to test with TORCHLIB_EXPERIMENTAL_PREFER_TRACING on - Test with Python 3.11 as well - Fixes #1061 [ghstack-poisoned]

…TAL_PREFER_TRACING on | test(torchlib)" ### Changes - Create env to test with TORCHLIB_EXPERIMENTAL_PREFER_TRACING on - Test with Python 3.11 as well - Fixes #1061 [ghstack-poisoned]

…NG on | test(torchlib)" ### Changes - Create env to test with TORCHLIB_EXPERIMENTAL_PREFER_TRACING on - Test with Python 3.11 as well - Fixes #1061 [ghstack-poisoned]

…TAL_PREFER_TRACING on | test(torchlib)" ### Changes - Create env to test with TORCHLIB_EXPERIMENTAL_PREFER_TRACING on - Test with Python 3.11 as well - Fixes #1061 - Fix aten::any.dims and aten::all.dims [ghstack-poisoned]

…NG on | test(torchlib)" ### Changes - Create env to test with TORCHLIB_EXPERIMENTAL_PREFER_TRACING on - Test with Python 3.11 as well - Fixes #1061 - Fix aten::any.dims and aten::all.dims [ghstack-poisoned]

…TAL_PREFER_TRACING on | test(torchlib)" ### Changes - Create env to test with TORCHLIB_EXPERIMENTAL_PREFER_TRACING on - Test with Python 3.11 as well - Fixes #1061 - Fix aten::any.dims and aten::all.dims [ghstack-poisoned]

…NG on | test(torchlib)" ### Changes - Create env to test with TORCHLIB_EXPERIMENTAL_PREFER_TRACING on - Test with Python 3.11 as well - Fixes #1061 - Fix aten::any.dims and aten::all.dims [ghstack-poisoned]

justinchuby added 2 commits November 22, 2023 03:26

Define the EXPERIMENTAL_PREFER_TRACING flag

75551b5

Signed-off-by: Justin Chu <justinchu@microsoft.com> [ghstack-poisoned]

Create env to test with TORCHLIB_EXPERIMENTAL_PREFER_TRACING on | tes…

b513a23

…t(torchlib) [ghstack-poisoned]

justinchuby requested a review from BowenBao November 22, 2023 19:56

justinchuby added the module: torchlib Related to the torch/aten function lib in development label Nov 22, 2023

Update on "Create env to test with TORCHLIB_EXPERIMENTAL_PREFER_TRACI…

581e0b4

…NG on | test(torchlib)" ### Changes - Create env to test with TORCHLIB_EXPERIMENTAL_PREFER_TRACING on - Test with Python 3.11 as well [ghstack-poisoned]

justinchuby added the change base before merge Remember to change the merge base to main when the PR is ready to merge label Nov 22, 2023

justinchuby added 3 commits November 22, 2023 20:10

Update on "Create env to test with TORCHLIB_EXPERIMENTAL_PREFER_TRACI…

ce12b39

…NG on | test(torchlib)" ### Changes - Create env to test with TORCHLIB_EXPERIMENTAL_PREFER_TRACING on - Test with Python 3.11 as well [ghstack-poisoned]

Update base for Update on "Create env to test with TORCHLIB_EXPERIMEN…

e47392f

…TAL_PREFER_TRACING on | test(torchlib)" ### Changes - Create env to test with TORCHLIB_EXPERIMENTAL_PREFER_TRACING on - Test with Python 3.11 as well [ghstack-poisoned]

Update on "Create env to test with TORCHLIB_EXPERIMENTAL_PREFER_TRACI…

3bec3f8

…NG on | test(torchlib)" ### Changes - Create env to test with TORCHLIB_EXPERIMENTAL_PREFER_TRACING on - Test with Python 3.11 as well [ghstack-poisoned]

BowenBao approved these changes Nov 22, 2023

View reviewed changes

justinchuby changed the base branch from gh/justinchuby/48/base to main November 22, 2023 23:47

justinchuby added 10 commits November 22, 2023 23:48

Update base for Update on "Create env to test with TORCHLIB_EXPERIMEN…

1d83826

…TAL_PREFER_TRACING on | test(torchlib)" ### Changes - Create env to test with TORCHLIB_EXPERIMENTAL_PREFER_TRACING on - Test with Python 3.11 as well [ghstack-poisoned]

Update on "Create env to test with TORCHLIB_EXPERIMENTAL_PREFER_TRACI…

0e1b0e6

…NG on | test(torchlib)" ### Changes - Create env to test with TORCHLIB_EXPERIMENTAL_PREFER_TRACING on - Test with Python 3.11 as well [ghstack-poisoned]

Update base for Update on "Create env to test with TORCHLIB_EXPERIMEN…

ae2388e

…TAL_PREFER_TRACING on | test(torchlib)" ### Changes - Create env to test with TORCHLIB_EXPERIMENTAL_PREFER_TRACING on - Test with Python 3.11 as well - Fixes #1061 [ghstack-poisoned]

Update on "Create env to test with TORCHLIB_EXPERIMENTAL_PREFER_TRACI…

ffae748

…NG on | test(torchlib)" ### Changes - Create env to test with TORCHLIB_EXPERIMENTAL_PREFER_TRACING on - Test with Python 3.11 as well - Fixes #1061 [ghstack-poisoned]

Update base for Update on "Create env to test with TORCHLIB_EXPERIMEN…

599b841

…TAL_PREFER_TRACING on | test(torchlib)" ### Changes - Create env to test with TORCHLIB_EXPERIMENTAL_PREFER_TRACING on - Test with Python 3.11 as well - Fixes #1061 [ghstack-poisoned]

Update on "Create env to test with TORCHLIB_EXPERIMENTAL_PREFER_TRACI…

c8692c3

…NG on | test(torchlib)" ### Changes - Create env to test with TORCHLIB_EXPERIMENTAL_PREFER_TRACING on - Test with Python 3.11 as well - Fixes #1061 [ghstack-poisoned]

Update on "Create env to test with TORCHLIB_EXPERIMENTAL_PREFER_TRACI…

8c1525e

…NG on | test(torchlib)" ### Changes - Create env to test with TORCHLIB_EXPERIMENTAL_PREFER_TRACING on - Test with Python 3.11 as well - Fixes #1061 - Fix aten::any.dims and aten::all.dims [ghstack-poisoned]

Update on "Create env to test with TORCHLIB_EXPERIMENTAL_PREFER_TRACI…

55bdb27

…NG on | test(torchlib)" ### Changes - Create env to test with TORCHLIB_EXPERIMENTAL_PREFER_TRACING on - Test with Python 3.11 as well - Fixes #1061 - Fix aten::any.dims and aten::all.dims [ghstack-poisoned]

justinchuby merged commit a89a2a9 into main Nov 23, 2023

justinchuby deleted the gh/justinchuby/48/head branch November 23, 2023 18:25

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Comments

Create env to test with TORCHLIB_EXPERIMENTAL_PREFER_TRACING on | test(torchlib)#1180

Create env to test with TORCHLIB_EXPERIMENTAL_PREFER_TRACING on | test(torchlib)#1180
justinchuby merged 16 commits intomainfrom
gh/justinchuby/48/head

justinchuby commented Nov 22, 2023 •

edited

Loading

Uh oh!

codecov bot commented Nov 22, 2023 •

edited

Loading

Uh oh!

github-actions bot commented Nov 22, 2023 •

edited

Loading

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

Comments

Conversation

justinchuby commented Nov 22, 2023 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Changes

Uh oh!

codecov bot commented Nov 22, 2023 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Codecov Report

Uh oh!

github-actions bot commented Nov 22, 2023 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Test Results

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

justinchuby commented Nov 22, 2023 •

edited

Loading

codecov bot commented Nov 22, 2023 •

edited

Loading

github-actions bot commented Nov 22, 2023 •

edited

Loading