Skip to content

[IR] Handle external initializers in subgraphs #2347

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 6 commits into from
May 27, 2025

Conversation

justinchuby
Copy link
Collaborator

@justinchuby justinchuby commented May 27, 2025

Support converting external initializers in subgraphs.

@justinchuby justinchuby requested a review from Copilot May 27, 2025 19:31
@justinchuby justinchuby changed the title [IR] Handle initializers in subgraphs [IR] Handle external initializers in subgraphs May 27, 2025
@justinchuby justinchuby requested a review from titaiwangms May 27, 2025 19:31
@justinchuby justinchuby added the module: IR Intermediate representation label May 27, 2025
@justinchuby justinchuby mentioned this pull request May 27, 2025
9 tasks
Copy link
Contributor

@Copilot Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull Request Overview

This PR enhances handling of external initializers by traversing subgraphs, enforces unique initializer names, and adjusts test formatting.

  • Iterate through all subgraphs in both load_to_model and unload_from_model to process initializers.
  • Introduce a validation step to ensure initializer names are unique across the entire model.
  • Remove a stray blank line in the test file before the if __name__ == "__main__" guard.

Reviewed Changes

Copilot reviewed 2 out of 2 changed files in this pull request and generated 3 comments.

File Description
onnxscript/ir/external_data.py Updated loops to include subgraphs, added name uniqueness checks, updated docstrings
onnxscript/ir/external_data_test.py Removed extra blank line before the top-level test runner guard
Comments suppressed due to low confidence (1)

onnxscript/ir/external_data_test.py:500

  • [nitpick] Add an extra blank line before the top-level if __name__ == "__main__" guard to comply with PEP8 style conventions.
if __name__ == "__main__":

Copy link

codecov bot commented May 27, 2025

❌ 23 Tests Failed:

Tests completed Failed Passed Skipped
16754 23 16731 2527
View the top 3 failed test(s) by shortest run time
onnxscript.backend.onnx_export_test.TestOnnxBackEnd::test_export2python_produces_correct_onnx_script_model_0755_test_min_int64
Stack Traces | 0.003s run time
onnxscript\backend\onnx_export_test.py:137: in extract_functions
    mod = importlib.import_module(import_name)
C:\hostedtoolcache\windows\Python\3.11.9\x64\Lib\importlib\__init__.py:126: in import_module
    return _bootstrap._gcd_import(name[level:], package, level)
E   ModuleNotFoundError: No module named 'tests.onnx_backend_test_code.test_min_int64'

The above exception was the direct cause of the following exception:
.nox\test_onnx_weekly\Lib\site-packages\parameterized\parameterized.py:620: in standalone_func
    return func(*(a + p.args), **p.kwargs, **kw)
onnxscript\backend\onnx_export_test.py:271: in test_export2python_produces_correct_onnx_script_model
    functions = extract_functions(backend_test.name, code, self.test_folder)
onnxscript\backend\onnx_export_test.py:139: in extract_functions
    raise AssertionError(
E   AssertionError: Unable to import 'tests.onnx_backend_test_code.test_min_int64' (e=No module named 'tests.onnx_backend_test_code.test_min_int64') (file: 'D:\\a\\onnxscript\\onnxscript\\tests\\onnx_backend_test_code\\test_min_int64.py', absolute path: 'D:\\a\\onnxscript\\onnxscript\\tests\\onnx_backend_test_code\\test_min_int64.py', current folder: D:\a\onnxscript\onnxscript
E   ---- CONTENT --
E   import numpy
E   from onnx import TensorProto
E   from onnx.helper import make_tensor
E   from onnxscript import script, external_tensor
E   from onnxscript.values import Opset
E   from onnxscript.onnx_types import INT64
E   from onnxscript.onnx_opset import opset13
E   
E   @script()
E   def bck_test_min_int64(data_0: INT64[3], data_1: INT64[3]) -> (INT64[3]):
E       result = opset13.Min(data_0, data_1)
E       return result
onnxscript.backend.onnx_export_test.TestOnnxBackEnd::test_export2python_produces_correct_onnx_script_model_0340_test_dropout_default_old
Stack Traces | 0.004s run time
onnxscript\backend\onnx_export_test.py:137: in extract_functions
    mod = importlib.import_module(import_name)
C:\hostedtoolcache\windows\Python\3.12.10\x64\Lib\importlib\__init__.py:90: in import_module
    return _bootstrap._gcd_import(name[level:], package, level)
E   ModuleNotFoundError: No module named 'tests.onnx_backend_test_code.test_dropout_default_old'

The above exception was the direct cause of the following exception:
.nox\test\Lib\site-packages\parameterized\parameterized.py:620: in standalone_func
    return func(*(a + p.args), **p.kwargs, **kw)
onnxscript\backend\onnx_export_test.py:271: in test_export2python_produces_correct_onnx_script_model
    functions = extract_functions(backend_test.name, code, self.test_folder)
onnxscript\backend\onnx_export_test.py:139: in extract_functions
    raise AssertionError(
E   AssertionError: Unable to import 'tests.onnx_backend_test_code.test_dropout_default_old' (e=No module named 'tests.onnx_backend_test_code.test_dropout_default_old') (file: 'D:\\a\\onnxscript\\onnxscript\\tests\\onnx_backend_test_code\\test_dropout_default_old.py', absolute path: 'D:\\a\\onnxscript\\onnxscript\\tests\\onnx_backend_test_code\\test_dropout_default_old.py', current folder: D:\a\onnxscript\onnxscript
E   ---- CONTENT --
E   import numpy
E   from onnx import TensorProto
E   from onnx.helper import make_tensor
E   from onnxscript import script, external_tensor
E   from onnxscript.values import Opset
E   from onnxscript.onnx_types import FLOAT
E   from onnxscript.onnx_opset import opset11
E   
E   @script()
E   def bck_test_dropout_default_old(x: FLOAT[3]) -> (FLOAT[3]):
E       y = opset11.Dropout(x)
E       return y
onnxscript.backend.onnx_export_test.TestOnnxBackEnd::test_export2python_produces_correct_onnx_script_model_0402_test_gemm_all_attributes
Stack Traces | 0.004s run time
onnxscript\backend\onnx_export_test.py:137: in extract_functions
    mod = importlib.import_module(import_name)
C:\hostedtoolcache\windows\Python\3.12.10\x64\Lib\importlib\__init__.py:90: in import_module
    return _bootstrap._gcd_import(name[level:], package, level)
E   ModuleNotFoundError: No module named 'tests.onnx_backend_test_code.test_gemm_all_attributes'

The above exception was the direct cause of the following exception:
.nox\test\Lib\site-packages\parameterized\parameterized.py:620: in standalone_func
    return func(*(a + p.args), **p.kwargs, **kw)
onnxscript\backend\onnx_export_test.py:271: in test_export2python_produces_correct_onnx_script_model
    functions = extract_functions(backend_test.name, code, self.test_folder)
onnxscript\backend\onnx_export_test.py:139: in extract_functions
    raise AssertionError(
E   AssertionError: Unable to import 'tests.onnx_backend_test_code.test_gemm_all_attributes' (e=No module named 'tests.onnx_backend_test_code.test_gemm_all_attributes') (file: 'D:\\a\\onnxscript\\onnxscript\\tests\\onnx_backend_test_code\\test_gemm_all_attributes.py', absolute path: 'D:\\a\\onnxscript\\onnxscript\\tests\\onnx_backend_test_code\\test_gemm_all_attributes.py', current folder: D:\a\onnxscript\onnxscript
E   ---- CONTENT --
E   import numpy
E   from onnx import TensorProto
E   from onnx.helper import make_tensor
E   from onnxscript import script, external_tensor
E   from onnxscript.values import Opset
E   from onnxscript.onnx_types import FLOAT
E   from onnxscript.onnx_opset import opset13
E   
E   @script()
E   def bck_test_gemm_all_attributes(a: FLOAT[4,3], b: FLOAT[5,4], c: FLOAT[1,5]) -> (FLOAT[3,5]):
E       y = opset13.Gemm(a, b, c, alpha=0.25, beta=0.3499999940395355, transA=1, transB=1)
E       return y

To view more test analytics, go to the Test Analytics Dashboard
📋 Got 3 mins? Take this short survey to help us improve Test Analytics.

@justinchuby justinchuby merged commit f7c3f5a into main May 27, 2025
24 of 29 checks passed
@justinchuby justinchuby deleted the justinchu/subgraph-external branch May 27, 2025 23:36
@justinchuby
Copy link
Collaborator Author

I merged prematurely. Will add test cases.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
module: IR Intermediate representation
Projects
Development

Successfully merging this pull request may close these issues.

2 participants