Skip to content

[IR] Handle external initializers in subgraphs #2347

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 6 commits into from
May 27, 2025
Merged
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
36 changes: 21 additions & 15 deletions onnxscript/ir/external_data.py
Original file line number Diff line number Diff line change
Expand Up @@ -306,17 +306,20 @@
def load_to_model(model: _core.Model) -> _core.Model:
"""Convert all external model initializers to memory tensors in-place.

All initializers in the main graph and subgraphs are handled.

Args:
model: Model to process.
"""
# TODO(justinchuby): Load attributes and initializers in subgraphs
# TODO(justinchuby): Load tensor attributes in subgraphs
values_to_convert = []
for value in model.graph.initializers.values():
if value.const_value is None:
# Filter out the uninitialized initializer values
continue
if isinstance(value.const_value, _core.ExternalTensor):
values_to_convert.append(value)
for graph in model.graphs():
for value in graph.initializers.values():
if value.const_value is None:
# Filter out the uninitialized initializer values
continue

Check warning on line 320 in onnxscript/ir/external_data.py

View check run for this annotation

Codecov / codecov/patch

onnxscript/ir/external_data.py#L320

Added line #L320 was not covered by tests
if isinstance(value.const_value, _core.ExternalTensor):
values_to_convert.append(value)

Check warning on line 322 in onnxscript/ir/external_data.py

View check run for this annotation

Codecov / codecov/patch

onnxscript/ir/external_data.py#L322

Added line #L322 was not covered by tests
loaded_tensors = convert_tensors_from_external(
[v.const_value for v in values_to_convert] # type: ignore[misc]
)
Expand Down Expand Up @@ -346,6 +349,8 @@
to load the newly saved model, or provide a different external data path that
is not currently referenced by any tensors in the model.

All initializers in the main graph and subgraphs are handled.

Args:
model: Model to process.
base_dir: Path the directory where the ONNX model file is.
Expand All @@ -361,14 +366,15 @@
initializers_to_become_external = []
# Existing external tensors, if below the threshold, should be loaded to memory
initializers_to_load_to_memory = []
for value in model.graph.initializers.values():
if value.const_value is None:
# Filter out the uninitialized initializer values
continue
if value.const_value.nbytes > size_threshold_bytes:
initializers_to_become_external.append(value)
elif isinstance(value.const_value, _core.ExternalTensor):
initializers_to_load_to_memory.append(value)
for graph in model.graphs():
for value in graph.initializers.values():
if value.const_value is None:
# Filter out the uninitialized initializer values
continue

Check warning on line 373 in onnxscript/ir/external_data.py

View check run for this annotation

Codecov / codecov/patch

onnxscript/ir/external_data.py#L373

Added line #L373 was not covered by tests
if value.const_value.nbytes > size_threshold_bytes:
initializers_to_become_external.append(value)
elif isinstance(value.const_value, _core.ExternalTensor):
initializers_to_load_to_memory.append(value)

Check warning on line 377 in onnxscript/ir/external_data.py

View check run for this annotation

Codecov / codecov/patch

onnxscript/ir/external_data.py#L377

Added line #L377 was not covered by tests

# Load to memory first, then convert to external tensors, because
# the existing external tensors may be overwritten by the new external data
Expand Down
Loading