[IR] Handle ONNX custom types in DataType.from_numpy

[justinchuby]

Fixes #1893 where the IR was confused about ONNX custom types. In the long run we should update onnx to use ml_dtypes.

[Copilot]

Pull Request Overview

This PR adds support for handling ONNX custom types in DataType.from_numpy to resolve confusion with custom dtypes. It also expands test coverage by introducing parameterized tests for both standard and custom numpy dtypes.

Reviewed Changes

Copilot reviewed 2 out of 2 changed files in this pull request and generated no comments.

File	Description
onnxscript/ir/_enums.py	Adds special case handling for custom ONNX dtypes based on dtype.names
onnxscript/ir/_enums_test.py	Introduces parameterized tests to cover new custom and standard types

[codecov]

❌ 7 Tests Failed:

Tests completed	Failed	Passed	Skipped
13898	7	13891	1901

View the top 3 failed test(s) by shortest run time

onnxscript.backend.onnx_export_test.TestOnnxBackEnd::test_export2python_produces_correct_onnx_script_model_0040_test_argmax_keepdims_example

Stack Traces | 0.004s run time

onnxscript\backend\onnx_export_test.py:137: in extract_functions
    mod = importlib.import_module(import_name)
C:\hostedtoolcache\windows\Python\3.11.9\x64\Lib\importlib\__init__.py:126: in import_module
    return _bootstrap._gcd_import(name[level:], package, level)
E   ModuleNotFoundError: No module named 'tests.onnx_backend_test_code.test_argmax_keepdims_example'

The above exception was the direct cause of the following exception:
.nox\test\Lib\site-packages\parameterized\parameterized.py:620: in standalone_func
    return func(*(a + p.args), **p.kwargs, **kw)
onnxscript\backend\onnx_export_test.py:271: in test_export2python_produces_correct_onnx_script_model
    functions = extract_functions(backend_test.name, code, self.test_folder)
onnxscript\backend\onnx_export_test.py:139: in extract_functions
    raise AssertionError(
E   AssertionError: Unable to import 'tests.onnx_backend_test_code.test_argmax_keepdims_example' (e=No module named 'tests.onnx_backend_test_code.test_argmax_keepdims_example') (file: 'D:\\a\\onnxscript\\onnxscript\\tests\\onnx_backend_test_code\\test_argmax_keepdims_example.py', absolute path: 'D:\\a\\onnxscript\\onnxscript\\tests\\onnx_backend_test_code\\test_argmax_keepdims_example.py', current folder: D:\a\onnxscript\onnxscript
E   ---- CONTENT --
E   import numpy
E   from onnx import TensorProto
E   from onnx.helper import make_tensor
E   from onnxscript import script, external_tensor
E   from onnxscript.values import Opset
E   from onnxscript.onnx_types import FLOAT, INT64
E   from onnxscript.onnx_opset import opset13
E   
E   @script()
E   def bck_test_argmax_keepdims_example(data: FLOAT[2,2]) -> (INT64[2,1]):
E       result = opset13.ArgMax(data, axis=1, keepdims=1)
E       return result

onnxscript.backend.onnx_export_test.TestOnnxBackEnd::test_export2python_produces_correct_onnx_script_model_0525_test_layer_normalization_4d_axis0

Stack Traces | 0.004s run time

onnxscript\backend\onnx_export_test.py:137: in extract_functions
    mod = importlib.import_module(import_name)
C:\hostedtoolcache\windows\Python\3.11.9\x64\Lib\importlib\__init__.py:126: in import_module
    return _bootstrap._gcd_import(name[level:], package, level)
E   ModuleNotFoundError: No module named 'tests.onnx_backend_test_code.test_layer_normalization_4d_axis0'

The above exception was the direct cause of the following exception:
.nox\test\Lib\site-packages\parameterized\parameterized.py:620: in standalone_func
    return func(*(a + p.args), **p.kwargs, **kw)
onnxscript\backend\onnx_export_test.py:271: in test_export2python_produces_correct_onnx_script_model
    functions = extract_functions(backend_test.name, code, self.test_folder)
onnxscript\backend\onnx_export_test.py:139: in extract_functions
    raise AssertionError(
E   AssertionError: Unable to import 'tests.onnx_backend_test_code.test_layer_normalization_4d_axis0' (e=No module named 'tests.onnx_backend_test_code.test_layer_normalization_4d_axis0') (file: 'D:\\a\\onnxscript\\onnxscript\\tests\\onnx_backend_test_code\\test_layer_normalization_4d_axis0.py', absolute path: 'D:\\a\\onnxscript\\onnxscript\\tests\\onnx_backend_test_code\\test_layer_normalization_4d_axis0.py', current folder: D:\a\onnxscript\onnxscript
E   ---- CONTENT --
E   import numpy
E   from onnx import TensorProto
E   from onnx.helper import make_tensor
E   from onnxscript import script, external_tensor
E   from onnxscript.values import Opset
E   from onnxscript.onnx_types import FLOAT
E   from onnxscript.onnx_opset import opset17
E   
E   @script()
E   def bck_test_layer_normalization_4d_axis0(X: FLOAT[2,3,4,5], W: FLOAT[2,3,4,5], B: FLOAT[2,3,4,5]) -> (FLOAT[2,3,4,5], FLOAT[1,1,1,1], FLOAT[1,1,1,1]):
E       Y, Mean, InvStdDev = opset17.LayerNormalization(X, W, B, axis=0)
E       return Y, Mean, InvStdDev

onnxscript.backend.onnx_export_test.TestOnnxBackEnd::test_export2python_produces_correct_onnx_script_model_1003_test_scatter_elements_with_duplicate_indices

Stack Traces | 0.004s run time

onnxscript\backend\onnx_export_test.py:137: in extract_functions
    mod = importlib.import_module(import_name)
C:\hostedtoolcache\windows\Python\3.11.9\x64\Lib\importlib\__init__.py:126: in import_module
    return _bootstrap._gcd_import(name[level:], package, level)
E   ModuleNotFoundError: No module named 'tests.onnx_backend_test_code.test_scatter_elements_with_duplicate_indices'

The above exception was the direct cause of the following exception:
.nox\test\Lib\site-packages\parameterized\parameterized.py:620: in standalone_func
    return func(*(a + p.args), **p.kwargs, **kw)
onnxscript\backend\onnx_export_test.py:271: in test_export2python_produces_correct_onnx_script_model
    functions = extract_functions(backend_test.name, code, self.test_folder)
onnxscript\backend\onnx_export_test.py:139: in extract_functions
    raise AssertionError(
E   AssertionError: Unable to import 'tests.onnx_backend_test_code.test_scatter_elements_with_duplicate_indices' (e=No module named 'tests.onnx_backend_test_code.test_scatter_elements_with_duplicate_indices') (file: 'D:\\a\\onnxscript\\onnxscript\\tests\\onnx_backend_test_code\\test_scatter_elements_with_duplicate_indices.py', absolute path: 'D:\\a\\onnxscript\\onnxscript\\tests\\onnx_backend_test_code\\test_scatter_elements_with_duplicate_indices.py', current folder: D:\a\onnxscript\onnxscript
E   ---- CONTENT --
E   import numpy
E   from onnx import TensorProto
E   from onnx.helper import make_tensor
E   from onnxscript import script, external_tensor
E   from onnxscript.values import Opset
E   from onnxscript.onnx_types import FLOAT, INT64
E   from onnxscript.onnx_opset import opset18
E   
E   @script()
E   def bck_test_scatter_elements_with_duplicate_indices(data: FLOAT[1,5], indices: INT64[1,2], updates: FLOAT[1,2]) -> (FLOAT[1,5]):
E       y = opset18.ScatterElements(data, indices, updates, axis=1, reduction='add')
E       return y

To view more test analytics, go to the Test Analytics Dashboard
_{📋 Got 3 mins? Take this short survey to help us improve Test Analytics.}

[Copilot]

Pull Request Overview

This PR updates the DataType.from_numpy function to correctly handle ONNX custom types by mapping custom numpy dtypes to the corresponding ONNX data types. Key changes include:

• Adding support for special cases (custom dtypes) based on the dtype’s "names" attribute.
• Updating the tests to use parameterized cases including both standard numpy types and ONNX custom types.
• Adding necessary imports (ml_dtypes, onnx._custom_element_types, parameterized) to support the new test cases.

Reviewed Changes

Copilot reviewed 2 out of 2 changed files in this pull request and generated no comments.

File	Description
onnxscript/ir/_enums.py	Updated from_numpy to support custom ONNX dtypes using dtype.names
onnxscript/ir/_enums_test.py	Refactored tests to use parameterized inputs for broader coverage

Comments suppressed due to low confidence (1)

onnxscript/ir/_enums_test.py:109

• [nitpick] Consider renaming the '_' parameter to a more descriptive name such as 'test_name' to improve readability in the test signature.

def test_from_numpy_takes_np_dtype_and_returns_data_type(self, _: str, np_dtype: np.dtype, onnx_type: _enums.DataType):