`MLModelBenchmarker.benchmark_predict` fails for models with `ImageType` inputs (unsupported feature type error)

[sisaman]

🐞Describing the bug

MLModelBenchmarker.benchmark_predict fails when the model has an ImageType input. The internal random input generator (_gen_random_inputs) does not correctly support ImageType, leading to a ValueError.

Stack Trace

Traceback (most recent call last):
  File "main.py", line 21, in <module>
    execution_time_measurement = asyncio.run(
                                 ^^^^^^^^^^^^
  File ".local/share/uv/python/cpython-3.12.12-macos-aarch64-none/lib/python3.12/asyncio/runners.py", line 195, in run
    return runner.run(main)
           ^^^^^^^^^^^^^^^^
  File ".local/share/uv/python/cpython-3.12.12-macos-aarch64-none/lib/python3.12/asyncio/runners.py", line 118, in run
    return self._loop.run_until_complete(task)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File ".local/share/uv/python/cpython-3.12.12-macos-aarch64-none/lib/python3.12/asyncio/base_events.py", line 691, in run_until_complete
    return future.result()
           ^^^^^^^^^^^^^^^
  File ".venv/lib/python3.12/site-packages/coremltools/models/ml_program/experimental/perf_utils.py", line 390, in benchmark_predict
    inputs = _gen_random_inputs(model_description=self.model_spec.description)
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File ".venv/lib/python3.12/site-packages/coremltools/models/ml_program/experimental/perf_utils.py", line 138, in _gen_random_inputs
    result[name] = _gen_random_feature_value(type=type)
                   ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File ".venv/lib/python3.12/site-packages/coremltools/models/ml_program/experimental/perf_utils.py", line 126, in _gen_random_feature_value
    raise ValueError(f"Unsupported feature type: {type}")
ValueError: Unsupported feature type: imageType {
  width: 224
  height: 224
  colorSpace: RGB
}

To Reproduce

import asyncio
import coremltools as ct
import torch
import torchvision
from coremltools.models.ml_program.experimental.perf_utils import MLModelBenchmarker

# Get a pytorch model and save it as a *.pt file
model = torchvision.models.mobilenet_v2()
model.eval()
example_input = torch.rand(1, 3, 224, 224)
traced_model = torch.jit.trace(model, example_input)
traced_model.save("torchvision_mobilenet_v2.pt")

# Convert the saved PyTorch model to Core ML
mlmodel = ct.convert("torchvision_mobilenet_v2.pt",
                    inputs=[ct.ImageType(shape=(1, 3, 224, 224))])

# Benchmark
benchmarker = MLModelBenchmarker(model=mlmodel)

execution_time_measurement = asyncio.run(
    benchmarker.benchmark_predict(iterations=1000, warmup=True)
)
print(f"Median execution time: {execution_time_measurement.statistics.median} seconds")

System environment (please complete the following information):

• coremltools version: 9.0
• OS (e.g. MacOS version or Linux type): MacOS 26.4.1
• PyTorch version: 2.11