Cannot export Voxtral to Metal backend

[seyeong-han]

Failed to export Voxtral to Metal #15621

Incompatibility Issue

pip list | grep torch
executorch             1.0.0
optimum-executorch     0.1.0
pytorch_tokenizers     1.0.1
torch                  2.9.0
torchao                0.14.0

With these versions of libraries, exporting Voxtral model with "metal" recipe doesn't work.

I found the main reason was that the recipes/metal.py requirement for Metal backend was Executorch v1.1.0, which is not compatible with v1.0 for now.

Error messages:

optimum-cli export executorch \          
  --model "mistralai/Voxtral-Mini-3B-2507" \
  --task "multimodal-text-to-text" \
  --recipe "metal" \
  --dtype bfloat16 \
  --max_seq_len 1024 \
  --output_dir="voxtral"

Traceback (most recent call last):
  File "/Users/younghan/miniconda3/envs/executorch290/bin/optimum-cli", line 7, in <module>
    sys.exit(main())
             ^^^^^^
  File "/Users/younghan/miniconda3/envs/executorch290/lib/python3.12/site-packages/optimum/commands/optimum_cli.py", line 208, in main
    service.run()
  File "/Users/younghan/miniconda3/envs/executorch290/lib/python3.12/site-packages/optimum/commands/export/executorch.py", line 230, in run
    main_export(
  File "/Users/younghan/miniconda3/envs/executorch290/lib/python3.12/site-packages/optimum/exporters/executorch/__main__.py", line 145, in main_export
    return export_to_executorch(
           ^^^^^^^^^^^^^^^^^^^^^
  File "/Users/younghan/miniconda3/envs/executorch290/lib/python3.12/site-packages/optimum/exporters/executorch/convert.py", line 82, in export_to_executorch
    executorch_progs = recipe_func(model, **kwargs)
                       ^^^^^^^^^^^^^^^^^^^^^^^^^^^^
TypeError: 'NoneType' object is not callable

So, I upgraded the executorch to 1.1.0.dev20251105, but got different errors.

pip list | grep torch
executorch                1.1.0.dev20251105
optimum-executorch        0.2.0.dev0
pytorch_tokenizers        1.0.1
torch                     2.10.0.dev20251104
torchao                   0.15.0.dev20251104+cpu
torchaudio                2.10.0.dev20251104
torchvision               0.25.0.dev20251104

Error messages:

optimum-cli export executorch \
  --model "mistralai/Voxtral-Mini-3B-2507" \
  --task "multimodal-text-to-text" \
  --recipe "metal" \
  --dtype bfloat16 \
  --max_seq_len 1024 \
  --output_dir="voxtral"

...

torch._inductor.exc.InductorError: TypeError: collect_unsupported_fallback_kernels.<locals>.generate_c_shim_extern_kernel_call_and_collect_unsupported_kernels() got an unexpected keyword argument 'stack_traces'

[jackzhxng]

cc @manuelcandales can you take a look at this?

[seyeong-han]

Test on Executorch v1.1.0 & torch 2.9.0

1. Download nightly version of executorch

pip install executorch==1.1.0.dev20251105 --extra-index-url "https://download.pytorch.org/whl/nightly/cpu"

pip list | grep torch
executorch                         1.1.0.dev20251105
optimum-executorch        0.1.0
pytorch_tokenizers           1.0.1
torch                                  2.9.0
torchao                              0.14.0

2. Missing libraries

While running the export, I got two missing issues.

optimum-cli export executorch \
  --model "mistralai/Voxtral-Mini-3B-2507" \
  --task "multimodal-text-to-text" \
  --recipe "metal" \
  --dtype bfloat16 \
  --max_seq_len 1024 \
  --output_dir="voxtral"

1. mistral-common

ImportError: 
MistralCommonTokenizer requires the mistral-common library but it was not found in your environment. You can install it with pip: `pip install mistral-common`. Please note that you may need to restart your runtime after installation.

2. librosa

ImportError: 
load_audio_as requires the librosa library. But that was not found in your environment. You can install them with pip:
`pip install librosa`
Please note that you may need to restart your runtime after installation.

3. Final Error

optimum-cli export executorch \
  --model "mistralai/Voxtral-Mini-3B-2507" \
  --task "multimodal-text-to-text" \
  --recipe "metal" \
  --dtype bfloat16 \
  --max_seq_len 1024 \
  --output_dir="voxtral"
Skipping import of cpp extensions due to incompatible torch version 2.9.0 for torchao version 0.14.0         Please see GitHub issue #2919 for more info
W1105 16:35:04.332000 25122 site-packages/torch/distributed/elastic/multiprocessing/redirects.py:29] NOTE: Redirects are currently not supported in Windows or MacOs.
Loading checkpoint shards: 100%|██████████████████████████████████| 2/2 [00:00<00:00,  4.65it/s]
`torch_dtype` is deprecated! Use `dtype` instead!
WARNING:coremltools:scikit-learn version 1.7.1 is not supported. Minimum required version: 0.17. Maximum required version: 1.5.1. Disabling scikit-learn conversion API.
WARNING:coremltools:Torch version 2.9.0 has not been tested with coremltools. You may run into unexpected errors. Torch 2.7.0 is the most recent version that has been tested.
I tokenizers:regex.cpp:27] Registering override fallback regex
Fetching 1 files: 100%|█████████████████████████████████████████| 1/1 [00:00<00:00, 6615.62it/s]
Fetching 1 files: 100%|█████████████████████████████████████████| 1/1 [00:00<00:00, 7037.42it/s]
entering the lowerable parts in MetalBackend.preprocess....
Traceback (most recent call last):
  File "/Users/younghan/miniconda3/envs/executorch290/lib/python3.12/site-packages/torch/utils/_config_module.py", line 372, in __getattr__
    config = self._config[name]
             ~~~~~~~~~~~~^^^^^^
KeyError: 'aot_inductor.link_libtorch'

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/Users/younghan/miniconda3/envs/executorch290/bin/optimum-cli", line 7, in <module>
    sys.exit(main())
             ^^^^^^
  File "/Users/younghan/miniconda3/envs/executorch290/lib/python3.12/site-packages/optimum/commands/optimum_cli.py", line 208, in main
    service.run()
  File "/Users/younghan/miniconda3/envs/executorch290/lib/python3.12/site-packages/optimum/commands/export/executorch.py", line 230, in run
    main_export(
  File "/Users/younghan/miniconda3/envs/executorch290/lib/python3.12/site-packages/optimum/exporters/executorch/__main__.py", line 145, in main_export
    return export_to_executorch(
           ^^^^^^^^^^^^^^^^^^^^^
  File "/Users/younghan/miniconda3/envs/executorch290/lib/python3.12/site-packages/optimum/exporters/executorch/convert.py", line 86, in export_to_executorch
    executorch_progs = recipe_func(model, **kwargs)
                       ^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/younghan/miniconda3/envs/executorch290/lib/python3.12/site-packages/optimum/exporters/executorch/recipes/metal.py", line 129, in export_to_executorch_with_metal
    return _lower_to_executorch(exported_progs, model.metadata)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/younghan/miniconda3/envs/executorch290/lib/python3.12/site-packages/optimum/exporters/executorch/recipes/metal.py", line 97, in _lower_to_executorch
    et_prog = to_edge_transform_and_lower(
              ^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/younghan/miniconda3/envs/executorch290/lib/python3.12/site-packages/executorch/exir/program/_program.py", line 115, in wrapper
    return func(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^
  File "/Users/younghan/miniconda3/envs/executorch290/lib/python3.12/site-packages/executorch/exir/program/_program.py", line 1375, in to_edge_transform_and_lower
    edge_manager = edge_manager.to_backend(method_to_partitioner)
                   ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/younghan/miniconda3/envs/executorch290/lib/python3.12/site-packages/executorch/exir/program/_program.py", line 115, in wrapper
    return func(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^
  File "/Users/younghan/miniconda3/envs/executorch290/lib/python3.12/site-packages/executorch/exir/program/_program.py", line 1677, in to_backend
    new_edge_programs = to_backend(method_to_programs_and_partitioners)
                        ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/younghan/miniconda3/envs/executorch290/lib/python3.12/functools.py", line 912, in wrapper
    return dispatch(args[0].__class__)(*args, **kw)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/younghan/miniconda3/envs/executorch290/lib/python3.12/site-packages/executorch/exir/backend/backend_api.py", line 762, in _
    lower_all_submodules_to_backend(
  File "/Users/younghan/miniconda3/envs/executorch290/lib/python3.12/site-packages/executorch/exir/backend/backend_api.py", line 591, in lower_all_submodules_to_backend
    backend_name_to_subclass[backend_id].preprocess_multimethod(
  File "/Users/younghan/miniconda3/envs/executorch290/lib/python3.12/site-packages/executorch/exir/backend/backend_details.py", line 129, in preprocess_multimethod
    preprocess_result = cls.preprocess(program, compile_spec_for_program)
                        ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/younghan/miniconda3/envs/executorch290/lib/python3.12/site-packages/executorch/backends/apple/metal/metal_backend.py", line 122, in preprocess
    paths = torch._inductor.aot_compile(edge_program_module, tuple(user_input_placeholders), options=options)  # type: ignore[arg-type]
            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/younghan/miniconda3/envs/executorch290/lib/python3.12/site-packages/torch/_inductor/__init__.py", line 310, in aot_compile
    return compile_fx_aot(
           ^^^^^^^^^^^^^^^
  File "/Users/younghan/miniconda3/envs/executorch290/lib/python3.12/site-packages/torch/_inductor/compile_fx.py", line 1941, in compile_fx_aot
    compiled_artifacts = compile_fx(
                         ^^^^^^^^^^^
  File "/Users/younghan/miniconda3/envs/executorch290/lib/python3.12/site-packages/torch/_inductor/compile_fx.py", line 2412, in compile_fx
    with config.patch(config_patches):
         ^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/younghan/miniconda3/envs/executorch290/lib/python3.12/site-packages/torch/utils/_config_module.py", line 682, in __enter__
    prior[key] = config.__getattr__(key)
                 ^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/younghan/miniconda3/envs/executorch290/lib/python3.12/site-packages/torch/utils/_config_module.py", line 404, in __getattr__
    raise AttributeError(f"{self.__name__}.{name} does not exist") from e
AttributeError: torch._inductor.config.aot_inductor.link_libtorch does not exist

Collect Env

PyTorch version: 2.9.0
Is debug build: False
CUDA used to build PyTorch: None
ROCM used to build PyTorch: N/A

OS: macOS 15.7.1 (arm64)
GCC version: Could not collect
Clang version: 17.0.0 (clang-1700.3.19.1)
CMake version: version 4.0.3
Libc version: N/A

Python version: 3.12.12 | packaged by Anaconda, Inc. | (main, Oct 21 2025, 20:07:49) [Clang 20.1.8 ] (64-bit runtime)
Python platform: macOS-15.7.1-arm64-arm-64bit
Is CUDA available: False
CUDA runtime version: No CUDA
CUDA_MODULE_LOADING set to: N/A
GPU models and configuration: No CUDA
Nvidia driver version: No CUDA
cuDNN version: No CUDA
Is XPU available: False
HIP runtime version: N/A
MIOpen runtime version: N/A
Is XNNPACK available: True

CPU:
Apple M3 Pro

Versions of relevant libraries:
[pip3] executorch==1.1.0.dev20251105
[pip3] numpy==2.3.4
[pip3] optimum-executorch==0.1.0
[pip3] pytorch_tokenizers==1.0.1
[pip3] torch==2.9.0
[pip3] torchao==0.14.0
[conda] executorch                1.1.0.dev20251105          pypi_0    pypi
[conda] numpy                     2.3.4                    pypi_0    pypi
[conda] optimum-executorch        0.1.0                    pypi_0    pypi
[conda] pytorch-tokenizers        1.0.1                    pypi_0    pypi
[conda] torch                     2.9.0                    pypi_0    pypi
[conda] torchao                   0.14.0                   pypi_0    pypi

[seyeong-han]

cc. @jackzhxng @manuelcandales

Success!

Conclusion

Currently, executorch, torch and optimum-executorch versions look unstable and are incompatible with each other. I was able to find working compatible versions and you can find them at the bottom of this comment.

Solution

I installed the executorch again with this script, Building for Metal, to install the latest libraries for executorch and optimum-executorch.

Additionally, I should install brew install libomp due to have this error while running optimum-cli export.

ld: library 'omp' not found
clang++: error: linker command failed with exit code 1

Finally, I was able to export the Voxtral model like this.

optimum-cli export executorch \
  --model "mistralai/Voxtral-Mini-3B-2507" \
  --task "multimodal-text-to-text" \
  --recipe "metal" \
  --dtype bfloat16 \
  --max_seq_len 1024 \
  --output_dir="voxtral"
W1105 17:22:17.597000 7514 site-packages/torch/distributed/elastic/multiprocessing/redirects.py:29] NOTE: Redirects are currently not supported in Windows or MacOs.
Loading checkpoint shards: 100%|███████████████████████████████████████████████████████████████████████████████████████████████████| 2/2 [00:00<00:00,  6.93it/s]
`torch_dtype` is deprecated! Use `dtype` instead!
WARNING:coremltools:scikit-learn version 1.7.1 is not supported. Minimum required version: 0.17. Maximum required version: 1.5.1. Disabling scikit-learn conversion API.
WARNING:coremltools:Torch version 2.10.0.dev20251015 has not been tested with coremltools. You may run into unexpected errors. Torch 2.7.0 is the most recent version that has been tested.
I tokenizers:regex.cpp:27] Registering override fallback regex
Fetching 1 files: 100%|██████████████████████████████████████████████████████████████████████████████████████████████████████████| 1/1 [00:00<00:00, 6492.73it/s]
Fetching 1 files: 100%|██████████████████████████████████████████████████████████████████████████████████████████████████████████| 1/1 [00:00<00:00, 6297.75it/s]
entering the lowerable parts in MetalBackend.preprocess....
W1105 17:23:34.672000 7514 site-packages/torch/_inductor/utils.py:1616] Not enough SMs to use max_autotune_gemm mode
entering the lowerable parts in MetalBackend.preprocess....
entering the lowerable parts in MetalBackend.preprocess....

Worked Environment

PyTorch version: 2.10.0.dev20251015
Is debug build: False
CUDA used to build PyTorch: None
ROCM used to build PyTorch: N/A

OS: macOS 15.7.1 (arm64)
GCC version: Could not collect
Clang version: 17.0.0 (clang-1700.3.19.1)
CMake version: version 3.31.6
Libc version: N/A

Python version: 3.12.12 | packaged by Anaconda, Inc. | (main, Oct 21 2025, 20:07:49) [Clang 20.1.8 ] (64-bit runtime)
Python platform: macOS-15.7.1-arm64-arm-64bit
Is CUDA available: False
CUDA runtime version: No CUDA
CUDA_MODULE_LOADING set to: N/A
GPU models and configuration: No CUDA
Nvidia driver version: No CUDA
cuDNN version: No CUDA
Is XPU available: False
HIP runtime version: N/A
MIOpen runtime version: N/A
Is XNNPACK available: True

CPU:
Apple M3 Pro

Versions of relevant libraries:
[pip3] executorch==1.1.0a0+4a75896
[pip3] mypy_extensions==1.1.0
[pip3] numpy==2.3.4
[pip3] optimum-executorch==0.2.0.dev0
[pip3] pytorch_tokenizers==1.0.1
[pip3] torch==2.10.0.dev20251015
[pip3] torchao==0.14.0+git01849b2b1
[pip3] torchaudio==2.8.0.dev20251015
[pip3] torchdata==0.11.0
[pip3] torchsr==1.0.4
[pip3] torchtune==0.6.1
[pip3] torchvision==0.25.0.dev20251015
[conda] executorch                1.1.0a0+4a75896          pypi_0    pypi
[conda] numpy                     2.3.4                    pypi_0    pypi
[conda] optimum-executorch        0.2.0.dev0               pypi_0    pypi
[conda] pytorch-tokenizers        1.0.1                    pypi_0    pypi
[conda] torch                     2.10.0.dev20251015          pypi_0    pypi
[conda] torchao                   0.14.0+git01849b2b1          pypi_0    pypi
[conda] torchaudio                2.8.0.dev20251015          pypi_0    pypi
[conda] torchdata                 0.11.0                   pypi_0    pypi
[conda] torchsr                   1.0.4                    pypi_0    pypi
[conda] torchtune                 0.6.1                    pypi_0    pypi
[conda] torchvision               0.25.0.dev20251015          pypi_0    pypi

[manuelcandales]

The Metal recipe in Optimum-ExecuTorch requires executorch >= 1.1.0.dev20251017
That's why you are running into errors.
I should print an error message when someone tries to use the Metal recipe with the wrong executorch/torch versions.