Openvino backend for Executorch to enable inference on Intel CPUs, GPUs, NPUs

[ynimmaga]

Summary

This PR introduces support for the OpenVINO backend in Executorch, enabling accelerated inference on Intel hardware, including CPU, GPU, and NPU devices. OpenVINO optimizes deep learning model performance by leveraging hardware-specific enhancements. The PR also introduces the OpenVINO quantizer with NNCF (Neural Network Compression Framework) for model optimization. The functionality has been tested on several torchvision and timm models, with plans to test and enable support for additional model types in the future.

Below is a description of the features:

• OpenVINO Backend Integration: The backends/openvino directory includes build scripts, AOT components (partitioner, preprocesser), OpenVINO Quantizer, and runtime backend files that register the OpenVINO backend, manage OpenVINO’s inference engine interactions, including model execution, device-specific optimizations, and backend initialization. It also contains tests for layers and models. See backends/openvino/README.md for usage.

• OpenVINO Examples: The examples/openvino directory provides scripts for AOT optimization, quantization, and C++ executor examples. It includes instructions for optimizing the models, quantizing them, and exporting Executorch programs with OpenVINO optimizations. Refer to examples/openvino/README.md for details.

• E2E Tutorial: Added an end-to-end tutorial in docs/source/build-run-openvino.md.

Test plan

This PR is tested with OpenVINO backend on Intel Core Ultra 7 processors for CPU, GPU, and NPU devices. To run the layer tests and model tests, please refer to backends/openvino/tests/README.md

cc: @yury-gorbachev @alexsu52 @cavusmustafa @daniil-lyakhov @suryasidd @AlexKoff88 @MaximProshin @AlexanderDokuchaev

[kimishpatel]

@ynimmaga
In openvino install process make -j($nproc) fails for me. I am on a linux machine. Here is the paste https://gist.github.com/kimishpatel/5ec3448568c047a3524d49bc1c878ad3

Hi @kimishpatel, just want to confirm if you followed the below steps from the install instructions:
git submodule update --init --recursive sudo ./install_build_dependencies.sh

yes all of those. i was compiling some feedback on them as well, but yes i did all that. I can try to blow build directory and try rebuilding it, but cloning worked fine and build deps also installed ok. it seems there are some compilation errors for some targets. Not all. Target that built fine, via cmake, seems to ahve been installed. look at my gh gist

I was following instructions here https://github.com/ynimmaga/executorch/blob/openvino_backend/backends/openvino/README.md

Could you please try a clean build and also let us know more about the configuration (Linux version, system config etc.)? Here are more details regarding system requirements for OpenVINO: https://docs.openvino.ai/2025/about-openvino/release-notes-openvino/system-requirements.html

This is what I have

NAME="CentOS Stream"
VERSION="9"
ID="centos"
ID_LIKE="rhel fedora"
VERSION_ID="9"
PLATFORM_ID="platform:el9"
PRETTY_NAME="CentOS Stream 9"
ANSI_COLOR="0;31"

It is not ubuntu or other versions suppoted.

[mergennachin]

@ynimmaga thanks for the lint patches. a few still remaining looks like

https://github.com/pytorch/executorch/actions/runs/14099141987/job/39492002002?pr=8573

[suryasidd]

@ynimmaga thanks for the lint patches. a few still remaining looks like

https://github.com/pytorch/executorch/actions/runs/14099141987/job/39492002002?pr=8573

Hey @mergennachin, the remaining issues are because nncf and openvino packages are not installed while running the linter. When the packages are not installed mypy is not able to find them. Should we add an ignore for import-not-found errors ?

[kimishpatel]

https://github.com/pytorch/executorch/actions/runs/14099141987/job/39492002002?pr=8573

there is also sklearn related lint error

[kimishpatel]

@ynimmaga thanks for the lint patches. a few still remaining looks like
https://github.com/pytorch/executorch/actions/runs/14099141987/job/39492002002?pr=8573

Hey @mergennachin, the remaining issues are because nncf and openvino packages are not installed while running the linter. When the packages are not installed mypy is not able to find them. Should we add an ignore for import-not-found errors ?

has this been fixed?

[ynimmaga]

@ynimmaga thanks for the lint patches. a few still remaining looks like
https://github.com/pytorch/executorch/actions/runs/14099141987/job/39492002002?pr=8573

Hey @mergennachin, the remaining issues are because nncf and openvino packages are not installed while running the linter. When the packages are not installed mypy is not able to find them. Should we add an ignore for import-not-found errors ?

has this been fixed?

Hi @kimishpatel, thanks for approving the PR. Yes, this has been fixed with ynimmaga#45

[kimishpatel]

ok landed

[cbilgin]

As fyi, we have a discord channel and have highlighted this PR there too: https://discordapp.com/channels/1334270993966825602/1334274132182827069/1355286663701725356

cc: @ilya-lavrenov, @alexsu52, @AlexKoff88, @suryasidd, @cavusmustafa, @daniil-lyakhov

Thank you for this great contribution!