Accelerating generation with Mediatek NPU

[Georg2001H]

This is certainly not an idea, more of a question... why can't we implement acceleration of photo generation calculations using NPU? I recently read the news and realized that some Snapdragon 8 gen 1.2.3 or Elite processors can accelerate generation using NPU Hexagon with Adreno 7## video chip and higher series. However, Mediatek also has its own NPU. Of course, it is mostly used for generating language models, but the developers say that it can be used for Stable Diffusion calculations.

[rmatif]

Theoretically speaking, it is possible, but practically, we need low-level access (like an API or SDK) to use the NPU hardware, as well as the possibility to write custom kernels and operations for it.

As far as I know, those thing are lacking for Mediatek NPU. For Qualcomm we can use QNN to access the Hexagon NPU, folks from ggml are actively working to add this backend, see here and here. Hopefully it will be usable soon.

There are already some PoC that uses Hexagone NPU to run Stable Diffusion, see Xiaomi implementation or Qualcomm SD1.5/SD2.1

Hope it answers your question !