You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
I followed the instructions in the main README.md for OpenVino on Windows (https://github.com/ggerganov/whisper.cpp?tab=readme-ov-file#openvino-support), but my inference kept happening on the CPU. It worked after I added the "-oved GPU" option based on the description of the associated Pull Request: #1037 where it says:
"You can use -oved [DEVICE] argument to main to specify OpenVINO device to offload encoder inference to. For example:" (followed by an example).
Why is that last line left out of the main README? Is it a good idea to have it there as well?
The text was updated successfully, but these errors were encountered:
Also, the first line in the below logs when running OpenVino based inference says "No GPU found", whereas later it says using the GPU (see device = GPU in the last line below).
whisper_backend_init_gpu: no GPU found
whisper_init_state: kv self size = 6.29 MB
whisper_init_state: kv cross size = 18.87 MB
whisper_init_state: kv pad size = 3.15 MB
whisper_init_state: compute buffer (conv) = 16.26 MB
whisper_init_state: compute buffer (encode) = 85.86 MB
whisper_init_state: compute buffer (cross) = 4.65 MB
whisper_init_state: compute buffer (decode) = 96.35 MB
whisper_ctx_init_openvino_encoder_with_state: loading OpenVINO model from 'models/ggml-base.en-encoder-openvino.xml'
whisper_ctx_init_openvino_encoder_with_state: first run on a device may take a while ...
whisper_openvino_init: path_model = models/ggml-base.en-encoder-openvino.xml, device = GPU, cache_dir = models/ggml-base.en-encoder-openvino-cache
whisper_ctx_init_openvino_encoder_with_state: OpenVINO model loaded
I followed the instructions in the main README.md for OpenVino on Windows (https://github.com/ggerganov/whisper.cpp?tab=readme-ov-file#openvino-support), but my inference kept happening on the CPU. It worked after I added the "-oved GPU" option based on the description of the associated Pull Request: #1037 where it says:
"You can use -oved [DEVICE] argument to main to specify OpenVINO device to offload encoder inference to. For example:" (followed by an example).
Why is that last line left out of the main README? Is it a good idea to have it there as well?
The text was updated successfully, but these errors were encountered: