Help for implement CuBLAS

[DanielusG]

Hi, i create a fork of this project in https://github.com/DanielusG/pywhispercpp.git
In this fork, I created commits that allowed whisper to use CuBLAS and thus accelerate whisper using the GPU

It works fine on linux and shows my GPU in use, but when I try to compile on Windows, it compiles without errors, but when I import the library a DLL error comes up. I should have done most of the work, I just need some help getting it to work on Windows.

Even just a suggestion on how I can debug the dll would be fine. I am not an expert in this field so any suggestions would be well appreciated!

The error using my fork on windows:

Traceback (most recent call last):
  File "c:\Users\Ddany\foo\testwhisper.py", line 5, in <module>
    from pywhispercpp.model import Model,Segment
  File "C:\Users\Ddany\foo\venv\lib\site-packages\pywhispercpp\model.py", line 13, in <module>
    import _pywhispercpp as pw
ImportError: DLL load failed while importing _pywhispercpp: Impossibile trovare il modulo specificato.

There are a lot of things hardcoded in the fork, i will fix later, but generally now just clone my fork and build it, it should work

[absadiki]

Hi @DanielusG,

I only use Linux so I cannot help much debug the issue.

But I think the DLL issue is related to the MSVC (Microsoft Visual C and C++) runtime libraries, maybe they are missing.
Please check this link to install them and give it a try!

Otherwise, even if it didn't work on Windows, you can submit a PR and I can merge your changes!

[DanielusG]

Before I open a PR I would like to make it work on windows!

I used a python library called dlltracer to track this kind of error, and it shows that it fails to import some msvc libraries.
Which is strange, since manually searching in the same place as pywhispercpp I find that dll.

Besides, if I compile whisper.cpp with CUDA support it does not require any DLLs and everything works fine on windows.

So it would seem to be a problem related to pybind11. I copied pretty much all the compilation-related stuff for cuBLAS from the whisper.cpp repositories into yours, so it should compile the same dependencies...

Damn windows!!!

[absadiki]

I don't think it is related to pybind11! I think the _pywhispercpp dll is trying to load another dll from those msvc libraries and it failes because it cannot find it.
I guess CUDA does not fail because all of its dlls are in the environment PATH.
so I think this is similar to this issue, you will need to find what DLLs the module is requesting, and you'll need to to add them the PATH!

Damn Windows, Damn! 😅

[DanielusG]

This is very funny. i found with dlltracer what DLLs is missing, but those dlls are already in my PATH, which is system32!!! WHYY, and they was msvc14.dll and other two similar. I found a way to pack them together to _pywhispercpp library, but the problem is the same, using dlltracer again now there are no more dll errors but only when trying to import the _pywhispercpp library the error comes out

Now the error with dlltracer on:

Failed \Device\HarddiskVolume7\Users\Ddany\Desktop\python\test_whisper\venv\Lib\site-packages\_pywhispercpp.cp310-win_amd64.pyd

Traceback (most recent call last):
  File "c:\Users\Ddany\Desktop\python\test_whisper\testwhisper.py", line 6, in <module>
    from pywhispercpp.model import Model,Segment
  File "C:\Users\Ddany\Desktop\python\test_whisper\venv\lib\site-packages\pywhispercpp\model.py", line 13, in <module>
    import _pywhispercpp as pw
ImportError: DLL load failed while importing _pywhispercpp: Impossibile trovare il modulo specificato

i spent my last two days trying to resolve this stupid problem... the crazy thing is that on linux it works like a charm

I tried to open an issue in the whisper.cpp repository, maybe there is an expert on "windows stupidity" willing to help me.

If i found a solution i will open a PR :)

[absadiki]

"windows stupidity" you made my day 😂

If you tried to add all the missing DLLs to the PATH and didn't work, then something weird is happening ..

I didn't even know that whisper.cpp is supporting GPU now .. I hope you will find something who can help
keep me posted.