Faulty Python Installation

[RpfR2000]

Hi, I'm trying to install the Python version of clij2-fft to use it for Lucy-Richardson deconvolution. I first built clFFT, I believe successfully -- although I'm not sure how to test this. Then did pip install pyopencl, pip install clij2-fft. No apparent issues. But when I run the richardson_lucy python function, I get the following error. Do you know what might be wrong? Thanks!

Traceback (most recent call last):
File "B:\Jon\Software\image_processing\extract_psf.py", line 31, in
psf=richardson_lucy(im_32, centroids_32, 300, 0.0002)
File "C:\Users\Jon\AppData\Local\Packages\PythonSoftwareFoundation.Python.3.10_qbz5n2kfra8p0\LocalCache\local-packages\Python310\site-packages\clij2fft\richardson_lucy.py", line 52, in richardson_lucy
lib = getlib()
File "C:\Users\Jon\AppData\Local\Packages\PythonSoftwareFoundation.Python.3.10_qbz5n2kfra8p0\LocalCache\local-packages\Python310\site-packages\clij2fft\libs.py", line 29, in getlib
clij2fft=CDLL('clij2fft.dll', mode=RTLD_GLOBAL)
File "C:\Program Files\WindowsApps\PythonSoftwareFoundation.Python.3.10_3.10.3056.0_x64__qbz5n2kfra8p0\lib\ctypes_init_.py", line 374, in init
self._handle = _dlopen(self._name, mode)
FileNotFoundError: Could not find module 'clij2fft.dll' (or one of its dependencies). Try using the full path with constructor syntax.

[bnorthan]

Hi @RpfR2000

You shouldn't have to build clFFT. Even if you are wanting to install a development version you should get clFFT from their repo (I'll have to review the build instructions to make sure it isn't confusing). If you just want to use the Richardson Lucy pip install clij2-fft should be enough (pyopencl is a dependency).

I think the problem is that the install has only been tested and on Conda environments and based on your message you are not using conda.

clFFT.dll and clij2fft.dll have to be somewhere on the current path. In the case of conda they are but in the 'Library/bin' directory underneath your current environment directory. So on my machine, with a 'miniconda3' installation, and an environment called 'decon-dl-with-cellpose' the libs end up here

C:\Users\bnort\miniconda3\envs\decon-dl-with-cellpose\Library\bin

So it seems your python is in a local-cache? Not sure I've seen that before. If you search for 'clij2fft.dll' or 'clFFT.dll' under C:\Users\Jon\AppData\Local\Packages\PythonSoftwareFoundation.Python.3.10_qbz5n2kfra8p0 do you find them? Do you have a 'Library\bin' directory somewhere under your python directory?

What distribution of Python are you using? Do you know where the .dlls that your python needs go? I suppose You could always grab the .dlls from here and put them in a location that is on your system path.

[RpfR2000]

Thanks for the response. Yes, I was able to find the .dll files with the following paths:

%USERPROFILE%\AppData\Local\Packages\PythonSoftwareFoundation.Python.3.10_qbz5n2kfra8p0\LocalCache\local-packages\Library\bin\clij2fft.dll
%USERPROFILE%\AppData\Local\Packages\PythonSoftwareFoundation.Python.3.10_qbz5n2kfra8p0\LocalCache\local-packages\Library\bin\clFFT.dll

I added these to my system path, but I'm still getting the same error when I try to run the code.

[bnorthan]

Hmm. Not sure why it still isn't working. You could try moving the .dlls to a different (simpler) directory like make a directory c:\pythondlls\ and add that to the path. I don't think that's a good long term solution but it could at least help troubleshoot.

Another approach is to move the .dlls into the same directory as libs.py (C:\Users\Jon\AppData\Local\Packages\PythonSoftwareFoundation.Python.3.10_qbz5n2kfra8p0\LocalCache\local-packages\Python310\site-packages\clij2fft)

Could also be that it isn't finding an opencl installation. Do you have cuda installed?

[RpfR2000]

Thanks for bearing with me. I tried your suggestions but the error is persisting -- so I assume it must be that there is some dependency I don't have? I do have CUDA installed and I know it works, as I've used it previously with this computer.

On a side note, but somewhat related -- I noticed you wrote some scripts to perform Richardson-Lucy deconvolution here: https://github.com/True-North-Intelligent-Algorithms/tnia-python/tree/main/tnia/deconvolution. Is there a significant difference between these implementations? I could presumably run these fairly easily on my machine since they have few dependencies (and I've used cupy before).

[bnorthan]

The version of RL in tnia-python is almost the same except it doesn't yet have total variation regularization, (plus it doesn't run on newer macs (because newer macs don't support cuda)) and it isn't as well tested. I guess there is also some dask stuff in clij2fft that isn't in tnia-python yet.

However if you want to give it a try that would be great. Let me know how it goes.

[RpfR2000]

Alright, I used the richardson_lucy_cp function to do PSF distillation and the PSF looks pretty reasonable. However, when I use the PSF to deconvolve an image of an actual specimen (a 3D cluster of bacterial cells), the result looks pretty strange. The slices of the stack seem to be split into 4 equal quadrants (in the XY plane) and there's some weird heterogeneity in the illumination. Maybe you know what the underlying problem is? I have no doubt that the non-circular option would improve the result but I run out of memory with the extra padding operations. If you want, I can share the images. Not sure if this is the best place to discuss.

[bnorthan]

If you are able, share the images on https://forum.image.sc/

[bnorthan]

Also if you are having trouble with memory dask is helpful.

I added an example of how to use richardson_lucy_cp and richardson_lucy_dask here. It requires the 'live' installation (not on pypi yet) but if you manage to run it and get different result then I do let me know.

The dask code can be found here.

A lot of the utilities and the design of the dask approach was done by @acyliq.