avoid that path to CUDA stubs library is linked via RPATH to hipSYCL's librt-backend-cuda.so library

[stigrj]

For whatever reason, this change in the cuda.py easyblock triggered an error in the hipSYCL build.

Before this easyblock change the hipSYCL CMake installation reports

-- Set runtime path of "/cluster/software/hipSYCL/0.9.1-gcccuda-2020b/lib/hipSYCL/librt-backend-cuda.so" to ""

which allows it to pick up the correct runtime CUDA library, i.e. the CUDAcore "stub" library when building on a machine without a CUDA driver, but a proper CUDA lib when running on a CUDA machine. After said easyblock change the hipSYCL installation instead reports

-- Set runtime path of "/cluster/software/hipSYCL/0.9.1-gcccuda-2020b/lib/hipSYCL/librt-backend-cuda.so" to "/cluster/software/CUDAcore/11.1.1/lib64/stubs"

which fixes the runtime path to the stub library, even if a proper CUDA lib is present on the machine.

The proposed change will again set the runtime path to empty "" so that the expected behavior is retained. Not sure if the original correct behavior was a result of "two wrongs make a right", but at least the -DWITH_INSTALL_RPATH_USE_LINK_PATH=OFF options seems appropriate here.

[boegel]

@stigrj I doubt this problem is related to the changes in easybuilders/easybuild-easyblocks#2561, but more with the changes in easybuilders/easybuild-easyblocks#2373, which make changes to ensure that the stubs library is considered before the libcuda.so.1 in /lib64 (which is exactly what you don't want here).

It seems like the -DWITH_INSTALL_RPATH_USE_LINK_PATH=OFF that you added doesn't make any difference at all though, I still still a line like this in the log file (on a CentOS 7 system that has /lib64/libcuda.so.1 installed in the OS)

-- Set runtime path of "/software/hipSYCL/0.9.1-gcccuda-2020b/lib/hipSYCL/librt-backend-cuda.so" to "/software/CUDAcore/11.1.1/lib64/stubs"

I also see this:

CMake Warning:
  Manually-specified variables were not used by the project:

    CMAKE_Fortran_COMPILER
    CMAKE_Fortran_FLAGS
    WITH_INSTALL_RPATH_USE_LINK_PATH

So the added configure option is not used at all?

I was also going to propose to add a sanity check command that checks the output produced by ldd lib/hipSYCL/librt-backend-cuda.so to ensure that the CUDA stubs is not picked up, but that's tricky because if there's no libcuda.so.1 installed in the OS, then you'll probably always get the linking to stubs (since there's no other option then)?

[stigrj]

Thanks, @boegel. I tested this successfully on two different HPC systems a month ago, but now trying to reproduce I don't see the same behavior as I remember. I need some time to load the problem back into mental cache

[stigrj]

Ok, there's definitely a typo, should be -DCMAKE_INSTALL_RPATH_USE_LINK_PATH (not -DWITH_INSTALL_...), but I'm still not reproducing the correct behavior...

[stigrj]

Working patch for this issue presented here: #15074

[boegel]

Closing this since gcccuda/2020b is no longer supported, see https://docs.easybuild.io/policies/toolchains

Sorry for not getting back to this @stigrj

If this is still relevant, please consider opening a new pull request using a more recent toolchain