`eb git-lfs-3.2.0.eb --rpath` fails on RPATH sanity check

[robogast]

Paraphrasing from a eb slack conversation:

git-lfs depends on Go, and since Go only uses static build, rpath shouldn't be checked at all, even when forced with --rpath

[boegel]

That's something to fix in the GoPackage easyblock, which should override sanity_check_rpath to be a no-op...

@robogast Are you up for trying to make that change yourself?

[robogast]

I mean, yes, but other things are more urgent on my calendar at the moment (looking at you PT2.0), so: at some point? :)

[casparvl]

@robogast you can take a look what was done for the binary easyblock. It's essentially a copy paste :)

But since @bedroge also ran into this, and is into RPATHs as well lately, he might be willing to try it as well...

[bedroge]

I'm no Go expert at all, but it does seem to be possible to link Go-compiled binaries against C libraries (see e.g. https://github.com/lxwagn/using-go-with-c-libraries), in which case you could make use of RPATH. In that case you will have to set CGO_LDFLAGS and inject the right RPATH flags. However, I'm not sure if that could/should always be done when --rpath is used; in that case it would require some more changes in the framework.

Shall I for now just disable the RPATH sanity check?

[casparvl]

Ok, I did a little bit of research for the git-lfs case:

$ ldd /scratch-nvme/1/casparl/generic/software/git-lfs/3.2.0/bin/git-lfs
        linux-vdso.so.1 (0x00007ffd359a7000)
        libpthread.so.0 => /lib64/libpthread.so.0 (0x000014de1ff0b000)
        libc.so.6 => /lib64/libc.so.6 (0x000014de1fb45000)
        /lib64/ld-linux-x86-64.so.2 (0x000014de2012b000)

Clearly, this binary doesn't need an RPATH, as these are libs that always should be picked up from the system.

As Bob mentioned, the only example would be if Go-compiled binaries are linked against C libraries. For this to work would require a change to the rpath_args.py script, specifically, here. From what I understand from go help build, -L is not a valid argument for the go compiler, and neither is -Wl. The way to pass it is probably by adding -ldflags=-rpath /some/path. Thus, the rpath_args.py script would need to be adapted so that it can generate that syntax.

However, at this time, we don't have many GoPackage builds in EasyBuild, and I don't think any of them use external dependencies (they seem to all be at the system toolchain level). I don't think it's worth trying to get the ldflags thing in the RPATH wrapper if we don't have a concrete use case we can experiment with. We could create a toy problem for it (which would also serve as a nice unit test btw). However, I personally wouldn't want to spend the time now.

I'm wondering how we can adapt the rpath_sanity_check step such that we effectively skip it when we're not using external C linkage, but that we'll get an error again if a Go package does use external linkeage... We probably want to unset LD_LIBRARY_PATH, run ldd's and make sure that everything is found, but skip the check for RPATH (i.e. execute this but not this). Current Go packages should pass that test, but a Go package with external linkeage would not. We could put this issue in the raised error, since that would be the right time to look into RPATH wrapper support...

[bedroge]

Unfortunately, the fix in the PR does not solve the issue for git-lfs, since this is using the MakeCp instead of GoPackage easyblock. So we probably need a custom solution for git-lfs?