No such file or directory: '/tmp/pip-build-env-e2vjq07w/lib/python3.6/site-packages/cmake/data/bin/cmake'

[certik]

When cmake is present in pyproject.toml (see PEP 518) as a build dependency, then when I pip install my package, scikit-build fails to launch cmake with the error:

No such file or directory: '/tmp/pip-build-env-e2vjq07w/lib/python3.6/site-packages/cmake/data/bin/cmake'

Full log:

https://gitlab.com/certik/lfortran/-/jobs/214005303

I don't know why that would be.

[certik]

I figured it out. I am using Alpine image, and I needed to install binary packages, not build from source, so I hacked it by:

echo "manylinux1_compatible = True" > /usr/lib/python3.6/site-packages/_manylinux.py

which allows to install cmake and ninja binary wheel packages in Alpine Docker image.

However, the ninja or cmake executable is not actually executable due to the different libc (musl vs glibc) implementation. Linux / Bash should give a much better error message, such as "corrupted executable", but instead it just says "not found":

/tmp $ ls -l ninja 
-rwxr-xr-x    1 root     root        212720 May 16 21:15 ninja
/tmp $ file ninja 
ninja: ELF 64-bit LSB executable, x86-64, version 1 (SYSV), dynamically linked, interpreter /lib64/ld-linux-x86-64.so.2, for GNU/Linux 2.6.9, BuildID[sha1]=613309615a4c5304ce8b43c23fa71bc94ca66be5, stripped
/tmp $ ldd ninja 
	/lib64/ld-linux-x86-64.so.2 (0x7f1f20abe000)
	libstdc++.so.6 => /usr/lib/libstdc++.so.6 (0x7f1f20969000)
	libm.so.6 => /lib64/ld-linux-x86-64.so.2 (0x7f1f20abe000)
	libgcc_s.so.1 => /usr/lib/libgcc_s.so.1 (0x7f1f20955000)
	libc.so.6 => /lib64/ld-linux-x86-64.so.2 (0x7f1f20abe000)
Error relocating ninja: __strtod_internal: symbol not found
/tmp $ ./ninja 
/bin/sh: ./ninja: not found

As you can see, the permissions are correct, one can read the file, only ldd suggests that something is wrong. When you try to execute it, you get a "not found" error, which instead should have been "executable corrupted" or "relocation error" or something like that. This "not found" error then propagates through the Python stack into the error that I reported above, or that you can see in the log, and it was very confusing, as it suggests that the cmake executable didn't properly install (I thought perhaps the Python package got removed somehow under the hood, or something like that), while the answer was simple: everything is installed properly, just the binary is incompatible with Alpine.

Anyway, I am posting the above here in case somebody runs into the same issue. This is not your fault, closing.