Makefile.PL uses wrong system header path when Perl has been compiled with -Dsysroot

[vrkosk]

Makefile.PL sets include directories like so:

push @h_dirs, split /:/, $ENV{INCLUDE} if $ENV{INCLUDE};
push @h_dirs, split ' ', $Config{libsdirs};
push @h_dirs, qw(/include /usr/include /usr/local/include /usr/include/mach);

Why is it using $Config{libsdirs}? I have built a (relocatable) Perl using -Dsysroot, which means libsdirs is set to:

$ perl -MConfig -E 'say $Config{libsdirs}'
/pkgs64/centos6-sysroot/usr/lib/../lib64

When I run perl Makefile.PL, it emits a compiler warning that causes gettime to be disabled:

Looking for header files and functions...
/usr/include/time.h:37:10: fatal error: bits/types/clock_t.h: No such file or directory
   37 | #include <bits/types/clock_t.h>
      |          ^~~~~~~~~~~~~~~~~~~~~~
compilation terminated.

The reason is, $Config{libsdirs} doesn't contain any system headers from the sysroot, so it's defaulting to the host system's /usr/include. Changing Makefile.PL to $Config{incpth} fixes it. In my Perl, it's:

$ perl -MConfig -E 'say $Config{incpth}'
/pkgs64/gcc/11.3.0-c6/bin/../lib/gcc/x86_64-pc-linux-gnu/11.3.0/include /pkgs64/gcc/11.3.0-c6/bin/../lib/gcc/x86_64-pc-linux-gnu/11.3.0/include-fixed /pkgs64/centos6-sysroot/usr/local/include /pkgs64/gcc/11.3.0-c6/bin/../lib/gcc/../../include /pkgs64/centos6-sysroot/usr/include

Unfortunately, this isn't the end of the story! With the defaults, the link command is:

LD_RUN_PATH="/pkgs64/centos6-sysroot/lib/../lib64" gcc  -shared -O2 --sysroot=/pkgs64/centos6-sysroot -L/pkgs64/centos6-sysroot/usr/local/lib -fstack-protector-strong  FileHandle.o NYTProf.o  -o blib/arch/auto/Devel/NYTProf/NYTProf.so  \
   -lrt   

This looks OK, but I think the LD_RUN_PATH is wrong (and this could be an ExtUtils::MakeMaker problem), because the tests fail immediately with:

Can't load 'Devel-NYTProf-6.14/blib/arch/auto/Devel/NYTProf/NYTProf.so' for module Devel::NYTProf: /pkgs64/centos6-sysroot/lib/../lib64/librt.so.1: undefined symbol: __vdso_clock_gettime, version GLIBC_PRIVATE

I can work around it by clearing LIBS:

PERL_MM_OPT="LIBS= " perl Makefile.PL

Which results in the link command:

gcc  -shared -O2 --sysroot=/pkgs64/centos6-sysroot -L/pkgs64/centos6-sysroot/usr/local/lib -fstack-protector-strong  FileHandle.o NYTProf.o  -o blib/arch/auto/Devel/NYTProf/NYTProf.so  

And now all Devel::NYTProf tests pass. To summarise, $Config{libsdirs} definitely seems wrong, but the -lrt issue could be a fault in my sysroot or Perl compilation.

[vrkosk]

Re the LIBS issue, the workaround is wrong. After replacing $Config{libsdirs} with $Config{incpth}, the solution is:

$ perl Makefile.PL
$ make
$ patchelf --print-rpath blib/arch/auto/Devel/NYTProf/NYTProf.so
/pkgs64/centos6-sysroot/lib64
$ patchelf --set-rpath '' blib/arch/auto/Devel/NYTProf/NYTProf.so
$ make test

Something in the toolchain (GNU ld? ExtUtils::MakeMaker?) hardcodes the rpath to the sysroot when linking against -lrt.

[jkeenan]

Can you supply the complete ./Configure command you are using to build this relocatable perl? (Or, output of perl -V:config_args.)

Also, can you explain why you are using such a relocatable perl as a place in which to do profiling with Devel-NYTProf?

Thanks.

[vrkosk]

It's an unusual setup. I have a sysroot in /pkgs64/centos6-sysroot, which is a bare CentOS 6 system. I have compiled GCC 11 using --sysroot=/pkgs64/centos6-sysroot and installed the binaries in /pkgs64/gcc/11.3.0-c6. The goal is to be able to use a new(er) toolchain while creating binaries that target glibc 2.12 (CentOS 6), without being stuck on running everything in a CentOS 6 VM.

The actual use case is, we are bundling Perl as part of a product. The product doesn't have a fixed install path; the end user can install it in any path. We build a relocatable Perl built using the above sysroot, which means we don't need to relocate anything at install time, and we don't need to worry about glibc compatibility.

We don't need Devel::NYTprof on end user systems, but it's really useful to have it for development (it's a great profiler!). Please treat the bug report as a nice to have improvement than something many people will need.

export COMPILER_PATH=/pkgs64/gcc/11.3.0-c6/bin
export PATH=$COMPILER_PATH:$PATH

export SYSROOT=/pkgs64/centos6-sysroot

./Configure -esdr \
-Dcc=gcc \
-Dsysroot=$SYSROOT \
-Duserelocatableinc \
-Duseithreads \
-Dusemulti \
-Dprefix=$XYZZY/perl_538 \
-Dprivlib=.../../lib \
-Darchlib=.../../lib \
-Dsitelib=.../../lib \
-Dsitearch=.../../lib \
-Dotherlibdirs=.../../site/lib

Here $XYZZY is some arbitrary directory on the build system.