Skip to content

Use library search paths of compiler for RPATH when building binutils with system compiler + enhance sanity check by running --version for binutils commands #2323

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged

Conversation

Flamefire
Copy link
Contributor

@Flamefire Flamefire commented Jan 27, 2021

(created using eb --new-pr)

Fixes easybuilders/easybuild-easyconfigs#10805
Fixes easybuilders/easybuild-easyconfigs#11988

Example with objdump -x $(which ld.gold) | grep RPATH:

  • prior: $ORIGIN/../lib:/usr/lib:/usr/lib64
  • after: $ORIGIN/../lib:/sw/installed/zlib/1.2.11/lib64:/sw/installed/flex/2.6.4/lib64:/usr/lib/gcc/ppc64le-redhat-linux/4.8.5:/usr/lib64

Maybe we should do this before loading any dependencies? Or is this ok?

This allows compiling with a "system" compiler in a non-standard location
@Flamefire Flamefire changed the title Add sanity check commands to binutils Add sanity check commands to binutils and use library search paths of compiler for rpaths Jan 28, 2021
@krometis
Copy link

I reinstalled binutils-2.35.eb with this PR and with (non-EB) gcc/9.2.0 still loaded

$ eb -r --force --detect-loaded-modules=error --allow-loaded-modules=EasyBuild --include-easyblocks-from-pr 2323 /apps/easybuild/software/infer-skylake/EasyBuild/4.3.2/easybuild/easyconfigs/b/binutils/binutils-2.35.eb

and the linking appears to be fixed:

$ which ld.gold
/apps/easybuild/software/infer-skylake/binutils/2.35/bin/ld.gold
$ ldd $( which ld.gold )
	linux-vdso.so.1 =>  (0x00002aaaaaacd000)
	libdl.so.2 => /lib64/libdl.so.2 (0x00002aaaaaccf000)
	libstdc++.so.6 => /cm/local/apps/gcc/9.2.0/lib64/libstdc++.so.6 (0x00002aaaaaed3000)
	libm.so.6 => /lib64/libm.so.6 (0x00002aaaab2ac000)
	libgcc_s.so.1 => /cm/local/apps/gcc/9.2.0/lib64/libgcc_s.so.1 (0x00002aaaab5ae000)
	libpthread.so.0 => /lib64/libpthread.so.0 (0x00002aaaab7c6000)
	libc.so.6 => /lib64/libc.so.6 (0x00002aaaab9e2000)
	/lib64/ld-linux-x86-64.so.2 (0x00002aaaaaaab000)

I can also confirm that binutils-2.35-GCCcore-10.2.0.eb builds with --include-easyblocks-from-pr 2323, i.e., 2323 doesn’t break non-system toolchain builds, which is nice to confirm.

@Flamefire
Copy link
Contributor Author

Test report by @Flamefire

Overview of tested easyconfigs (in order)

  • SUCCESS binutils-2.26.eb
  • SUCCESS binutils-2.27.eb
  • SUCCESS binutils-2.28.eb
  • SUCCESS binutils-2.30.eb
  • SUCCESS binutils-2.32.eb
  • SUCCESS binutils-2.34.eb
  • SUCCESS binutils-2.34-GCCcore-9.3.0.eb
  • SUCCESS binutils-2.32-GCCcore-9.2.0.eb
  • SUCCESS binutils-2.35-GCCcore-10.2.0.eb
  • SUCCESS binutils-2.32-GCCcore-8.3.0.eb
  • SUCCESS binutils-2.31.1-GCCcore-8.2.0.eb
  • SUCCESS flex-2.5.39.eb
  • SUCCESS binutils-2.25.eb
  • SUCCESS GCCcore-10.1.0.eb
  • SUCCESS GCCcore-7.4.0.eb
  • SUCCESS zlib-1.2.11-GCCcore-10.1.0.eb
  • SUCCESS zlib-1.2.11-GCCcore-7.4.0.eb
  • SUCCESS help2man-1.47.15-GCCcore-10.1.0.eb
  • SUCCESS help2man-1.47.8-GCCcore-7.4.0.eb
  • SUCCESS M4-1.4.18-GCCcore-10.1.0.eb
  • SUCCESS M4-1.4.18-GCCcore-7.4.0.eb
  • SUCCESS Bison-3.6.1-GCCcore-10.1.0.eb
  • SUCCESS Bison-3.2.2-GCCcore-7.4.0.eb
  • SUCCESS flex-2.6.4-GCCcore-10.1.0.eb
  • SUCCESS flex-2.6.4-GCCcore-7.4.0.eb
  • SUCCESS binutils-2.34-GCCcore-10.1.0.eb
  • SUCCESS binutils-2.31.1-GCCcore-7.4.0.eb

Build succeeded for 27 out of 27 (14 easyconfigs in total)
taurusi4244.taurus.hrsk.tu-dresden.de - Linux RHEL 7.9, x86_64, Intel(R) Xeon(R) CPU E5-2680 v4 @ 2.40GHz, Python 3.6.8
See https://gist.github.com/b91a98018f465c0ec9a598858e270f62 for a full test report.

@boegel boegel added this to the next release (4.3.3?) milestone Jan 31, 2021
@boegel boegel added the bug fix label Jan 31, 2021
@boegel boegel changed the title Add sanity check commands to binutils and use library search paths of compiler for rpaths Use library search paths of compiler for RPATH when building binutils with system compiler + enhance sanity check by running --version for binutils commands Jan 31, 2021
@boegel
Copy link
Member

boegel commented Jan 31, 2021

Test report by @boegel

Overview of tested easyconfigs (in order)

  • SUCCESS binutils-2.34.eb
  • SUCCESS binutils-2.34-GCCcore-9.3.0.eb

Build succeeded for 2 out of 2 (2 easyconfigs in total)
node3147.skitty.os - Linux UNKNOWN UNKNOWN, x86_64, Intel(R) Xeon(R) Gold 6140 CPU @ 2.30GHz (skylake_avx512), Python 3.8.6
See https://gist.github.com/70c8c47526d049d3c92b603ab9e9ae75 for a full test report.

edit: tested under --sysroot, looks good, taken from log for binutils-2.34.eb:

== 2021-01-31 10:07:38,564 binutils.py:72 DEBUG Unique library search paths from compiler gcc: ['/cvmfs/pilot.eessi-hpc.org/2020.12/software/x86_64/intel/skylake_avx512/software/Bison/3.5.3/lib/x86_64-pc-linux-gnu/10.2.0', '/cvmfs/pilot.eessi-hpc.org/2020.12/software/x86_64/intel/skylake_avx512/software/Bison/3.5.3/lib64', '/cvmfs/pilot.eessi-hpc.org/2020.12/compat/linux/x86_64/usr/lib/gcc/x86_64-pc-linux-gnu/10.2.0', '/cvmfs/pilot.eessi-hpc.org/2020.12/compat/linux/x86_64/usr/x86_64-pc-linux-gnu/lib/x86_64-pc-linux-gnu/10.2.0', '/cvmfs/pilot.eessi-hpc.org/2020.12/compat/linux/x86_64/usr/x86_64-pc-linux-gnu/lib64', '/cvmfs/pilot.eessi-hpc.org/2020.12/compat/linux/x86_64/usr/lib/x86_64-pc-linux-gnu/10.2.0', '/cvmfs/pilot.eessi-hpc.org/2020.12/compat/linux/x86_64/usr/lib64', '/cvmfs/pilot.eessi-hpc.org/2020.12/compat/linux/x86_64/lib/x86_64-pc-linux-gnu/10.2.0', '/cvmfs/pilot.eessi-hpc.org/2020.12/compat/linux/x86_64/lib64', '/cvmfs/pilot.eessi-hpc.org/2020.12/software/x86_64/intel/skylake_avx512/software/Bison/3.5.3/lib', '/cvmfs/pilot.eessi-hpc.org/2020.12/compat/linux/x86_64/usr/x86_64-pc-linux-gnu/lib', '/cvmfs/pilot.eessi-hpc.org/2020.12/compat/linux/x86_64/usr/lib', '/cvmfs/pilot.eessi-hpc.org/2020.12/compat/linux/x86_64/lib']
== 2021-01-31 10:07:38,565 binutils.py:76 DEBUG Existing library search paths: /cvmfs/pilot.eessi-hpc.org/2020.12/software/x86_64/intel/skylake_avx512/software/Bison/3.5.3/lib64, /cvmfs/pilot.eessi-hpc.org/2020.12/compat/linux/x86_64/usr/lib/gcc/x86_64-pc-linux-gnu/10.2.0, /cvmfs/pilot.eessi-hpc.org/2020.12/compat/linux/x86_64/usr/lib64, /cvmfs/pilot.eessi-hpc.org/2020.12/compat/linux/x86_64/lib64, /cvmfs/pilot.eessi-hpc.org/2020.12/software/x86_64/intel/skylake_avx512/software/Bison/3.5.3/lib, /cvmfs/pilot.eessi-hpc.org/2020.12/compat/linux/x86_64/usr/x86_64-pc-linux-gnu/lib, /cvmfs/pilot.eessi-hpc.org/2020.12/compat/linux/x86_64/usr/lib, /cvmfs/pilot.eessi-hpc.org/2020.12/compat/linux/x86_64/lib
== 2021-01-31 10:07:38,565 binutils.py:83 DEBUG Skipping path with no shared libraries: /cvmfs/pilot.eessi-hpc.org/2020.12/software/x86_64/intel/skylake_avx512/software/Bison/3.5.3/lib64
== 2021-01-31 10:07:38,580 binutils.py:83 DEBUG Skipping path with no shared libraries: /cvmfs/pilot.eessi-hpc.org/2020.12/software/x86_64/intel/skylake_avx512/software/Bison/3.5.3/lib
== 2021-01-31 10:07:38,580 binutils.py:83 DEBUG Skipping path with no shared libraries: /cvmfs/pilot.eessi-hpc.org/2020.12/compat/linux/x86_64/usr/x86_64-pc-linux-gnu/lib
== 2021-01-31 10:07:38,581 binutils.py:83 DEBUG Skipping path with no shared libraries: /cvmfs/pilot.eessi-hpc.org/2020.12/compat/linux/x86_64/lib
== 2021-01-31 10:07:38,581 binutils.py:87 INFO Determined library search paths: /cvmfs/pilot.eessi-hpc.org/2020.12/compat/linux/x86_64/usr/lib/gcc/x86_64-pc-linux-gnu/10.2.0, /cvmfs/pilot.eessi-hpc.org/2020.12/compat/linux/x86_64/usr/lib64, /cvmfs/pilot.eessi-hpc.org/2020.12/compat/linux/x86_64/lib64, /cvmfs/pilot.eessi-hpc.org/2020.12/compat/linux/x86_64/usr/lib
== 2021-01-31 10:07:38,581 environment.py:97 INFO Environment variable LIBS set to -Wl,-rpath='$$ORIGIN/../lib' -Wl,-rpath='/cvmfs/pilot.eessi-hpc.org/2020.12/compat/linux/x86_64/usr/lib/gcc/x86_64-pc-linux-gnu/10.2.0' -Wl,-rpath='/cvmfs/pilot.eessi-hpc.org/2020.12/compat/linux/x86_64/usr/lib64' -Wl,-rpath='/cvmfs/pilot.eessi-hpc.org/2020.12/compat/linux/x86_64/lib64' -Wl,-rpath='/cvmfs/pilot.eessi-hpc.org/2020.12/compat/linux/x86_64/usr/lib' (previously undefined)

@boegel
Copy link
Member

boegel commented Jan 31, 2021

Test report by @boegel

Overview of tested easyconfigs (in order)

  • SUCCESS binutils-2.28.eb
  • SUCCESS binutils-2.30-GCCcore-7.3.0.eb
  • SUCCESS binutils-2.30.eb
  • SUCCESS binutils-2.31.1-GCCcore-8.2.0.eb
  • SUCCESS binutils-2.31.1.eb
  • SUCCESS binutils-2.32-GCCcore-8.3.0.eb
  • SUCCESS binutils-2.32-GCCcore-9.2.0.eb
  • SUCCESS binutils-2.32.eb
  • SUCCESS binutils-2.34-GCCcore-9.3.0.eb
  • SUCCESS binutils-2.34.eb
  • SUCCESS binutils-2.35-GCCcore-10.2.0.eb
  • SUCCESS binutils-2.35.eb

Build succeeded for 12 out of 12 (12 easyconfigs in total)
node3566.doduo.os - Linux RHEL 8.2, x86_64, AMD EPYC 7552 48-Core Processor (zen2), Python 3.6.8
See https://gist.github.com/3d2a03da7a49a681e286f404c0f62985 for a full test report.

@boegel
Copy link
Member

boegel commented Jan 31, 2021

Test report by @boegel

Overview of tested easyconfigs (in order)

  • SUCCESS binutils-2.25.eb
  • SUCCESS binutils-2.26.eb
  • SUCCESS binutils-2.27.eb
  • SUCCESS binutils-2.28.eb
  • SUCCESS binutils-2.29.eb
  • SUCCESS binutils-2.30.eb
  • SUCCESS binutils-2.31.1.eb
  • SUCCESS binutils-2.32.eb
  • SUCCESS binutils-2.34.eb
  • SUCCESS binutils-2.35.eb
  • SUCCESS binutils-2.25-GCCcore-4.9.2.eb
  • SUCCESS binutils-2.25-GCCcore-4.9.3.eb
  • SUCCESS binutils-2.25-GCCcore-4.9.4.eb
  • SUCCESS binutils-2.26-GCCcore-5.3.0.eb
  • SUCCESS binutils-2.26-GCCcore-5.4.0.eb
  • SUCCESS binutils-2.26-GCCcore-5.5.0.eb
  • SUCCESS binutils-2.26-GCCcore-6.3.0.eb
  • SUCCESS binutils-2.27-GCCcore-6.1.0.eb
  • SUCCESS binutils-2.27-GCCcore-6.2.0.eb
  • SUCCESS binutils-2.27-GCCcore-6.3.0.eb
  • SUCCESS binutils-2.28-GCCcore-6.3.0.eb
  • SUCCESS binutils-2.28-GCCcore-6.4.0.eb
  • SUCCESS binutils-2.28-GCCcore-7.1.0.eb
  • SUCCESS binutils-2.29-GCCcore-7.2.0.eb
  • SUCCESS binutils-2.30-GCCcore-7.3.0.eb
  • SUCCESS binutils-2.30-GCCcore-8.1.0.eb
  • SUCCESS binutils-2.31.1-GCCcore-7.4.0.eb
  • SUCCESS binutils-2.31.1-GCCcore-8.2.0.eb
  • SUCCESS binutils-2.32-GCCcore-8.3.0.eb
  • SUCCESS binutils-2.32-GCCcore-9.1.0.eb
  • SUCCESS binutils-2.32-GCCcore-9.2.0.eb
  • SUCCESS binutils-2.34-GCCcore-10.1.0.eb
  • SUCCESS binutils-2.34-GCCcore-9.3.0.eb
  • SUCCESS binutils-2.35-GCCcore-10.2.0.eb

Build succeeded for 34 out of 34 (34 easyconfigs in total)
node2611.swalot.os - Linux centos linux 7.9.2009, x86_64, Intel(R) Xeon(R) CPU E5-2660 v3 @ 2.60GHz (haswell), Python 3.6.8
See https://gist.github.com/4af2456cc242ea7f8697d6662ea4be01 for a full test report.

@boegel
Copy link
Member

boegel commented Jan 31, 2021

Test report by @boegel

Overview of tested easyconfigs (in order)

  • SUCCESS binutils-2.31.1.eb
  • SUCCESS binutils-2.31.1-GCCcore-8.2.0.eb
  • SUCCESS binutils-2.32.eb
  • SUCCESS binutils-2.32-GCCcore-8.3.0.eb
  • SUCCESS binutils-2.34.eb
  • SUCCESS binutils-2.34-GCCcore-9.3.0.eb
  • SUCCESS binutils-2.35.eb
  • SUCCESS binutils-2.35-GCCcore-10.2.0.eb

Build succeeded for 8 out of 8 (8 easyconfigs in total)
easybuild2.novalocal - Linux centos linux 8.3.2011, POWER, IBM pSeries (emulated by qemu) (power9le), Python 3.6.8
See https://gist.github.com/a4cca5313cf81626d5484a8b97ff159f for a full test report.

Copy link
Member

@boegel boegel left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

lgtm

@boegel boegel merged commit 305c5c0 into easybuilders:develop Jan 31, 2021
@boegel
Copy link
Member

boegel commented Jan 31, 2021

Thanks a lot for your efforts on this @Flamefire, and to @krometis for your persistence in helping to track down the underlying cause for easybuilders/easybuild-easyconfigs#10805

# Escaping: Double $$ for Make to get literal $ORIGIN in the file
libdirs = [r'$$ORIGIN/../lib'] + self.determine_used_library_paths()
# https://github.com/easybuilders/easybuild-easyconfigs/issues/10056;
# double $$ to get literal $ORIGIN in the file when command is run
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This comment is actually wrong. The $$ is (still) for make as in the comment above.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The comment (which was there originally) didn't make sense to me, so I tweaked it.

Which make is being referred to here?

Please open an issue or PR to follow up, keeping track of things in closed/merged PRs is close to impossible to keep up with...

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@@ -211,7 +219,7 @@ def sanity_check_step(self):
])

# All binaries support --version, check that they can be run
custom_commands = ['%s --version' % os.path.join(self.installdir, 'bin', b) for b in binaries]
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The abs path was intentional, so those binaries are really used and not anything from the system. But I guess it doesn't matter due to PATH being set?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Right before the sanity check the binutils module is loaded, so we're quite sure it's the correct binary.

The commands run in the sanity check should be run like an end user would.

We can extend the sanity check to verify the location of the binaries via the which function perhaps?

@Flamefire Flamefire deleted the 20210127162337_new_pr_XaolwpgpTl branch February 1, 2021 11:07
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

Successfully merging this pull request may close these issues.

"multiple definition" error in binutils-2.35-GCCcore-10.2.0.eb -fuse-linker-plugin not supported error when building Python with LTO enabled
3 participants