remove cc-ldoption

[nickdesaulniers]

Forked from: ClangBuiltLinux/continuous-integration#122

Some parts of the kernel invoke the linker via $(CC) using -Wl,... flags. This is problematic because setting LD=ld.lld is not enough for this to work correctly.

If compiling with Clang AND linking with LLD, -fuse-ld=lld needs to be added to KBUILD_CFLAGS.

The pixel kernels have a ld-name macro to distinguish between bfd and gold. I've extended them to support lld, but seems the base patch is not upstreamed.

cc @samitolvanen

[nathanchance]

Should be a fairly innocuous change to upstream as it isn't tied to upstreaming LTO support.

At the same time though, do we actually need an ld-name variable right now? Wouldn't this change work?

diff --git a/Makefile b/Makefile
index 141653226f3c..5e9781490ea6 100644
--- a/Makefile
+++ b/Makefile
@@ -500,6 +500,9 @@ ifneq ($(GCC_TOOLCHAIN),)
 CLANG_FLAGS	+= --gcc-toolchain=$(GCC_TOOLCHAIN)
 endif
 CLANG_FLAGS	+= -no-integrated-as
+ifneq ($(shell $(LD) --version 2>&1 | head -n 1 | grep LLD),)
+CLANG_FLAGS	+= -fuse-ld=lld
+endif
 KBUILD_CFLAGS	+= $(CLANG_FLAGS)
 KBUILD_AFLAGS	+= $(CLANG_FLAGS)
 export CLANG_FLAGS

If we do want to introduce something like ld-name, won't we want to mirror the CC_IS_ and _VERSION kconfig variables, since cc-name was been killed off in favor of those?

[nathanchance]

Just FYI, the following diff builds and boots on arm64 in QEMU (I built with V=1 to make sure everything shows up as it should), which deals with #340, #341, this issue, and #343.

diff --git a/Makefile b/Makefile
index 18a02e4c611e..aa1d73084fe6 100644
--- a/Makefile
+++ b/Makefile
@@ -492,7 +492,7 @@ endif
 ifneq ($(shell $(CC) --version 2>&1 | head -n 1 | grep clang),)
 ifneq ($(CROSS_COMPILE),)
 CLANG_FLAGS    := --target=$(notdir $(CROSS_COMPILE:%-=%))
-GCC_TOOLCHAIN_DIR := $(dir $(shell which $(LD)))
+GCC_TOOLCHAIN_DIR := $(dir $(shell which $(AS)))
 CLANG_FLAGS    += --prefix=$(GCC_TOOLCHAIN_DIR)
 GCC_TOOLCHAIN  := $(realpath $(GCC_TOOLCHAIN_DIR)/..)
 endif
@@ -500,6 +500,10 @@ ifneq ($(GCC_TOOLCHAIN),)
 CLANG_FLAGS    += --gcc-toolchain=$(GCC_TOOLCHAIN)
 endif
 CLANG_FLAGS    += -no-integrated-as
+ifneq ($(shell $(LD) --version 2>&1 | head -n 1 | grep LLD),)
+CLANG_FLAGS    += -fuse-ld=lld
+KBUILD_LDFLAGS += -O2
+endif
 KBUILD_CFLAGS  += $(CLANG_FLAGS)
 KBUILD_AFLAGS  += $(CLANG_FLAGS)
 export CLANG_FLAGS
diff --git a/arch/arm64/kernel/vdso/Makefile b/arch/arm64/kernel/vdso/Makefile
index b215c712d897..41a611256476 100644
--- a/arch/arm64/kernel/vdso/Makefile
+++ b/arch/arm64/kernel/vdso/Makefile
@@ -55,7 +55,7 @@ $(obj-vdso): %.o: %.S FORCE
 
 # Actual build commands
 quiet_cmd_vdsold = VDSOL   $@
-      cmd_vdsold = $(CC) $(c_flags) -Wl,-n -Wl,-T $^ -o $@
+      cmd_vdsold = $(CC) $(c_flags) -Wl,-N -Wl,-T $^ -o $@
 quiet_cmd_vdsoas = VDSOA   $@
       cmd_vdsoas = $(CC) $(a_flags) -c -o $@ $<
 

[nickdesaulniers]

@nathanchance , thanks for testing. If you stripped out the arm64 vdso stuff, I'd sign off on the Makefile change. I think there's more to do for arm64 vdso (both on the lld side, and maybe in that vdso's Makefile. If you could add those links to these separate issues, then give Rui (LLD maintainer) and I suggested by tags, that would be appreciated. Otherwise I will do so myself once I get back from FOSDEM/Brussels.

[nathanchance]

I went ahead and tested arm32 as well, which now fails to build with LD=ld.lld: https://gist.github.com/9fffddefc50fa9d22983b59f4813145c

I think something is goofy with cc-option with -fuse-ld=lld.

Without that diff:

/usr/bin/ccache clang -Wp,-MD,arch/arm/vdso/.vgettimeofday.o.d  -nostdinc -isystem /home/nathan/cbl/usr/lib/clang/9.0.0/include -I/home/nathan/cbl/ln/arch/arm/include -I./arch/arm/include/generated  -I/home/nathan/cbl/ln/include -I./include -I/home/nathan/cbl/ln/arch/arm/include/uapi -I./arch/arm/include/generated/uapi -I/home/nathan/cbl/ln/include/uapi -I./include/generated/uapi -include /home/nathan/cbl/ln/include/linux/kconfig.h -include /home/nathan/cbl/ln/include/linux/compiler_types.h  -I/home/nathan/cbl/ln/arch/arm/vdso -Iarch/arm/vdso -D__KERNEL__ -mlittle-endian -Qunused-arguments -Wall -Wundef -Werror=strict-prototypes -Wno-trigraphs -fno-strict-aliasing -fno-common -fshort-wchar -fno-PIE -Werror-implicit-function-declaration -Werror=implicit-int -Wno-format-security -std=gnu89 --target=arm-linux-gnueabi --prefix=/home/nathan/cbl/usr/bin/ --gcc-toolchain=/home/nathan/cbl/usr -no-integrated-as -fno-dwarf2-cfi-asm -mabi=aapcs-linux -mfpu=vfp -funwind-tables -marm -Wa,-mno-warn-deprecated -D__LINUX_ARM_ARCH__=7 -march=armv7-a -msoft-float -Uarm -fno-delete-null-pointer-checks -O2 -fstack-protector-strong -Wno-format-invalid-specifier -Wno-gnu -Wno-address-of-packed-member -Wno-tautological-compare -mno-global-merge -Wno-unused-const-variable -fomit-frame-pointer -Wdeclaration-after-statement -Wvla -Wno-pointer-sign -fno-strict-overflow -fno-merge-all-constants -fno-stack-check -Werror=date-time -Werror=incompatible-pointer-types -Wno-initializer-overrides -Wno-unused-value -Wno-format -Wno-sign-compare -Wno-format-zero-length -Wno-uninitialized -fPIC -fno-common -fno-builtin -fno-stack-protector -DDISABLE_BRANCH_PROFILING -O2    -DKBUILD_BASENAME='"vgettimeofday"' -DKBUILD_MODNAME='"vgettimeofday"' -c -o arch/arm/vdso/vgettimeofday.o /home/nathan/cbl/ln/arch/arm/vdso/vgettimeofday.c

With that diff:

/usr/bin/ccache clang -Wp,-MD,arch/arm/vdso/.vgettimeofday.o.d  -nostdinc -isystem /home/nathan/cbl/usr/lib/clang/9.0.0/include -I/home/nathan/cbl/ln/arch/arm/include -I./arch/arm/include/generated  -I/home/nathan/cbl/ln/include -I./include -I/home/nathan/cbl/ln/arch/arm/include/uapi -I./arch/arm/include/generated/uapi -I/home/nathan/cbl/ln/include/uapi -I./include/generated/uapi -include /home/nathan/cbl/ln/include/linux/kconfig.h -include /home/nathan/cbl/ln/include/linux/compiler_types.h  -I/home/nathan/cbl/ln/arch/arm/vdso -Iarch/arm/vdso -D__KERNEL__ -mlittle-endian -Qunused-arguments -Wall -Wundef -Werror=strict-prototypes -Wno-trigraphs -fno-strict-aliasing -fno-common -fshort-wchar -fno-PIE -Werror-implicit-function-declaration -Werror=implicit-int -Wno-format-security -std=gnu89 --target=arm-linux-gnueabi --prefix=/home/nathan/cbl/usr/bin/ --gcc-toolchain=/home/nathan/cbl/usr -no-integrated-as -fuse-ld=lld -mabi=aapcs-linux -mfpu=vfp -funwind-tables -Wa,-mno-warn-deprecated -D__LINUX_ARM_ARCH__=7 -march=armv5t -Wa,-march=armv7-a -msoft-float -Uarm -O2 -fstack-protector-strong -Wno-format-invalid-specifier -Wno-gnu -Wno-address-of-packed-member -Wno-tautological-compare -mno-global-merge -Wno-unused-const-variable -fomit-frame-pointer -Wdeclaration-after-statement -Wvla -Wno-pointer-sign -fno-strict-overflow -fno-merge-all-constants -fno-stack-check -Werror=date-time -Werror=incompatible-pointer-types -Wno-initializer-overrides -Wno-unused-value -Wno-format -Wno-sign-compare -Wno-format-zero-length -Wno-uninitialized -fPIC -fno-common -fno-builtin -fno-stack-protector -DDISABLE_BRANCH_PROFILING -O2    -DKBUILD_BASENAME='"vgettimeofday"' -DKBUILD_MODNAME='"vgettimeofday"' -c -o arch/arm/vdso/vgettimeofday.o /home/nathan/cbl/ln/arch/arm/vdso/vgettimeofday.c

This particular error is coming from the lack of -march=armv7-a (instead, -march=armv5t -Wa,-march=armv7-a is present), which is cc-option'd:

linux/arch/arm/Makefile

Line 67 in b23b0ea

arch-$(CONFIG_CPU_32v7) =-D__LINUX_ARM_ARCH__=7 $(call cc-option,-march=armv7-a,-march=armv5t -Wa$(comma)-march=armv7-a)

[nathanchance]

There's also this:

linux/arch/arm/vdso/Makefile

Line 18 in b23b0ea

VDSO_LDFLAGS += $(call cc-ldoption, -fuse-ld=bfd)

Introduced by d2b30cd

[dileks]

@nathanchance

Ah great. You had similiar ideas and a use-case/test-case. Makes sense to distinguish on $LD_FOR_BUILD aka ld.bfd, ld.gold and ld.lld.

[nickdesaulniers]

https://lore.kernel.org/r/20190207220155.81351-3-ndesaulniers@google.com/

Nathan, please file a separate bug for arm(32) vdso.

[nathanchance]

Done: #356 and #357.

Also, handy trick from the LLD documentation for verifying if something was linked with LLD:

$ readelf --string-dump .comment /mnt/build/kernel/vmlinux

String dump of section '.comment':
  [     0]  Linker: LLD 9.0.0
  [    12]  clang version 9.0.0 (https://github.com/llvm/llvm-project 5c086c310615873a4e6a83e0981c9e680d695819)