cmd/link: does not support multiple TOCs in internal linking mode

[mkumatag]

Please answer these questions before submitting your issue. Thanks!

What version of Go are you using (go version)?

master

root@localhost:~/go# git rev-parse HEAD
eca45997dfd6cd14a59fbdea2385f6648a0dc786
root@localhost:~/go#

Does this issue reproduce with the latest release?

yes

What operating system and processor architecture are you using (go env)?

ppc64le

What did you do?

export PATH=/usr/local/go/bin/:$PATH
export GOROOT_BOOTSTRAP=go env GOROOT

root@localhost:~/go/bin# go version
go version go1.8.3 linux/ppc64le

cloned the latest code here:

root@localhost:~/go/src# pwd
/root/go/src

run make.bash
root@localhost:~/go/src# ./make.bash

Use the build go binary:

root@localhost:~/go/bin# pwd
/root/go/bin
root@localhost:~/go/bin# ./go version
fatal error: unexpected signal during runtime execution
[signal SIGSEGV: segmentation violation code=0x1 addr=0xc88320 pc=0x112c8]

runtime stack:
invalid spdelta runtime/cgo(.text).sigfillset 0x112c0 0x112c8 0x0 -1
runtime.throw(0x44825a, 0x2a)
	/root/go/src/runtime/panic.go:606 +0x68
runtime.sigpanic()
	/root/go/src/runtime/signal_unix.go:360 +0x264
invalid spdelta runtime/cgo(.text).sigfillset 0x112c0 0x112c8 0x0 -1

goroutine 1 [running]:
runtime.systemstack_switch()
	/root/go/src/runtime/asm_ppc64x.s:209 +0x10 fp=0xc420036758 sp=0xc420036738 pc=0x64e40
runtime.main()
	/root/go/src/runtime/proc.go:128 +0x60 fp=0xc4200367c0 sp=0xc420036758 pc=0x3cf80
runtime.goexit()
	/root/go/src/runtime/asm_ppc64x.s:1353 +0x4 fp=0xc4200367c0 sp=0xc4200367c0 pc=0x67464
root@localhost:~/go/bin#

If possible, provide a recipe for reproducing the error.
A complete runnable program is good.
A link on play.golang.org is best.

What did you expect to see?

Instead, I should see a golang version

What did you see instead?

I see a SIGSEGV

[mkumatag]

/cc @laboger

[mkumatag]

@ceseo

[mundaym]

@mkumatag which revision of Go are you using? Can you try rolling back to 101fbc2 (git checkout 101fbc2) and retrying? It looks like the ppc64le builder has been failing since 8e5ac83 (cmd/go: stop linking cgo objects together with ld -r).

[mkumatag]

@mundaym I'm using latest one

root@localhost:~/go# git rev-parse HEAD
eca45997dfd6cd14a59fbdea2385f6648a0dc786
root@localhost:~/go#

[mkumatag]

I rolled back to 101fbc2 and things are fine,

root@localhost:~/go/bin# ./go version
go version devel +101fbc2 Wed Sep 20 20:27:13 2017 +0000 linux/ppc64le
root@localhost:~/go/bin#

@mundaym @ianlancetaylor not really sure what is wrong with the latest version!

[laboger]

Yikes I haven't been watching the build board lately and I see we have been randomly failing in runtime since e7e4a4f and this latest failure of SEGV was introduced with @ianlancetaylor 's change 8e5ac83.

segv happens here:

0x112c0 <runtime/cgo(.text).sigfillset>: std r2,24(r1)
0x112c4 <runtime/cgo(.text).sigfillset+4>: addis r12,r2,100
=> 0x112c8 <runtime/cgo(.text).sigfillset+8>: ld r12,192(r12)
0x112cc <runtime/cgo(.text).sigfillset+12>: mtctr r12
0x112d0 <runtime/cgo(.text).sigfillset+16>: bctr

In golang prior to the change to omit the ld -r where it works, this code looks like this:

=> 0x112c0 <runtime/cgo(.text).sigfillset>: std r2,24(r1)
0x112c4 <runtime/cgo(.text).sigfillset+4>: addis r12,r2,-1
0x112c8 <runtime/cgo(.text).sigfillset+8>: ld r12,32368(r12)
0x112cc <runtime/cgo(.text).sigfillset+12>: mtctr r12
0x112d0 <runtime/cgo(.text).sigfillset+16>: bctr

So my theory is that the TOCs from the individual .o files should be getting merged into a single TOC at some point, but if the external linker isn't involved in the final link, how's it going to happen except by the ld -r? My understanding is that the go tool is linked with the internal linker and not the external linker, so in that case the ld -r is still useful for ppc64le.

This brings up a different concern however, why this wasn't caught in the buildbots?

[mundaym]

You mean the trybots? They don't run on ppc64x, they only check that the compilation works.

[laboger]

FYI... this doesn't affect ppc64 because 'go' is built statically there.

[laboger]

@mundaym This breaks the 'go' tool so I was somewhat thinking that it should fail during the build. But I see now that go_bootstrap is built as static but the final go tool is built as a dynamic binary containing cgo using the internal linker which is the case that is broken by the removal of ld -r.

[mkumatag]

Any updates? builds are still failing with the latest code. @ianlancetaylor any comments?

[laboger]

@ianlancetaylor I don't know what your thoughts are on this. This regression makes the go tool unusable on the golang build so we would like to get this fixed.

I can create a fix to back out your change for ppc64le, i.e., continue to use the ld -r on ppc64le but not on other platforms.

I believe the other option is to fix the internal linker to correctly handle merging of the TOCs, but that might take me a while because I don't know the internal linker well enough.

[ianlancetaylor]

What happens if we simply add sys.PPC64 to this list in cmd/link/internal/ld/config.go?

	if iscgo && SysArch.InFamily(sys.ARM64, sys.MIPS64, sys.MIPS) {
		return true, objabi.GOARCH + " does not support internal cgo"
	}

That seems to me like the right temporary fix unless and until somebody fixes the internal linker. The downside is that we won't generate fully statically linked binaries for pure Go programs on ppc64le. Is that acceptable?

[laboger]

Isn't this going to force external linking only when iscgo is true? So pure Go programs on ppc64le should still be truly statically linked with this change.

I just tried it, and that's what happened with a simple hello.go without cgo, which is pure Go AFAIK?

I think this is the correct temporary fix, but this means the 'go' tool is linked with the external linker and not the internal linker. In both cases it is a dynamic binary. I'm not sure what the downside might be for using a different linker.