runtime, internal/cpu: internal compiler error: Ctz64 & Initialize when bootstrapping a build

[0paIescent]

What version of Go are you using (go version)?

go version go1.12.9 linux/i686

Does this issue reproduce with the latest release?

Yes

What operating system and processor architecture are you using (go env)?

Ubuntu 18.04 i686

What did you do?

Submitted a PR to void-linux/void-packages, and the i686 travis build panicked twice.

What did you expect to see?

Done. Your build exited with 0.

What did you see

Two panics, one of which was while compiling Ctz64, and the other was while compiling Initialize.

[randall77]

That stack trace doesn't make much sense to me:

Building Go bootstrap cmd/go (go_bootstrap) using Go toolchain1.
runtime
internal/cpu
runtime/internal/sys
runtime/internal/atomic
runtime/internal/math
internal/bytealg
/builddir/go-1.12.9/go/src/runtime/internal/sys/intrinsics.go:40:11: internal compiler error: 'Ctz64': panic during lower while compiling Ctz64:
unused during bootstrap
goroutine 1 [running]:
bootstrap/cmd/compile/internal/ssa.func·004()
	/builddir/go-1.12.9/go/src/cmd/compile/internal/ssa/compile.go:45 +0xab
bootstrap/cmd/compile/internal/ssa.rewriteBlockAMD64(0x18e933b0, 0x1)
	/builddir/go-1.12.9/go/pkg/bootstrap/src/bootstrap/cmd/compile/internal/ssa/rewriteAMD64.go:6 +0x54
bootstrap/cmd/compile/internal/ssa.applyRewrite(0x18c540b0, 0x87c6af0, 0x87c6b24)
	/builddir/go-1.12.9/go/src/cmd/compile/internal/ssa/rewrite.go:33 +0xf8
bootstrap/cmd/compile/internal/ssa.lower(0x18c540b0)
	/builddir/go-1.12.9/go/src/cmd/compile/internal/ssa/lower.go:10 +0x3a
bootstrap/cmd/compile/internal/ssa.Compile(0x18c540b0)
	/builddir/go-1.12.9/go/src/cmd/compile/internal/ssa/compile.go:90 +0x71e
bootstrap/cmd/compile/internal/gc.buildssa(0x18c8e0c0, 0x0, 0x0)
	/builddir/go-1.12.9/go/src/cmd/compile/internal/gc/ssa.go:233 +0xc76
bootstrap/cmd/compile/internal/gc.compileSSA(0x18c8e0c0, 0x0)
	/builddir/go-1.12.9/go/src/cmd/compile/internal/gc/pgen.go:299 +0x3d
bootstrap/cmd/compile/internal/gc.compile(0x18c8e0c0)
	/builddir/go-1.12.9/go/src/cmd/compile/internal/gc/pgen.go:278 +0x241
bootstrap/cmd/compile/internal/gc.funccompile(0x18c8e0c0)
	/builddir/go-1.12.9/go/src/cmd/compile/internal/gc/pgen.go:221 +0x194
bootstrap/cmd/compile/internal/gc.Main(0x87c67b8)
	/builddir/go-1.12.9/go/src/cmd/compile/internal/gc/main.go:665 +0x2e01
main.main()
	/builddir/go-1.12.9/go/src/cmd/compile/main.go:51 +0x158

There's no code at cmd/compile/internal/ssa/rewriteAMD64.go:6.

[0paIescent]

Yeah that is weird, seems that the trace actually looks like this, I just stepped through everything manually:

cmd/compile/main.go:51 
cmd/compile/internal/gc/main.go:665
cmd/compile/internal/gc/pgen.go:221
cmd/compile/internal/gc/pgen.go:278
cmd/compile/internal/gc/pgen.go:299
cmd/compile/internal/gc/ssa.go:233
cmd/compile/internal/ssa/compile.go:408 & 90
cmd/compile/internal/ssa/lower.go:10
cmd/compile/internal/ssa/rewrite.go:80
cmd/compile/internal/ssa/rewriteAMD64:? not sure about the line in this file, but I'm sure it's somewhere in the rewriteBlockAMD64 function on line 66518
runtime/internal/sys/intrinsics.go:35

[cherrymui]

What version is the bootstrap compiler (GOROOT_BOOTSTRAP is set to)? The panic happens in toolchain1, which is compiled by the bootstrap compiler. The stack trace bootstrap/cmd/compile/... also suggests this. Maybe there is a bug in the bootstrap compiler, that miscompiled the Go 1.12.9 compiler. Maybe try a different version of Go to bootstrap?

[nilium]

This is still visible in 1.13 as well:

• https://travis-ci.org/void-linux/void-packages/jobs/580854496#L4187-L4225
• https://travis-ci.org/void-linux/void-packages/jobs/580854496#L4231-L4262

Edit: Updated with void-packages link now that i686 has run there.

The bootstrap compiler is go1.4-bootstrap-20171003, as used in the documentation on bootstrapping the compiler from source.

It's possible we could work around this by introducing a second bootstrap version to build after we build 1.4. For example, build 1.4, build 1.11 with 1.4, then build 1.13 (or 1.12.x or later) using that. However, I'm not sure that really helps us understand why the compiler panics.

[cherrymui]

Thanks, I can reproduce it now. I need to set GOHOSTARCH=386 GOARCH=amd64 GOROOT_BOOTSTRAP=$HOME/src/go1.4 on an AMD64 machine. What is your GOHOSTARCH and GOARCH setting? Presumably they are not the same?

I think at some point we might have decided to not supporting GOHOSTARCH != GOARCH during bootstrap. Could you set them the same by any chance?

[cherrymui]

The panic unused during bootstrap comes from https://go.googlesource.com/go/+/refs/tags/go1.13/src/cmd/dist/buildtool.go#271
which is introduced in CL https://go-review.googlesource.com/c/go/+/40503, which is years ago. I don't know why it starts failing only recently...

cc @bradfitz

[cherrymui]

It also explains why it fails at line 6. In the generated code by cmd/dist, the panic is indeed at line 6.

[bradfitz]

I've seen some +build line changes recently, adding a space in //+build. Perhaps dist parses them differently, at least in old Go.

/cc @tklauser

[nilium]

Has there been any further investigation into this? Or is there any other info I or someone else can provide here to help it along (since the issue is still labeled WaitingForInfo)?

[cherrymui]

@nilium, could you answer

What is your GOHOSTARCH and GOARCH setting? Presumably they are not the same?

Is there a reason that you need them to be different? Could you set them the same by any chance?

[gopherbot]

Change https://golang.org/cl/197602 mentions this issue: cmd/dist: fix bootstrap failure when GOHOSTARCH is set

[bradfitz]

I just sent https://go-review.googlesource.com/c/go/+/197602 which fixes @cherrymui's repro.

Could @0paIescent or @nilium confirm that fixes it for you?

[nilium]

I'll try applying the patch in the 1.13.1 build to see if that fixes it once I get off work.

[nilium]

Looks like the patch resolves the initial panics. I'm still seeing i686 fail to build on Travis CI (not when cross-compiling from x86_64 locally), but there's no output to dig into at the moment. For now I'm assuming this is a separate issue. I'm going to need to get a test machine to get a better look at it, since I can't reproduce the newest Travis failure on my workstation.