"LLVM ERROR: Cannot select: t3: i16 = JumpTable" in CI on AVR target

[dkegel-fastly]

Got an ugly llvm crash in CI just now: https://github.com/tinygo-org/tinygo/runs/4722070252

=== CONT  TestBuild/AVR/goroutines.go
LLVM ERROR: Cannot select: t3: i16 = JumpTable<0>
In function: machine.AdjustMonotonicTimer
SIGABRT: abort
...
FAIL	github.com/tinygo-org/tinygo	62.689s
make: *** [Makefile:196: test] Error 1

Happened again in https://github.com/tinygo-org/tinygo/runs/4722481707

=== CONT  TestBuild/AVR/slice.go
LLVM ERROR: Cannot select: t3: i16 = JumpTable<0>
In function: runtime.alloc
SIGABRT: abort

Seems to happen at about 2m10s into Test TinyGo; successful runs seem to take longer to get to the same point.

Here’s a good run (a retry of the previous one): https://github.com/tinygo-org/tinygo/runs/4722602927

No idea why it doesn't happen every time. Different software or hardware on different github ci nodes?

[dkegel-fastly]

AVR may have some dirty laundry: llvm/llvm-project#31494 (comment)

[dkegel-fastly]

Happened again in https://github.com/tinygo-org/tinygo/runs/4729218984, again at 2 minutes 16 seconds, line 413.

=== CONT  TestBuild/AVR/slice.go
LLVM ERROR: Cannot select: t3: i16 = JumpTable<0>
In function: runtime.alloc
SIGABRT: abort

[dkegel-fastly]

FWIW, I snuck a "cat /proc/cpuinfo" into the top of the apt-get install step in #2487, the pull request that fails half the time with this problem.

On the first run, it said the cpu was a 2 core Xeon 8370C (so a small slice of a 32 core chip), and it did not hit this bug.

[aykevl]

AVR may have some dirty laundry: llvm/llvm-project#31494 (comment)

Not anymore :) see https://reviews.llvm.org/D107853#3224989

My guess would be a concurrency issue, so happening only sometimes. If someone can reproduce this locally that would be a very big help towards fixing this issue (especially if I manage to reproduce this on my own laptop).
What is especially strange is that JumpTable should not be used anyway, because it is disabled. However, I have a slight suspicion this commit might have changed things: llvm/llvm-project@567f6c1 - although I don't really know how.

[dkegel-fastly]

I seem to be able to reproduce something similar locally on my mac with the command

go test -run TestTest

on #2500, on a hex core macbook pro, using go version go1.17.6 and LLVM version 12.0.1

Log:

...
=== PAUSE TestTest/EmulatedRISCV/Fail
=== CONT  TestTest/EmulatedRISCV/Pass
=== CONT  TestTest/EmulatedRISCV/Fail
=== CONT  TestTest/Host/Nothing
    main_test.go:612: ?         github.com/tinygo-org/tinygo/tests/testing/nothing      [no test files]
LLVM ERROR: Cannot select: 0x614eba0: i32 = JumpTable<0>
In function: (reflect.Value).Len
SIGABRT: abort
PC=0x7fff2075d92e m=10 sigcode=0
...

[dkegel-fastly]

I'm preserved the failing commit as the tip of https://github.com/dkegel-fastly/tinygo/tree/dkegel-llvm12-error-2492

That commit seems to fail reliably locally, but only intermittently in CI.

[deadprogram]

Here is example of CI fail https://github.com/tinygo-org/tinygo/runs/4774165553?check_suite_focus=true#step:17:415

[aykevl]

I wonder if we could perhaps try to build LLVM with AddressSanitizer or ThreadSanitizer enabled? Not sure whether that works together with Go though. But those sanitizers might be able to find something.

[aykevl]

Or if this becomes reproducible on Linux, that would be awesome because then I can investigate this myself :)

[deadprogram]

@aykevl @dkegel-fastly is this issue still valid?

[aykevl]

I haven't seen this issue in quite some time, so it's probably fixed.

[deadprogram]

OK, in that case closing. Thank you!