cmd/compile: longtest builder failing on TestNexting/gdb-opt-hist

[randall77]

After CL 174617, the longtest builder is failing on TestNexting/gdb-opt-hist.
Here's a sample of the log:

--- FAIL: TestNexting (13.35s)
    --- FAIL: TestNexting/gdb-opt-hist (2.98s)
        debug_test.go:245: step/next histories differ, diff=
            --- testdata/hist.gdb-opt.nexts	2019-04-30 21:30:30.000000000 +0000
            +++ /workdir/tmp/debug_test382328161/test-hist.gdb-opt.nexts	2019-04-30 22:37:43.423960976 +0000
            @@ -28,8 +28,6 @@
             i = 1
             81:			hist = ensure(int(i), hist)
             82:			hist[int(i)]++
            -74:		for scanner.Scan() { //gdb-opt=(scanner/A)
            -scanner = (bufio.Scanner *) <A>
             75:			s := scanner.Text()
             76:			i, err := strconv.ParseInt(s, 10, 64)
             77:			if err != nil { //gdb-dbg=(i) //gdb-opt=(err,hist,i)
            @@ -38,8 +36,6 @@
             i = 1
             81:			hist = ensure(int(i), hist)
             82:			hist[int(i)]++
            -74:		for scanner.Scan() { //gdb-opt=(scanner/A)
            -scanner = (bufio.Scanner *) <A>
             75:			s := scanner.Text()
             76:			i, err := strconv.ParseInt(s, 10, 64)
             77:			if err != nil { //gdb-dbg=(i) //gdb-opt=(err,hist,i)
            @@ -48,8 +44,6 @@
             i = 1
             81:			hist = ensure(int(i), hist)
             82:			hist[int(i)]++
            -74:		for scanner.Scan() { //gdb-opt=(scanner/A)
            -scanner = (bufio.Scanner *) <A>
             75:			s := scanner.Text()

When I compile src/cmd/compile/internal/ssa/testdata/hist.go both before and after this change, with both -S and -S=2, I don't see any line number differences. This is the extent of the difference of the -S output:

$ diff old new
1611c1611
< 	0x0020 02 07 01 0e 02 12 01 6e 02 06 01 1c 02 08 01 04  .......n........
---
> 	0x0020 02 07 01 0e 02 12 01 6e 02 02 01 20 02 08 01 04  .......n... ....

This difference is part of the go.isstmt."".test SDWARFMISC size=0 symbol.
So somehow my change messed up the statement marks? Is there a way to display those differences?

@dr2chase , any ideas?

[bradfitz]

Dup of #31263?

[dr2chase]

Working on it...

go test debug_test.go -v -args  -f -n

produces

...
=== RUN   TestNexting/gdb-opt-hist
( /Users/drchase/work/go-quick/bin/go build -o testdata/test-hist.gdb-opt '-gcflags=all= -l' -ldflags=-compressdwarf=false testdata/hist.go )
( TERM=dumb gdb -nx -iex 'add-auto-load-safe-path /Users/drchase/work/go-quick/src/runtime' -ex 'set startup-with-shell off' testdata/test-hist.gdb-opt )
tbreak main.test
run
# Tag for above is gdb-opt
...

Redo good and bad compilations prepending GOSSAFUNC=test to both of them.

Also compare dwarfdump -debug-line for both, search forward for testdata:
"good"

..................     74      0      2   0             0  is_stmt
..................     82      0      2   0             0  is_stmt
..................     82      0      2   0             0 
..................     81      0      2   0             0 
..................     74      0      2   0             0 
..................     81      0      2   0             0 
..................     74      0      2   0             0  is_stmt
..................     74      0      2   0             0 

"bad"

..................     74      0      2   0             0  is_stmt
(missing)
..................     82      0      2   0             0 
..................     81      0      2   0             0 
..................     74      0      2   0             0 
..................     81      0      2   0             0 
..................     74      0      2   0             0  is_stmt
..................     74      0      2   0             0 

It works okay with delve, I am still searching through the ssa output to see where the difference appears. It's in blocks 17 and 19, v264 and v275.

edit: The problem is in the original assignment of statement marks (numberlines).

[dr2chase]

I have a CL coming that doesn't fix this, but it at least stops us from relying on dumb luck to make gdb happy. I have not yet figured out what would make gdb happy; a naive reading of the marked lines here (the loop in question, after coming CL) shows more than enough is_stmt=1 lines.

[gopherbot]

Change https://golang.org/cl/174948 mentions this issue: cmd/compile: make numberlines line mismatch check ignore columns

[bcmills]

If the fix isn't going to land today, please add a skip so that the longtest builder will be able to proceed past TestNexting again. (The build dashboard is still unpleasantly red and I don't want to mask any more regressions.)

[dr2chase]

How do you feel about me changing it to use Delve instead?

[bradfitz]

@dr2chase, you're asking the wrong people. We don't know what this is testing. We just know it's red & harmful to people trying to get a useful regression signal out of this builder, and making people get used to broken builds.

[dr2chase]

Should have a fix (switch to delve, tested for 100+ repetitions on Linux and Mac) later today.
Tests running now will I had off to a talk.

[gopherbot]

Change https://golang.org/cl/176058 mentions this issue: cmd/compile: test delve instead of gdb in ssa/debug_test.go