maybe a deadlock in golang : runtime.selectgo while runtime.stopTheWorldWithSema #44047
Comments
If you are able to reproduce the problem again, please capture the entire go stack trace by sending SIGQUIT to the process. Also, please build your application under the race detector to confirm that there are no data races in the codebase. See https://blog.golang.org/race-detector . |
davecheney
added
the
WaitingForInfo
|
so, data race can cause such a problem ?. there are many data race in my program indeed..... |
|
yes. If your program has any data races then nothing can be guaranteed. You must correct any data race in your Go program. I'm going to close this issue now as it doesn't look like there is a bug in Go. Once you've fixed the data races in your program, please comment if you are still experiencing an issue. |
$ go version
go version go1.15.6 linux/amd64
$ go env
GO111MODULE="off"
GOARCH="amd64"
GOBIN=""
GOCACHE="/home/zhuoyikang/.cache/go-build"
GOENV="/home/zhuoyikang/.config/go/env"
GOEXE=""
GOFLAGS=""
GOHOSTARCH="amd64"
GOHOSTOS="linux"
GOINSECURE=""
GOMODCACHE="/home/zhuoyikang/Go/pkg/mod"
GONOPROXY=""
GONOSUMDB=""
GOOS="linux"
GOPATH="/home/zhuoyikang/Go"
GOPRIVATE=""
GOPROXY="https://proxy.golang.org,direct"
GOROOT="/home/zhuoyikang/GoSDK/go1.15.6"
GOSUMDB="sum.golang.org"
GOTMPDIR=""
GOTOOLDIR="/home/zhuoyikang/GoSDK/go1.15.6/pkg/tool/linux_amd64"
GCCGO="gccgo"
AR="ar"
CC="gcc"
CXX="g++"
CGO_ENABLED="1"
GOMOD=""
CGO_CFLAGS="-g -O2"
CGO_CPPFLAGS=""
CGO_CXXFLAGS="-g -O2"
CGO_FFLAGS="-g -O2"
CGO_LDFLAGS="-g -O2"
PKG_CONFIG="pkg-config"
GOGCCFLAGS="-fPIC -m64 -pthread -fmessage-length=0 -fdebug-prefix-map=/tmp/go-build298471110=/tmp/go-build -gno-record-gcc-switches"
in my production env, the program stopped, and I get a core dump by gdb, see the stack below:
Thread 16 (LWP 5120):
#0 runtime.futex () at /home/zhuoyikang/GoSDK/go1.15.6/src/runtime/sys_linux_amd64.s:588
#1 0x00000000004358e6 in runtime.futexsleep (addr=0xc016986148, val=0, ns=-1) at /home/zhuoyikang/GoSDK/go1.15.6/src/runtime/os_linux.go:45
#2 0x000000000040e33f in runtime.notesleep (n=0xc016986148) at /home/zhuoyikang/GoSDK/go1.15.6/src/runtime/lock_futex.go:159
#3 0x0000000000440105 in runtime.stopm () at /home/zhuoyikang/GoSDK/go1.15.6/src/runtime/proc.go:1924
#4 0x00000000004418df in runtime.findrunnable (gp=0xc00005f000, inheritTime=false) at /home/zhuoyikang/GoSDK/go1.15.6/src/runtime/proc.go:2485
#5 0x0000000000442557 in runtime.schedule () at /home/zhuoyikang/GoSDK/go1.15.6/src/runtime/proc.go:2683
#6 0x0000000000442afd in runtime.park_m (gp=0xc00285bc80) at /home/zhuoyikang/GoSDK/go1.15.6/src/runtime/proc.go:2851
#7 0x000000000047067b in runtime.mcall () at /home/zhuoyikang/GoSDK/go1.15.6/src/runtime/asm_amd64.s:318
#8 0x0000000000000000 in ?? ()
Thread 15 (LWP 30825):
#0 runtime.futex () at /home/zhuoyikang/GoSDK/go1.15.6/src/runtime/sys_linux_amd64.s:588
#1 0x00000000004358e6 in runtime.futexsleep (addr=0xc003afc948, val=0, ns=-1) at /home/zhuoyikang/GoSDK/go1.15.6/src/runtime/os_linux.go:45
#2 0x000000000040e33f in runtime.notesleep (n=0xc003afc948) at /home/zhuoyikang/GoSDK/go1.15.6/src/runtime/lock_futex.go:159
#3 0x0000000000440105 in runtime.stopm () at /home/zhuoyikang/GoSDK/go1.15.6/src/runtime/proc.go:1924
#4 0x00000000004418df in runtime.findrunnable (gp=0xc0052c3800, inheritTime=false) at /home/zhuoyikang/GoSDK/go1.15.6/src/runtime/proc.go:2485
#5 0x0000000000442557 in runtime.schedule () at /home/zhuoyikang/GoSDK/go1.15.6/src/runtime/proc.go:2683
#6 0x0000000000442afd in runtime.park_m (gp=0xc0027d8f00) at /home/zhuoyikang/GoSDK/go1.15.6/src/runtime/proc.go:2851
#7 0x000000000047067b in runtime.mcall () at /home/zhuoyikang/GoSDK/go1.15.6/src/runtime/asm_amd64.s:318
#8 0x0000000000000000 in ?? ()
Thread 14 (LWP 27358):
#0 runtime.futex () at /home/zhuoyikang/GoSDK/go1.15.6/src/runtime/sys_linux_amd64.s:588
#1 0x00000000004358e6 in runtime.futexsleep (addr=0xc0004f8148, val=0, ns=-1) at /home/zhuoyikang/GoSDK/go1.15.6/src/runtime/os_linux.go:45
#2 0x000000000040e33f in runtime.notesleep (n=0xc0004f8148) at /home/zhuoyikang/GoSDK/go1.15.6/src/runtime/lock_futex.go:159
#3 0x0000000000440105 in runtime.stopm () at /home/zhuoyikang/GoSDK/go1.15.6/src/runtime/proc.go:1924
#4 0x00000000004418df in runtime.findrunnable (gp=0xc0052c3800, inheritTime=false) at /home/zhuoyikang/GoSDK/go1.15.6/src/runtime/proc.go:2485
#5 0x0000000000442557 in runtime.schedule () at /home/zhuoyikang/GoSDK/go1.15.6/src/runtime/proc.go:2683
#6 0x0000000000442afd in runtime.park_m (gp=0xc0026a1c80) at /home/zhuoyikang/GoSDK/go1.15.6/src/runtime/proc.go:2851
#7 0x000000000047067b in runtime.mcall () at /home/zhuoyikang/GoSDK/go1.15.6/src/runtime/asm_amd64.s:318
#8 0x0000000000000000 in ?? ()
Thread 13 (LWP 27357):
#0 runtime.futex () at /home/zhuoyikang/GoSDK/go1.15.6/src/runtime/sys_linux_amd64.s:588
#1 0x0000000000435957 in runtime.futexsleep (addr=0x500efa8 <compress/flate.fixedHuffmanDecoder+264>, val=0, ns=100000) at /home/zhuoyikang/GoSDK/go1.15.6/src/runtime/os_linux.go:51
#2 0x000000000040e4d9 in runtime.notetsleep_internal (n=0x500efa8 <compress/flate.fixedHuffmanDecoder+264>, ns=100000, ~r2=) at /home/zhuoyikang/GoSDK/go1.15.6/src/runtime/lock_futex.go:201
#3 0x000000000040e5b1 in runtime.notetsleep (n=0x500efa8 <compress/flate.fixedHuffmanDecoder+264>, ns=100000, ~r2=) at /home/zhuoyikang/GoSDK/go1.15.6/src/runtime/lock_futex.go:224
#4 0x000000000043e728 in runtime.stopTheWorldWithSema () at /home/zhuoyikang/GoSDK/go1.15.6/src/runtime/proc.go:1020
#5 0x0000000000470726 in runtime.systemstack () at /home/zhuoyikang/GoSDK/go1.15.6/src/runtime/asm_amd64.s:370
#6 0x000000000043e9c0 in ?? () at /home/zhuoyikang/GoSDK/go1.15.6/src/runtime/signal_unix.go:1043
#7 0x0000000000000000 in ?? ()
Thread 12 (LWP 27351):
#0 runtime.futex () at /home/zhuoyikang/GoSDK/go1.15.6/src/runtime/sys_linux_amd64.s:588
#1 0x00000000004358e6 in runtime.futexsleep (addr=0xc00006d548, val=0, ns=-1) at /home/zhuoyikang/GoSDK/go1.15.6/src/runtime/os_linux.go:45
#2 0x000000000040e33f in runtime.notesleep (n=0xc00006d548) at /home/zhuoyikang/GoSDK/go1.15.6/src/runtime/lock_futex.go:159
#3 0x0000000000440105 in runtime.stopm () at /home/zhuoyikang/GoSDK/go1.15.6/src/runtime/proc.go:1924
#4 0x00000000004418df in runtime.findrunnable (gp=0xc00005c800, inheritTime=false) at /home/zhuoyikang/GoSDK/go1.15.6/src/runtime/proc.go:2485
#5 0x0000000000442557 in runtime.schedule () at /home/zhuoyikang/GoSDK/go1.15.6/src/runtime/proc.go:2683
#6 0x0000000000442afd in runtime.park_m (gp=0xc00290d800) at /home/zhuoyikang/GoSDK/go1.15.6/src/runtime/proc.go:2851
#7 0x000000000047067b in runtime.mcall () at /home/zhuoyikang/GoSDK/go1.15.6/src/runtime/asm_amd64.s:318
#8 0x0000000000000000 in ?? ()
Thread 11 (LWP 27350):
#0 runtime.futex () at /home/zhuoyikang/GoSDK/go1.15.6/src/runtime/sys_linux_amd64.s:588
#1 0x00000000004358e6 in runtime.futexsleep (addr=0xc0004f8948, val=0, ns=-1) at /home/zhuoyikang/GoSDK/go1.15.6/src/runtime/os_linux.go:45
#2 0x000000000040e33f in runtime.notesleep (n=0xc0004f8948) at /home/zhuoyikang/GoSDK/go1.15.6/src/runtime/lock_futex.go:159
#3 0x0000000000440105 in runtime.stopm () at /home/zhuoyikang/GoSDK/go1.15.6/src/runtime/proc.go:1924
#4 0x00000000004418df in runtime.findrunnable (gp=0xc000061800, inheritTime=false) at /home/zhuoyikang/GoSDK/go1.15.6/src/runtime/proc.go:2485
#5 0x0000000000442557 in runtime.schedule () at /home/zhuoyikang/GoSDK/go1.15.6/src/runtime/proc.go:2683
#6 0x0000000000442afd in runtime.park_m (gp=0xc002a80600) at /home/zhuoyikang/GoSDK/go1.15.6/src/runtime/proc.go:2851
#7 0x000000000047067b in runtime.mcall () at /home/zhuoyikang/GoSDK/go1.15.6/src/runtime/asm_amd64.s:318
#8 0x0000000000000000 in ?? ()
Thread 10 (LWP 27349):
#0 runtime.futex () at /home/zhuoyikang/GoSDK/go1.15.6/src/runtime/sys_linux_amd64.s:588
#1 0x00000000004358e6 in runtime.futexsleep (addr=0xc000184948, val=0, ns=-1) at /home/zhuoyikang/GoSDK/go1.15.6/src/runtime/os_linux.go:45
#2 0x000000000040e33f in runtime.notesleep (n=0xc000184948) at /home/zhuoyikang/GoSDK/go1.15.6/src/runtime/lock_futex.go:159
#3 0x00000000004408ad in runtime.stoplockedm () at /home/zhuoyikang/GoSDK/go1.15.6/src/runtime/proc.go:2085
#4 0x000000000044270c in runtime.schedule () at /home/zhuoyikang/GoSDK/go1.15.6/src/runtime/proc.go:2617
#5 0x0000000000442afd in runtime.park_m (gp=0xc000001980) at /home/zhuoyikang/GoSDK/go1.15.6/src/runtime/proc.go:2851
#6 0x000000000047067b in runtime.mcall () at /home/zhuoyikang/GoSDK/go1.15.6/src/runtime/asm_amd64.s:318
#7 0x0000000000000000 in ?? ()
Thread 9 (LWP 27348):
#0 runtime.futex () at /home/zhuoyikang/GoSDK/go1.15.6/src/runtime/sys_linux_amd64.s:588
#1 0x00000000004358e6 in runtime.futexsleep (addr=0xc00006d148, val=0, ns=-1) at /home/zhuoyikang/GoSDK/go1.15.6/src/runtime/os_linux.go:45
---Type to continue, or q to quit---
#2 0x000000000040e33f in runtime.notesleep (n=0xc00006d148) at /home/zhuoyikang/GoSDK/go1.15.6/src/runtime/lock_futex.go:159
#3 0x0000000000440105 in runtime.stopm () at /home/zhuoyikang/GoSDK/go1.15.6/src/runtime/proc.go:1924
#4 0x00000000004418df in runtime.findrunnable (gp=0xc000061800, inheritTime=false) at /home/zhuoyikang/GoSDK/go1.15.6/src/runtime/proc.go:2485
#5 0x0000000000442557 in runtime.schedule () at /home/zhuoyikang/GoSDK/go1.15.6/src/runtime/proc.go:2683
#6 0x0000000000442afd in runtime.park_m (gp=0xc000186600) at /home/zhuoyikang/GoSDK/go1.15.6/src/runtime/proc.go:2851
#7 0x000000000047067b in runtime.mcall () at /home/zhuoyikang/GoSDK/go1.15.6/src/runtime/asm_amd64.s:318
#8 0x0000000000000000 in ?? ()
Thread 8 (LWP 27347):
#0 runtime.futex () at /home/zhuoyikang/GoSDK/go1.15.6/src/runtime/sys_linux_amd64.s:588
#1 0x00000000004358e6 in runtime.futexsleep (addr=0xc000184548, val=0, ns=-1) at /home/zhuoyikang/GoSDK/go1.15.6/src/runtime/os_linux.go:45
#2 0x000000000040e33f in runtime.notesleep (n=0xc000184548) at /home/zhuoyikang/GoSDK/go1.15.6/src/runtime/lock_futex.go:159
#3 0x0000000000440105 in runtime.stopm () at /home/zhuoyikang/GoSDK/go1.15.6/src/runtime/proc.go:1924
#4 0x00000000004418df in runtime.findrunnable (gp=0xc0052c1000, inheritTime=false) at /home/zhuoyikang/GoSDK/go1.15.6/src/runtime/proc.go:2485
#5 0x0000000000442557 in runtime.schedule () at /home/zhuoyikang/GoSDK/go1.15.6/src/runtime/proc.go:2683
#6 0x0000000000442afd in runtime.park_m (gp=0xc00290d800) at /home/zhuoyikang/GoSDK/go1.15.6/src/runtime/proc.go:2851
#7 0x000000000047067b in runtime.mcall () at /home/zhuoyikang/GoSDK/go1.15.6/src/runtime/asm_amd64.s:318
Thread 7 (LWP 27346):
#0 runtime.futex () at /home/zhuoyikang/GoSDK/go1.15.6/src/runtime/sys_linux_amd64.s:588
#1 0x00000000004358e6 in runtime.futexsleep (addr=0x5040798 <runtime.fwdSig+88>, val=0, ns=-1) at /home/zhuoyikang/GoSDK/go1.15.6/src/runtime/os_linux.go:45
#2 0x000000000040e33f in runtime.notesleep (n=0x5040798 <runtime.fwdSig+88>) at /home/zhuoyikang/GoSDK/go1.15.6/src/runtime/lock_futex.go:159
#3 0x000000000044001a in runtime.templateThread () at /home/zhuoyikang/GoSDK/go1.15.6/src/runtime/proc.go:1902
#4 0x000000000043eb28 in runtime.mstart1 () at /home/zhuoyikang/GoSDK/go1.15.6/src/runtime/proc.go:1172
#5 0x000000000043ea2e in runtime.mstart () at /home/zhuoyikang/GoSDK/go1.15.6/src/runtime/proc.go:1137
#6 0x00000000004019bc in runtime/cgo(.text) ()
#7 0x0000000000000000 in ?? ()
Thread 6 (LWP 27345):
#0 runtime.futex () at /home/zhuoyikang/GoSDK/go1.15.6/src/runtime/sys_linux_amd64.s:588
#1 0x00000000004358e6 in runtime.futexsleep (addr=0x50408a0 <runtime.fwdSig+352>, val=0, ns=-1) at /home/zhuoyikang/GoSDK/go1.15.6/src/runtime/os_linux.go:45
#2 0x000000000040e426 in runtime.notetsleep_internal (n=0x50408a0 <runtime.fwdSig+352>, ns=-1, ~r2=) at /home/zhuoyikang/GoSDK/go1.15.6/src/runtime/lock_futex.go:182
#3 0x000000000040e64c in runtime.notetsleepg (n=0x50408a0 <runtime.fwdSig+352>, ns=-1, ~r2=) at /home/zhuoyikang/GoSDK/go1.15.6/src/runtime/lock_futex.go:236
#4 0x000000000046ec3d in os/signal.signal_recv (~r0=) at /home/zhuoyikang/GoSDK/go1.15.6/src/runtime/sigqueue.go:147
#5 0x0000000001e3a5a5 in os/signal.loop () at /home/zhuoyikang/GoSDK/go1.15.6/src/os/signal/signal_unix.go:23
#6 0x0000000000472481 in runtime.goexit () at /home/zhuoyikang/GoSDK/go1.15.6/src/runtime/asm_amd64.s:1374
#7 0x0000000000000000 in ?? ()
Thread 5 (LWP 27344):
#0 runtime.futex () at /home/zhuoyikang/GoSDK/go1.15.6/src/runtime/sys_linux_amd64.s:588
#1 0x00000000004358e6 in runtime.futexsleep (addr=0xc00006cd48, val=0, ns=-1) at /home/zhuoyikang/GoSDK/go1.15.6/src/runtime/os_linux.go:45
#2 0x000000000040e33f in runtime.notesleep (n=0xc00006cd48) at /home/zhuoyikang/GoSDK/go1.15.6/src/runtime/lock_futex.go:159
#3 0x0000000000440105 in runtime.stopm () at /home/zhuoyikang/GoSDK/go1.15.6/src/runtime/proc.go:1924
#4 0x00000000004418df in runtime.findrunnable (gp=0xc0052bc000, inheritTime=false) at /home/zhuoyikang/GoSDK/go1.15.6/src/runtime/proc.go:2485
#5 0x0000000000442557 in runtime.schedule () at /home/zhuoyikang/GoSDK/go1.15.6/src/runtime/proc.go:2683
#6 0x0000000000442afd in runtime.park_m (gp=0xc006b6b680) at /home/zhuoyikang/GoSDK/go1.15.6/src/runtime/proc.go:2851
#7 0x000000000047067b in runtime.mcall () at /home/zhuoyikang/GoSDK/go1.15.6/src/runtime/asm_amd64.s:318
#8 0x0000000000000000 in ?? ()
Thread 4 (LWP 27343):
#0 runtime.futex () at /home/zhuoyikang/GoSDK/go1.15.6/src/runtime/sys_linux_amd64.s:588
#1 0x00000000004358e6 in runtime.futexsleep (addr=0xc00006c948, val=0, ns=-1) at /home/zhuoyikang/GoSDK/go1.15.6/src/runtime/os_linux.go:45
#2 0x000000000040e33f in runtime.notesleep (n=0xc00006c948) at /home/zhuoyikang/GoSDK/go1.15.6/src/runtime/lock_futex.go:159
#3 0x0000000000440105 in runtime.stopm () at /home/zhuoyikang/GoSDK/go1.15.6/src/runtime/proc.go:1924
#4 0x00000000004418df in runtime.findrunnable (gp=0xc0052bc000, inheritTime=false) at /home/zhuoyikang/GoSDK/go1.15.6/src/runtime/proc.go:2485
#5 0x0000000000442557 in runtime.schedule () at /home/zhuoyikang/GoSDK/go1.15.6/src/runtime/proc.go:2683
#6 0x0000000000442afd in runtime.park_m (gp=0xc0026b4300) at /home/zhuoyikang/GoSDK/go1.15.6/src/runtime/proc.go:2851
#7 0x000000000047067b in runtime.mcall () at /home/zhuoyikang/GoSDK/go1.15.6/src/runtime/asm_amd64.s:318
#8 0x0000000000000000 in ?? ()
Thread 3 (LWP 27342):
#0 runtime.usleep () at /home/zhuoyikang/GoSDK/go1.15.6/src/runtime/sys_linux_amd64.s:146
#1 0x00000000004471ad in runtime.sysmon () at /home/zhuoyikang/GoSDK/go1.15.6/src/runtime/proc.go:4660
#2 0x000000000043eb28 in runtime.mstart1 () at /home/zhuoyikang/GoSDK/go1.15.6/src/runtime/proc.go:1172
#3 0x000000000043ea2e in runtime.mstart () at /home/zhuoyikang/GoSDK/go1.15.6/src/runtime/proc.go:1137
#4 0x00000000004019bc in runtime/cgo(.text) ()
#5 0x0000000000000000 in ?? ()
Thread 2 (LWP 27233):
#0 runtime.futex () at /home/zhuoyikang/GoSDK/go1.15.6/src/runtime/sys_linux_amd64.s:588
#1 0x00000000004358e6 in runtime.futexsleep (addr=0x500f7e8 <math/big.cacheBase10+264>, val=0, ns=-1) at /home/zhuoyikang/GoSDK/go1.15.6/src/runtime/os_linux.go:45
#2 0x000000000040e33f in runtime.notesleep (n=0x500f7e8 <math/big.cacheBase10+264>) at /home/zhuoyikang/GoSDK/go1.15.6/src/runtime/lock_futex.go:159
#3 0x0000000000440105 in runtime.stopm () at /home/zhuoyikang/GoSDK/go1.15.6/src/runtime/proc.go:1924
#4 0x00000000004418df in runtime.findrunnable (gp=0xc0052c3800, inheritTime=false) at /home/zhuoyikang/GoSDK/go1.15.6/src/runtime/proc.go:2485
#5 0x0000000000442557 in runtime.schedule () at /home/zhuoyikang/GoSDK/go1.15.6/src/runtime/proc.go:2683
#6 0x0000000000442afd in runtime.park_m (gp=0xc002951c80) at /home/zhuoyikang/GoSDK/go1.15.6/src/runtime/proc.go:2851
#7 0x000000000047067b in runtime.mcall () at /home/zhuoyikang/GoSDK/go1.15.6/src/runtime/asm_amd64.s:318
#8 0x0000000000470574 in runtime.rt0_go () at /home/zhuoyikang/GoSDK/go1.15.6/src/runtime/asm_amd64.s:220
---Type to continue, or q to quit---
#9 0x0000000000000000 in ?? ()
Thread 1 (LWP 5119):
#0 runtime.selectgo (cas0=0xc039d3cee8, order0=0xc039d3ceb4, ncases=3, ~r3=, ~r4=) at /home/zhuoyikang/GoSDK/go1.15.6/src/runtime/select.go:357
#1 0x0000000001d7e3b9 in oops/lib.(*Worker).startWork (w=0xc02b3dad80, ret=false) at /home/zhuoyikang/Go/src/oops/lib/workers.go:603
#2 0x0000000001d7ff5a in oops/lib.(*WorkerMgr).newWorker.func1 (wm=0xc003ae6840, ow=0x0, w=0xc02b3dad80) at /home/zhuoyikang/Go/src/oops/lib/workers.go:381
#3 0x0000000000472481 in runtime.goexit () at /home/zhuoyikang/GoSDK/go1.15.6/src/runtime/asm_amd64.s:1374
#4 0x000000c003ae6840 in ?? ()
#5 0x0000000000000000 in ?? ()
(gdb)
The text was updated successfully, but these errors were encountered: