One CPU bound goroutine blocks other goroutines (on multicore)

[jaffee]

What version of Go are you using (go version)?

$ go version
1.12.7

also 1.12.6 and 1.12.5

Does this issue reproduce with the latest release?

Yes

What operating system and processor architecture are you using (go env)?

go env Output

$ go env
GOARCH="amd64"
GOBIN=""
GOCACHE="/Users/jaffee/Library/Caches/go-build"
GOEXE=""
GOFLAGS=""
GOHOSTARCH="amd64"
GOHOSTOS="darwin"
GOOS="darwin"
GOPATH="/Users/jaffee/go"
GOPROXY=""
GORACE=""
GOROOT="/usr/local/Cellar/go/1.12.7/libexec"
GOTMPDIR=""
GOTOOLDIR="/usr/local/Cellar/go/1.12.7/libexec/pkg/tool/darwin_amd64"
GCCGO="gccgo"
CC="clang"
CXX="clang++"
CGO_ENABLED="1"
GOMOD=""
CGO_CFLAGS="-g -O2"
CGO_CPPFLAGS=""
CGO_CXXFLAGS="-g -O2"
CGO_FFLAGS="-g -O2"
CGO_LDFLAGS="-g -O2"
PKG_CONFIG="pkg-config"
GOGCCFLAGS="-fPIC -m64 -pthread -fno-caret-diagnostics -Qunused-arguments -fmessage-length=0 -fdebug-prefix-map=/var/folders/yw/d5lm39n95kz79tstt7tkd_b80000gp/T/go-build401461400=/tmp/go-build -gno-record-gcc-switches -fno-common"

I have also reproduced this on linux_amd64 on a 32 core VM, and several coworkers have reproduced it on linux/mac.

What did you do?

Example:
https://play.golang.org/p/5rs3K751_md

You can also:

go get github.com/jaffee/misc/schedmin

It first starts a goroutine which is making periodic HTTP requests, then sleeps for 4 seconds in order to allow a few requests to execute. It then generates and XORs random numbers together in a tight loop.

What did you expect to see?

On hardware with multiple cores, and GOMAXPROCS > 1, I expected the program to continue making HTTP requests for the duration of the work loop.

What did you see instead?

Shortly after the work loop starts, the HTTP loop hangs (typically on reading response body, though I've seen it happen at other places).

Periodically invoking runtime.Gosched() in the work loop allows the request loop to continue as expected.

I have a slightly more complex version of this example program which has flags for controlling various parameters at https://github.com/jaffee/misc/tree/master/schedissue/main.go

[av86743]

This could be by design. The problem is specifically with goroutine preemption being blocked, not with inefficient use of computational resources by run-time.

[jaffee]

I may be misunderstanding you, but I don't think this is an issue with goroutines not being pre-emptible. I have 8 cores, GOMAXPROCS=8, and I only have one goroutine running CPU bound code. Another goroutine which was started before the CPU bound code actually stops executing after the CPU bound code starts running.

It's my understanding that in this situation, the two goroutines should run in parallel since there are threads available for both of them.

[av86743]

Issues #24543 and possibly #32195 seem to be relevant to the subject. In particular, #32195 (comment) .

[ianlancetaylor]

Dup of #10958.