cmd/go: test Documentation: Package List Parallelisation

[Rossiar]

What version of Go are you using (go version)?

$ go version
go version go1.13.1 linux/amd64

Does this issue reproduce with the latest release?

Yes

What operating system and processor architecture are you using (go env)?

go env Output

$ go env
GO111MODULE=""
GOARCH="amd64"
GOBIN=""
GOCACHE="/home/ross/.cache/go-build"
GOENV="/home/ross/.config/go/env"
GOEXE=""
GOFLAGS=""
GOHOSTARCH="amd64"
GOHOSTOS="linux"
GOOS="linux"
GOPATH="/home/ross/go"
GOPROXY="https://proxy.golang.org,direct"
GOROOT="/usr/local/go"
GOSUMDB="sum.golang.org"
GOTMPDIR=""
GOTOOLDIR="/usr/local/go/pkg/tool/linux_amd64"
GCCGO="gccgo"
AR="ar"
CC="gcc"
CXX="g++"
CGO_ENABLED="1"
GOMOD=""
CGO_CFLAGS="-g -O2"
CGO_CPPFLAGS=""
CGO_CXXFLAGS="-g -O2"
CGO_FFLAGS="-g -O2"
CGO_LDFLAGS="-g -O2"
PKG_CONFIG="pkg-config"
GOGCCFLAGS="-fPIC -m64 -pthread -fmessage-length=0 -fdebug-prefix-map=/tmp/go-build910819809=/tmp/go-build -gno-record-gcc-switches"

What did you do?

ran go help test which displays the go test documentation

What did you expect to see?

While running go test ./... in a project with many subpackages, I noticed that integration tests in each subpackage were failing (randomly) when accessing a shared resource (such as a database).

But if I ran the same code with go test ./... -p 1 I see that my packages run in sequential order and the failures do not occur. I am happy with this outcome and I understand why parallelization is the default when running in this mode, but I feel that this behaviour could be explicitly highlighted in the documentation to avoid confusion.

Right now the existing documentation comes close to covering this, but does not:

The second, called package list mode, occurs when go test is invoked
with explicit package arguments (for example 'go test math', 'go
test ./...', and even 'go test .'). In this mode, go test compiles
and tests each of the packages listed on the command line. If a
package test passes, go test prints only the final 'ok' summary
line. If a package test fails, go test prints the full test output.
If invoked with the -bench or -v flag, go test prints the full
output even for passing package tests, in order to display the
requested benchmark results or verbose logging. After the package
tests for all of the listed packages finish, and their output is
printed, go test prints a final 'FAIL' status if any package test
has failed.

What did you see instead?

The documentation doesn't cover the topic that I want to understand.

Let me know if an example project is necessary to illustrate this concept better, or if I can provide any further detail in writing. I didn't want to post an issue straight away for this, but the issues channel in golang slack that I checked hasn't seen any activity since last year, which indicates that it's not very well used.

[toothrot]

It does mention In addition to the build flags, the flags handled by 'go test' itself are:. The help for go build covers this a bit:

	-p n
		the number of programs, such as build commands or
		test binaries, that can be run in parallel.
		The default is the number of CPUs available.

I'm not sure if an example project would be helpful, I think you've described the scenario clearly enough, thanks!

Do you have a specific suggestion on what could be improved? I understand that your scenario is different, but ideally people do not have to manipulate this, as avoiding order dependency in testing is a feature in my opinion.

/cc @bcmills @jayconrod

[Rossiar]

I would just add this definition of the -p flag to the list shown when running go help test - along with this could be an addition to the existing text around package list mode:

...In this mode, go test compiles and tests each of the packages listed on the command line, this can occur sequentially or in parallel depending on the value of the -p flag and number of CPUs available.

[matloob]

I think we can revisit if there are more reports of confusion, but this is probably not necessary for now. I think most people expect their test binaries to run in parallel. (I admit there's a bit of confusion here because individual test cases don't run in parallel by default)

The flags presented in the go test docs are presented as being in addition to the build flags and it would be a bit strange to highlight the -p flag in contrast to other flags.