cmd/go: inconsistent coverage when using -coverprofile with packages without tests

[NarcisDavins]

Go version

go version go1.23.3 linux/amd64

Output of go env in your module/workspace:

GO111MODULE=''
GOARCH='amd64'
GOBIN=''
GOCACHE='/root/.cache/go-build'
GOENV='/root/.config/go/env'
GOEXE=''
GOEXPERIMENT=''
GOFLAGS=''
GOHOSTARCH='amd64'
GOHOSTOS='linux'
GOINSECURE=''
GOMODCACHE='/go/pkg/mod'
GONOPROXY=''
GONOSUMDB=''
GOOS='linux'
GOPATH='/go'
GOPRIVATE=''
GOPROXY='https://proxy.golang.org,direct'
GOROOT='/usr/local/go'
GOSUMDB='sum.golang.org'
GOTMPDIR=''
GOTOOLCHAIN='local'
GOTOOLDIR='/usr/local/go/pkg/tool/linux_amd64'
GOVCS=''
GOVERSION='go1.23.3'
GODEBUG=''
GOTELEMETRY='local'
GOTELEMETRYDIR='/root/.config/go/telemetry'
GCCGO='gccgo'
GOAMD64='v1'
AR='ar'
CC='gcc'
CXX='g++'
CGO_ENABLED='1'
GOMOD='/dev/null'
GOWORK=''
CGO_CFLAGS='-O2 -g'
CGO_CPPFLAGS=''
CGO_CXXFLAGS='-O2 -g'
CGO_FFLAGS='-O2 -g'
CGO_LDFLAGS='-O2 -g'
PKG_CONFIG='pkg-config'
GOGCCFLAGS='-fPIC -m64 -pthread -Wl,--no-gc-sections -fmessage-length=0 -ffile-prefix-map=/tmp/go-build2962845657=/tmp/go-build -gno-record-gcc-switches'

What did you do?

Given the following project, with pkg1 not containing any test, and pkg2 with 1 test that is calling pkg1, when using go test with the -coverprofile option, the cover file generated contains pkg1 as covered

$->> cov $ tree
.
├── go.mod
├── pkg1
│   └── file.go
└── pkg2
    ├── file.go
    └── file_test.go

3 directories, 4 files
$->> cov $ cat go.mod 
module cov
$->> cov $ cat pkg1/file.go 
package pkg1

func DoSomething() bool {
    return true;
}
$->> cov $ cat pkg2/file.go 
package pkg2

func DoSomething() bool {
    return true;
}
$->> cov $ cat pkg2/file_test.go 
package pkg2

import "testing"
import "cov/pkg1"

func TestSmth(t * testing.T) {
  pkg1.DoSomething()
  DoSomething()
}

What did you see happen?

Running go test with -coverprofile the output correctly shows 0% coverage on the package without tests, but the generated file info is marking pkg1 as covered

$->> cov $ docker run --rm -it -v .:/code -w /code golang:1.23.3 go test ./... -coverprofile cov.out
	cov/pkg1		coverage: 0.0% of statements
ok  	cov/pkg2	0.003s	coverage: 100.0% of statements
$->> cov $ cat cov.out 
mode: set
cov/pkg1/file.go:3.25,5.2 1 0
cov/pkg1/file.go:3.25,5.2 1 1
cov/pkg2/file.go:3.25,5.2 1 1

What did you expect to see?

pkg1 should not be marked as covered in the cover file, as it does not have any test and I'm not using other params such as -coverpkg.

using GOEXPERIMENT=nocoverageredesign is currently a workaround we are using, but would stop being an option after: #55953.

Another workaround would be to generate empty test files on all packages, but as far as I can tell, this shouldn't be the expected behavior.

$->> cov $ echo "package pkg1" >> pkg1/file_test.go
$->> cov $ docker run --rm -it -v .:/code -w /code golang:1.23.3 go test ./... -coverprofile cov.out
ok  	cov/pkg1	0.003s	coverage: 0.0% of statements [no tests to run]
ok  	cov/pkg2	0.003s	coverage: 100.0% of statements
$->> cov $ cat cov.out 
mode: set
cov/pkg1/file.go:3.25,5.2 1 0
cov/pkg2/file.go:3.25,5.2 1 1

[gabyhelp]

Related Issues and Documentation

• cmd/go: should include packages without tests in cover profile file #27261 (closed)
• cmd/go: go 1.22.0: go test throws errors when processing folders not listed in coverpkg argument #65653 (closed)
• Unintuitive behavior when using -coverpkg #23786 (closed)
• Coverage incorrectly reported as 100% ( go test cover) #26760 (closed)
• cmd/go: go test -cover & go test -coverprofile should always output a coverage #24570 (closed)
• cmd/go: -coverpkg=all gives different coverage value when run on a package list vs ./... #23883 (closed)
• cmd/cover: regression in 1.20, cover tool no longer includes packages without tests #58770 (closed)
• cmd/go: coverpkg doesn't ignore directories starting with '.' #66038
• cmd/go: improperly synchronized -coverprofile output for packages with no tests #63356 (closed)
• cmd/covdata: percent does not support -coverpkg #66464

_{(Emoji vote if this was helpful or unhelpful; more detailed feedback welcome in this discussion.)}

[dr2chase]

@thanm we still list you as owner, maybe you still have background on this?
otherwise, @golang/compiler

[thanm]

Hi @NarcisDavins you wrote:

pkg1 should not be marked as covered in the cover file, as it does not have any test and I'm not using other params such as -coverpkg.

I'm confused about what you're asking here. In the contents of cov.out there is this line:

cov/pkg1/file.go:3.25,5.2 1 0

That indicates that package pkg1 has 1 statement and that it is not covered. When you say "marked as covered" do you mean that pkg should not be mentioned at all?

[NarcisDavins]

Hi @NarcisDavins you wrote:

pkg1 should not be marked as covered in the cover file, as it does not have any test and I'm not using other params such as -coverpkg.

I'm confused about what you're asking here. In the contents of cov.out there is this line:

cov/pkg1/file.go:3.25,5.2 1 0

That indicates that package pkg1 has 1 statement and that it is not covered. When you say "marked as covered" do you mean that pkg should not be mentioned at all?

@thanm pkg1 appears twice, the line you copied where it indeed shows as not covered, and the following one where it appears as covered. I'll paste again the "What did you see happen?" part of the issue:

$->> cov $ docker run --rm -it -v .:/code -w /code golang:1.23.3 go test ./... -coverprofile cov.out
	cov/pkg1		coverage: 0.0% of statements
ok  	cov/pkg2	0.003s	coverage: 100.0% of statements
$->> cov $ cat cov.out 
mode: set
cov/pkg1/file.go:3.25,5.2 1 0
cov/pkg1/file.go:3.25,5.2 1 1
cov/pkg2/file.go:3.25,5.2 1 1

Edit: that second line where it appears as covered is the one that should not be there.

[thanm]

Right, of course. Somehow didn't pick up on that the first time. Thanks for the clarification.

[gopherbot]

Change https://go.dev/cl/627315 mentions this issue: internal/coverage: refactor EmitTextual in preparation for bugfix

[gopherbot]

Change https://go.dev/cl/627316 mentions this issue: internal/coverage: pass selected package set to EmitTextual

[gopherbot]

Change https://go.dev/cl/646355 mentions this issue: cmd/go: update new test for removal of nocoverageredesign

[Emptyless]

Noticed that with go1.24 we also faced this for packages that contain generated example code. As an intermediate solution to adding tests to those packages you can replicate <go1.24's behaviour by combining coverpkg and go list:

-coverpkg=$(go list -f '{{ if .TestGoFiles }}{{ .ImportPath }}{{ end }}' ./... | paste -sd "," -)

This lists all import paths (delimited using ",") that contain test files