cmd/go: test coverpkg=all ./... with test only packages will fail to build #27333

AlexRouSg · 2018-08-29T07:29:31Z

Please answer these questions before submitting your issue. Thanks!

What version of Go are you using (`go version`)?

1.11

Does this issue reproduce with the latest release?

Yes

What operating system and processor architecture are you using (`go env`)?

linux amd64

What did you do?

If possible, provide a recipe for reproducing the error.
A complete runnable program is good.
A link on play.golang.org is best.

Ran go test -coverpkg=all ./... with the following folder structure:

├── foo.go
├── foo_test.go
└── tests
    └── tests_test.go

Content of files is irrelevant.
Apparently go test -cover ./... works, so it's specific to -coverpkg
Also deleting foo_test.go works

What did you expect to see?

ok      _/<HOME>/go/src/foo   0.001s  coverage: 0.0% of statements [no tests to run]
ok      _/<HOME>/go/src/foo/tests     0.004s  coverage: 0.0% of statements [no tests to run]

What did you see instead?

go build _/<HOME>/go/src/foo/tests: no non-test Go files in go/src/foo/tests
FAIL    _/<HOME>/go/src/foo [build failed]
ok      _/<HOME>/go/src/foo/tests     0.001s  coverage: 0.0% of statements in all [no tests to run]

The text was updated successfully, but these errors were encountered:

FiloSottile · 2018-08-30T23:00:35Z

/cc @bcmills @rsc

vvakame · 2019-02-27T06:31:09Z

I met this issue in real world.
This issue is very painful...
because, unrelated packages will be build failed without an error message...
I found go build go.mercari.io/datastore/testbed: no non-test Go files in /go/src/go.mercari.io/datastore/testbed message, but I can't found relation between this message to build faled. and I see it occasionally, but I never had a problem until now.

https://github.com/mercari/datastore/tree/v1.4.0/testbed
https://circleci.com/gh/mercari/datastore/219

$ go test go.mercari.io/datastore go.mercari.io/datastore/aedatastore go.mercari.io/datastore/aeprodtest go.mercari.io/datastore/boom go.mercari.io/datastore/clouddatastore go.mercari.io/datastore/dsmiddleware go.mercari.io/datastore/dsmiddleware/aememcache go.mercari.io/datastore/dsmiddleware/chaosrpc go.mercari.io/datastore/dsmiddleware/dslog go.mercari.io/datastore/dsmiddleware/fishbone go.mercari.io/datastore/dsmiddleware/localcache go.mercari.io/datastore/dsmiddleware/noop go.mercari.io/datastore/dsmiddleware/rediscache go.mercari.io/datastore/dsmiddleware/rpcretry go.mercari.io/datastore/dsmiddleware/storagecache go.mercari.io/datastore/internal go.mercari.io/datastore/internal/shared go.mercari.io/datastore/internal/testutils go.mercari.io/datastore/testbed go.mercari.io/datastore/testsuite go.mercari.io/datastore/testsuite/dsmiddleware/dslog go.mercari.io/datastore/testsuite/dsmiddleware/fishbone go.mercari.io/datastore/testsuite/dsmiddleware/localcache go.mercari.io/datastore/testsuite/dsmiddleware/rpcretry go.mercari.io/datastore/testsuite/favcliptools go.mercari.io/datastore/testsuite/realworld/recursivebatch go.mercari.io/datastore/testsuite/realworld/tbf -count 1 -p 1 -coverpkg=go.mercari.io/datastore/... -covermode=atomic -coverprofile=coverage.txt
go build go.mercari.io/datastore/testbed: no non-test Go files in /go/src/go.mercari.io/datastore/testbed
FAIL	go.mercari.io/datastore [build failed]
FAIL	go.mercari.io/datastore/aedatastore [build failed]
?   	go.mercari.io/datastore/aeprodtest	[no test files]
FAIL	go.mercari.io/datastore/boom [build failed]
FAIL	go.mercari.io/datastore/clouddatastore [build failed]
?   	go.mercari.io/datastore/dsmiddleware	[no test files]
FAIL	go.mercari.io/datastore/dsmiddleware/aememcache [build failed]
FAIL	go.mercari.io/datastore/dsmiddleware/chaosrpc [build failed]
FAIL	go.mercari.io/datastore/dsmiddleware/dslog [build failed]
FAIL	go.mercari.io/datastore/dsmiddleware/fishbone [build failed]
FAIL	go.mercari.io/datastore/dsmiddleware/localcache [build failed]
?   	go.mercari.io/datastore/dsmiddleware/noop	[no test files]
FAIL	go.mercari.io/datastore/dsmiddleware/rediscache [build failed]
FAIL	go.mercari.io/datastore/dsmiddleware/rpcretry [build failed]
FAIL	go.mercari.io/datastore/dsmiddleware/storagecache [build failed]
?   	go.mercari.io/datastore/internal	[no test files]
?   	go.mercari.io/datastore/internal/shared	[no test files]
?   	go.mercari.io/datastore/internal/testutils	[no test files]
ok  	go.mercari.io/datastore/testbed	6.887s	coverage: 0.7% of statements in go.mercari.io/datastore/...
?   	go.mercari.io/datastore/testsuite	[no test files]
?   	go.mercari.io/datastore/testsuite/dsmiddleware/dslog	[no test files]
?   	go.mercari.io/datastore/testsuite/dsmiddleware/fishbone	[no test files]
?   	go.mercari.io/datastore/testsuite/dsmiddleware/localcache	[no test files]
?   	go.mercari.io/datastore/testsuite/dsmiddleware/rpcretry	[no test files]
?   	go.mercari.io/datastore/testsuite/favcliptools	[no test files]
?   	go.mercari.io/datastore/testsuite/realworld/recursivebatch	[no test files]
?   	go.mercari.io/datastore/testsuite/realworld/tbf	[no test files]

gopherbot · 2019-02-28T05:52:38Z

Change https://golang.org/cl/164198 mentions this issue: cmd/go: ignore test only packages with -coverpkg

BinaryHexer · 2019-03-23T17:51:34Z

This is something that has just bit me as well.

Anyways if anyone is stuck because of this like me, here is tiny workaround:

go test -v -race -coverpkg=$(go list ./... grep -v "/test") -coverprofile=coverage.txt $(go list ./...)

The above workaround only works when your directory structure is as follows:

├── foo.go
└── test
    └── foo_test.go
├── bar.go
└── test
    └── bar_test.go

That is your test only package are always inside a directory named test.
If you have different naming style please modify the grep -v "/test" to suit your use case.

byxor · 2019-04-29T18:48:45Z

I found a tool called go-acc that solved this problem for me. 🎉

# Download and install go-acc
$ go get -u github.com/ory/go-acc

# Run it against some package(s)
$ go-acc github.com/some/package
$ go-acc .
$ go-acc ./...
$ go-acc $(glide novendor)

Here's the blogpost I found this tool on: https://www.ory.sh/golang-go-code-coverage-accurate

Thanks ory; if you see this, you saved me hours of pain.

Add a stub file for golang/go#27333

renld · 2020-04-22T05:11:14Z

when i use go1.14, it appears too..

What version of Go are you using (`go version`)?

$ go version
go version go1.14.1 linux/amd64

Does this issue reproduce with the latest release?

not try...

What operating system and processor architecture are you using (`go env`)?

go env Output

$ go env
GOHOSTARCH="amd64"
GOHOSTOS="linux"
GOARCH="amd64"

What did you do?

example
/

main.go
main_test.go
pkg1/a_test.go
pkg2/b.go
pkg2/b_test.go

go test -gcflags=-l -test.coverpkg=./... ./pkg1 => test build ok
go test -gcflags=-l ./... => test build ok
go test -gcflags=-l -test.coverpkg=./... ./... => test build fail when some sub dir only contains _test.go

after rm pkg1/a_test.go
/

main.go
main_test.go
pkg2/b.go
pkg2/b_test.go
go test -gcflags=-l -test.coverpkg=./... ./... => test build ok

What did you expect to see?

go test -gcflags=-l -test.coverpkg=./... ./... test build ok

What did you see instead?

go test -gcflags=-l -test.coverpkg=./... ./... [build failed] ( include pkg2)

hitzhangjie · 2020-08-27T08:32:32Z

go test -v -race -coverpkg=$(go list ./... grep -v "/test") -coverprofile=coverage.txt $(go list ./...)

I just come across this problem as well.

I tested with GOTMPDIR=. go test ...... , then 2 tmpdir is generated under current directory. I inspect the directory content and find that:
for each package something like 'hello', it will generate 3 files:

_testmain.go
importcfg.link
a built binary hello.test

I think the hello.test is built upon _testmain.go.

In _testmain.go, there's some function calls like coverRegisterFile(path/to/some.go).

Apparently, it registers some duplicate go files that doesn't import in current package 'hello'.

If coverRegisterFile(path/go/some.go is a file represents *.pb.go, go test -cover panics.

Maybe the flags passed to go test caused the duplicate coverRegisterFile statements.

I use this for i in go list ./...; do go test -v -coverpkg=$i -covermode=count -coverprofile=cover.out 2>&1 | tee -a report.out; done to work around this problem.

itsmeadi · 2020-10-14T18:43:38Z

The coverpkg expects comma seperated package names, so the workaround that worked for me
go test ./... -coverpkg=$(go list ./... | grep -v test | tr "\n" ",") -coverprofile=coverage.out assuming your test package name contains test as a substring

rspechenkin · 2020-11-17T11:24:48Z

Hello, any chance that this will be fixed anytime soon? This issue if very painful for us

elichai · 2021-01-05T15:49:55Z

Any updates here? I'm also getting different results when using @itsmeadi suggestion

djboris9 · 2021-01-26T12:35:41Z

We can reproduce it with the following setup and the workaround we use is written below. It might help @elichai and others.
In this scenario a parent test-only package affects the subpackage.

Reproduction

$ cat go.mod 
module example.com/repro

go 1.15

$ cat provoke_test.go 
package provoke

import "testing"

func TestHelloWorld(t *testing.T) {
	// t.Fatal("not implemented")
}

$ cat foo/foo.go 
package foo

func bar() string {
	return "bar"
}

$ cat foo/foo_test.go 
package foo

import "testing"

func TestBar(t *testing.T) {
	r := bar()
	if r != "bar" {
		t.Error("Bar is not bar")
	}
}

So we have a tree:

.
├── foo
│   ├── foo.go
│   └── foo_test.go
├── go.mod
└── provoke_test.go

The coverage test fails with the following message:

$ go test -v -coverpkg=$(go list -m)/... -coverprofile=profile.cov $(go list -m)/...
go build example.com/repro: no non-test Go files in /home/djurdjevicb/repro
=== RUN   TestHelloWorld
--- PASS: TestHelloWorld (0.00s)
PASS
coverage: 0.0% of statements in example.com/repro/...
ok  	example.com/repro	0.002s	coverage: 0.0% of statements in example.com/repro/...
FAIL	example.com/repro/foo [build failed]
FAIL

This message above suggests that the error is caused by sub package foo as build failed. Which is not the case.

Workaround

The workaround for the example above is to create an empty package for provoke so it's not a "test only package" anymore:

$ echo "package provoke" > provoke.go

$ go test -v -coverpkg=$(go list -m)/... -coverprofile=profile.cov $(go list -m)/...
=== RUN   TestHelloWorld
--- PASS: TestHelloWorld (0.00s)
PASS
coverage: 0.0% of statements in example.com/repro/...
ok  	example.com/repro	0.014s	coverage: 0.0% of statements in example.com/repro/...
=== RUN   TestBar
--- PASS: TestBar (0.00s)
PASS
coverage: 100.0% of statements in example.com/repro/...
ok  	example.com/repro/foo	0.001s	coverage: 100.0% of statements in example.com/repro/...

This worked, as every package has at least one non-test file:

.
├── foo
│   ├── foo.go
│   └── foo_test.go
├── go.mod
├── provoke.go
└── provoke_test.go

gopherbot · 2021-01-30T11:45:59Z

Change https://golang.org/cl/288292 mentions this issue: cmd/go: make -coverpkg=all skip test-only packages

mvdan · 2021-01-30T11:50:32Z

I've sent a fix above, which will presumably make it into Go 1.17. Until then, I agree with previous posters that the easiest workaround is to avoid test-only packages by dropping a no-op non-test file in those packages. For example: echo "package foo" >foo/foo.go. That workaround can be undone when the fix above ships in a stable release.

elichai · 2021-01-30T16:07:27Z

@mvdan what's the best way to check if it solves the problems? (I know how to install tip but not how to install a compiler from an open PR )

seankhliao · 2021-01-30T16:26:52Z

@elichai gotip download <CL number> eg: gotip download 288292

Adds a workaround for golang/go#27333 to make `make coverage` work.

elichai · 2021-02-02T09:33:03Z

@seankhliao it Works!
Output on go version go1.15.7 linux/amd64: https://termbin.com/g9of
Output on go version devel +c267ca1450 Sat Jan 30 11:43:57 2021 +0000 linux/amd64: https://termbin.com/gjsr

…#1480) * Add dummy go files for test only package, to mitigate golang/go#27333 * Stop ignoring errors when producing the coverage * Add comments explaining the dummy go files * Make the coverage output non-json

Add a dummy file so that we can measure coverage for the whole project Check golang/go#27333.

Separate the loading of repo definitions from JSON file from `LoadRepositories()` to a standalone function `loadRepositoriesFromFile()`, to make it easy to reuse it in the future. Add unit tests for `LoadRepositories()` function. Exclude github.com/osbuild/osbuild-composer/internal/rpmmd/test package from test coverage. Package with just tests and no other code makes `go test` to fail. This should be fixed in go 1.17. See golang/go#27333 Signed-off-by: Tomas Hozza <[email protected]>

FiloSottile added the NeedsInvestigation Someone must examine and confirm this is a valid issue and not a duplicate of an existing one. label Aug 30, 2018

FiloSottile added this to the Go1.12 milestone Aug 30, 2018

bcmills modified the milestones: Go1.12, Go1.13 Nov 15, 2018

andybons modified the milestones: Go1.13, Go1.14 Jul 8, 2019

rsc modified the milestones: Go1.14, Backlog Oct 9, 2019

ferhatelmas mentioned this issue Mar 5, 2020

Make workaround for go coverpkg bug GetStream/machinery#19

Merged

ferhatelmas added a commit to GetStream/machinery that referenced this issue Mar 5, 2020

Make workaround for go coverpkg bug (#19)

d5cd52d

Add a stub file for golang/go#27333

jayconrod mentioned this issue Dec 17, 2020

cmd/go: test with coverage unexpect fail #43242

Closed

forsaken628 mentioned this issue Dec 18, 2020

cmd/go: exclude empty cover package #43243

Closed

2uasimojo added a commit to 2uasimojo/aws-account-operator that referenced this issue Feb 1, 2021

Fix make coverage

64b7c8a

Adds a workaround for golang/go#27333 to make `make coverage` work.

2uasimojo mentioned this issue Feb 1, 2021

Fix make coverage openshift/aws-account-operator#511

Merged

elichai added a commit to kaspanet/kaspad that referenced this issue Feb 2, 2021

Add dummy go files for test only package, to mitigate golang/go#27333

b37911b

elichai mentioned this issue Feb 2, 2021

Add dummy go files for test only package, to mitigate golang/go#27333 kaspanet/kaspad#1480

Merged

gopherbot closed this as completed in c59b17e Mar 23, 2021

didrocks added a commit to ubuntu/adsys that referenced this issue Apr 9, 2021

Workaround a coverage go bug for test only package

1f93d98

Add a dummy file so that we can measure coverage for the whole project Check golang/go#27333.

golang locked and limited conversation to collaborators Mar 23, 2022

gopherbot added the FrozenDueToAge label Mar 23, 2022

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

cmd/go: test coverpkg=all ./... with test only packages will fail to build #27333

cmd/go: test coverpkg=all ./... with test only packages will fail to build #27333

AlexRouSg commented Aug 29, 2018 •

edited

Loading

FiloSottile commented Aug 30, 2018

vvakame commented Feb 27, 2019

gopherbot commented Feb 28, 2019

BinaryHexer commented Mar 23, 2019

byxor commented Apr 29, 2019 •

edited

Loading

renld commented Apr 22, 2020 •

edited

Loading

hitzhangjie commented Aug 27, 2020 •

edited

Loading

itsmeadi commented Oct 14, 2020

rspechenkin commented Nov 17, 2020

elichai commented Jan 5, 2021

djboris9 commented Jan 26, 2021

gopherbot commented Jan 30, 2021

mvdan commented Jan 30, 2021 •

edited

Loading

elichai commented Jan 30, 2021

seankhliao commented Jan 30, 2021

elichai commented Feb 2, 2021

cmd/go: test coverpkg=all ./... with test only packages will fail to build #27333

cmd/go: test coverpkg=all ./... with test only packages will fail to build #27333

Comments

AlexRouSg commented Aug 29, 2018 • edited Loading

What version of Go are you using (go version)?

Does this issue reproduce with the latest release?

What operating system and processor architecture are you using (go env)?

What did you do?

What did you expect to see?

What did you see instead?

FiloSottile commented Aug 30, 2018

vvakame commented Feb 27, 2019

gopherbot commented Feb 28, 2019

BinaryHexer commented Mar 23, 2019

byxor commented Apr 29, 2019 • edited Loading

renld commented Apr 22, 2020 • edited Loading

What version of Go are you using (go version)?

Does this issue reproduce with the latest release?

What operating system and processor architecture are you using (go env)?

What did you do?

What did you expect to see?

What did you see instead?

hitzhangjie commented Aug 27, 2020 • edited Loading

itsmeadi commented Oct 14, 2020

rspechenkin commented Nov 17, 2020

elichai commented Jan 5, 2021

djboris9 commented Jan 26, 2021

Reproduction

Workaround

gopherbot commented Jan 30, 2021

mvdan commented Jan 30, 2021 • edited Loading

elichai commented Jan 30, 2021

seankhliao commented Jan 30, 2021

elichai commented Feb 2, 2021

AlexRouSg commented Aug 29, 2018 •

edited

Loading

What version of Go are you using (`go version`)?

What operating system and processor architecture are you using (`go env`)?

byxor commented Apr 29, 2019 •

edited

Loading

renld commented Apr 22, 2020 •

edited

Loading

What version of Go are you using (`go version`)?

What operating system and processor architecture are you using (`go env`)?

hitzhangjie commented Aug 27, 2020 •

edited

Loading

mvdan commented Jan 30, 2021 •

edited

Loading