Coverage of go test

[GinFungYJF]

Does the go rule still have no support for calculating the coverage of go test now? If so, is there any way to calculate it?

[yugui]

It does not support coverage right now.
Probably we need to wait for bazelbuild/bazel#1118.

[GinFungYJF]

Any update on this? Or is there any usable workaround like
bazelbuild/bazel#1118 ?

[pmbethe09]

AFAIK no one is working on this. If it is something you need, you may have to be the one to implement it.

…

On Thu, Dec 1, 2016 at 7:18 AM, GinFungYJF ***@***.***> wrote: Any update on this? Or is there any usable workaround like bazelbuild/bazel#1118 <bazelbuild/bazel#1118> ? — You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub <#140 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/ALF-0gypcrlX564I8vd_Om9fLXppZ3mAks5rDrskgaJpZM4KVagd> .

[ghasemloo]

I think we can fix this directly, go tool itself support coverage independent of bazel.
I think what needs to be done is adding something similar to the block regarding coverage from go test main (https://golang.org/src/cmd/go/test.go?h=cover+flag#L1522)
to generate test main template: (https://github.com/bazelbuild/rules_go/blob/master/go/tools/generate_test_main.go#L129).

I tried it but I don't fully understand how flags are parsed in the go test, they are defined in
https://golang.org/src/cmd/go/testflag.go

[GinFungYJF]

@ghasemloo When I run go test -cover -x pkg, I see that it will do some preproccessing for the files that are not *_test.go files in the pkg before they are compiled.

go/pkg/tool/linux_amd64/cover -mode set -var GoCover_0 -o .A.go pkg/A.go
go/pkg/tool/linux_amd64/cover -mode set -var GoCover_0 -o .B.go pkg/B.go
...

Is it helpful to you?

[linuxerwang]

It would be great if the future bazel-go-coverage also supports external-test besides standard unit tests. Here are a few references of this technique:

https://blog.cloudflare.com/go-coverage-with-external-tests/
https://husobee.github.io/golang/test/coverage/2015/11/17/external-test-coverage.html
https://www.elastic.co/blog/code-coverage-for-your-golang-system-tests

I hope I could find time to work on it.

[pmbethe09]

@linuxerwang -- sounds great. Feel free to send some ideas here and then of course send a PR .

…

On Sat, Jan 21, 2017 at 7:41 AM, linuxerwang ***@***.***> wrote: It would be great if the future bazel-go-coverage also supports external-test besides standard unit tests. Here are a few references of this technique: https://blog.cloudflare.com/go-coverage-with-external-tests/ https://husobee.github.io/golang/test/coverage/2015/11/ 17/external-test-coverage.html https://www.elastic.co/blog/code-coverage-for-your-golang-system-tests I hope I could find time to work on it. — You are receiving this because you commented. Reply to this email directly, view it on GitHub <#140 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/ALF-0kVhZC4CZ0nFDvf92v2h8TbnemMdks5rUfz1gaJpZM4KVagd> .

[GinFungYJF]

Anybody has any idea about implementing it ? I don't know where to start :(

[jayconrod]

@GinFungYJF We haven't given this a lot of thought. It seems clear this will be a lot of work though.

Bazel has some support for exposing coverage information to Skylark. ctx.coverage_instrumented() would be called inside go_library_impl to check whether coverage instrumentation is needed. An instrumented_files provider should be returned (I'm not sure how Bazel uses that information though).

The Go code itself needs to be modified to include coverage instrumentation. The cover story provides some information on how this works. It looks like go test inserts coverage instrumentation at the source level before the compiler sees it. I'm not familiar with the implementation of go test, but test.go is the starting point. A lot of that implementation is private, and if it looks like we would need to duplicate it, we should talk to the Go Libraries team about providing a stable API we could use.

If you decide to work on this, please post a design doc that we can discuss and comment on.

[jayconrod]

@GinFungYJF I spoke with @alandonovan, and he filled in some gaps in my knowledge here.

go tool cover is the tool that adds coverage instrumentation to Go sources. That's already part of the standard distribution, so we wouldn't need to re-implement that. We'd just need to generate actions and files inside the rules to make sure that sources are processed with that before they are compiled when coverage is enabled.

Bazel expects coverage information to be in lcov format. We have some code in Blaze (Google-internal version of Bazel) that produces lcov data at the end of a Go program using the counters added by the instrumentation. This is linked into instrumented binaries. We should be able to open source this or re-implement it without too much work.

[linuxerwang]

@jayconrod Thank you for providing these info, although I've figured them out by myself. What held me back was actually I could not find a way to elegantly enable it in rules_go:

Suppose we have pkg deps like: A (main) -> B -> C. If we want to enable test coverage for A, we have to add corresponding BUILD targets also for B and C.

go_binary(name = "A",deps = ["B", "C"]
go_binary(name = "A_cover", test_coverage = true, deps = ["B_cover", "C_cover"])
go_library(name = "B")
go_library(name = "B_cover", test_coverage = true)
go_library(name = "C")
go_library(name = "C_cover", test_coverage = true)

This is too cumbersome. The ideal way is as follows:

go_binary(name = "A",deps = ["B", "C"]
go_test_coverage(name = "A_cover", deps = ["A"])

But it's probably impossible in bazel, since if A has been built before A_cover, A will not be triggered by A_cover to rebuild.

[pmbethe09]

This might be possible with aspects

https://bazel.build/versions/master/docs/skylark/aspects.html

[jayconrod]

@linuxerwang Ideally, you shouldn't need to define separate targets with and without coverage; it should be a configuration option. Bazel already has a coverage command that runs tests in coverage mode. We just need to wire up the rules to instrument code when that option is enabled and generate output in the appropriate format.

[linuxerwang]

@pmbethe09, @jayconrod Cool! I'll invest some time on it this weekend.