Code Coverage for multiple packages

[ajackson-cpi]

'go test cover' doesn't consider coverage via other packages' tests.
Please make it possible to use coverage tools that do, like:
https://github.com/haya14busa/goverage

[mattetti]

I would personally recommend to keep it simple for now. Also, the go recommended way is to have the tests with the code, what are the scenarios where you'd write tests in other packages that the original one?

[ajackson-cpi]

I'm building a simple REST server that reads from a database. Instead of reinventing DB processing for each call, most of the logic is in a "db-helpers" package.

To test both packages, I would need to copy-paste tests to both.
It would be far cleaner if the API tests could indicate that they tested the db-helpers package.

[gaffney]

Agreed, this is annoying... it has resulted in writing tests for the sake of coverage even though in reality the code in question is indirectly covered all over the place. Really slows down productivity and is an obstacle towards refactoring when you know you not only have to extract out all the common code, but you have to pull out all the common tests as well.

[ramya-rao-a]

We could add support for https://github.com/haya14busa/goverage, but keep it behind a setting, so that this a feature that users opt in to and is not the default one.

PRs are welcome

[AlekSi]

Go 1.10 supports go test -cover for multiple packages: https://golang.org/doc/go1.10#test Extending existing code to pass -coverpkg=all should be easy, no external tool is needed now.

[ramya-rao-a]

@mattetti, @AlekSi Could either of you try passing the -coverpkg=all flag via the go.testflags setting and share if any extra work is needed from the extension to achieve this?

[jirfag]

@ramya-rao-a I've added "go.testFlags": ["-coverpkg=all"], to settings, tests are running with this flag, but coverage is shown only in the tested package.

[ShoshinNikita]

@ramya-rao-a I've added "go.testFlags": ["-coverpkg=all"], to settings, tests are running with this flag, but coverage is shown only in the tested package.

I found the reason why coverage isn't shown. The bug is in applyCodeCoverageToAllEditors function in src/goCovers.ts file

Setup

Let's use this package structure (package is github.com/user/repo, root is /home/usr/go/src/github.com/user/repo)

|-- main.go
|-- first
    |-- first.go
|--second
    |-- second.go
    |-- second_test.go

Package first contains a function which is called in package second. When we run tests for package second (with -coverpkg=github.com/user/repo/...), we get the following cover profile:

mode: set
github.com/user/repo/first/first.go:5.16,7.2 1 1
github.com/user/repo/second/second.go:7.23,9.2 1 1

Bug

export function applyCodeCoverageToAllEditors(coverProfilePath: string, packageDirPath: string): Promise<void> {
  return new Promise((resolve, reject) => {
    try {

      // ...

      lines.on('line', function(data: string) {
          // go test coverageprofile generates output:
          //    filename:StartLine.StartColumn,EndLine.EndColumn Hits CoverCount
          // The first line will be "mode: set" which will be ignored
          const fileRange = data.match(/([^:]+)\:([\d]+)\.([\d]+)\,([\d]+)\.([\d]+)\s([\d]+)\s([\d]+)/);
          if (!fileRange) {
            return;
          }

          // => HERE
          const filePath = path.join(packageDirPath, path.basename(fileRange[1]));

          // ...
      }
    }
  }
}

Tests were run github.com/user/repo/second. So, packageDirPath is /home/usr/go/src/github.com/user/repo/second.

Let's examine processing of the cover profile file:

• First line (github.com/user/repo/first/first.go:5.16,7.2 1 1)

  • packageDirPath is /home/usr/go/src/github.com/user/repo/second
  • fileRange[1] is github.com/user/repo/first/first.go
  • path.basename(fileRange[1]) is first.go
  • filePath is /home/usr/go/src/github.com/user/repo/second/first.go

  There's no such file ❌ :(

• Second line (github.com/user/repo/second/second.go:7.23,9.2 1 1)

  • packageDirPath is /home/usr/go/src/github.com/user/repo/second
  • fileRange[1] is github.com/user/repo/second/second.go
  • path.basename(fileRange[1]) is second.go
  • filePath is /home/usr/go/src/github.com/user/repo/second/second.go

  There's such file ✔️

Solution

I think, the best solution is to pass the path of a project root instead of the path of tested package: /home/usr/go/src/github.com/user/repo instead of /home/usr/go/src/github.com/user/repo/second.

Then we should trim module path (github.com/user/repo, example.com/package and etc) from fileRange[1] instead of using path.basename() function.

Updated processing of the cover profile file:

• First line (github.com/user/repo/first/first.go:5.16,7.2 1 1)

  • packageRoot is /home/usr/go/src/github.com/user/repo
  • fileRange[1] is github.com/user/repo/first/first.go
  • fileRange[1].TrimPrefix(modulePath) is /first/first.go
  • filePath is /home/usr/go/src/github.com/user/repo/first/first.go

  There's such file ✔️ :)

• Second line (github.com/user/repo/second/second.go:7.23,9.2 1 1)

  • packageDirPath is /home/usr/go/src/github.com/user/repo
  • fileRange[1] is github.com/user/repo/second/second.go
  • fileRange[1].TrimPrefix(modulePath) is /second/second.go
  • filePath is /home/usr/go/src/github.com/user/repo/second/second.go

  There's such file ✔️

---

I tried to realize this, but I am not sure how to define Module Path (and I couldn't find any function for that). Also because of lack of experience with working on VS Code Extensions I am not sure how to pass project root (can we just use workspace root?)

I hope my little investigation will help to fix this bug :)