Data collection: Could not find data collector 'XPlat Code Coverage"

[itsryankwon]

Hello,

I can't show code due to it being for the company I work for but basically I am getting this error:

Data collection : Could not find data collector 'XPlat Code Coverage'

I saw this error on another post but I don't think my cause is the same.

What I'm doing to get this message:

dotnet vstest Tests.dll --collect:"XPlat Code Coverage" or
dotnet vstest Tests.dll --collect:"XPlat Code Coverage" --settings coverletArgs.runsettings

I tried using the runsettings file as well but that didn't change anything.

Output of dotnet --info:

.NET Core SDK (reflecting any global.json):
  Version:    2.2.401
  Commit:   7299b316c13

Runtime Environment:
  OS Name: centos
  OS Version: 7
  OS Platform: Linux
  RID: centos.7-x64
  Base Path:  /usr/share/dotnet/sdk/2.2.401/

Host (useful for support):
  Version: 2.2.6
  Commit: 7dac9b1b51

.NET Core SDKs installed:
  2.2.401 [/usr/share/dotnet/sdk]

.NET Core runtimes installed:
  Microsoft.AspNetCore.All 2.1.12 [/usr/share/dotnet/shared/Microsoft.AspNetCore.All]
  Microsoft.AspNetCore.All 2.2.6 [/usr/share/dotnet/shared/Microsoft.AspNetCore.All]
  Microsoft.AspNetCore.App 2.1.12 [/usr/share/dotnet/shared/Microsoft.AspNetCore.App]
  Microsoft.AspNetCore.App 2.2.6 [/usr/share/dotnet/shared/Microsoft.AspNetCore.App]
  Microsoft.NETCore.App 2.0.7 [/usr/share/dotnet/shared/Microsoft.NETCore.App]
  Microsoft.NETCore.App 2.1.12 [/usr/share/dotnet/shared/Microsoft.NETCore.App]
  Microsoft.NETCore.App 2.2.6 /usr/share/dotnet/shared/Microsoft.NETCore.App]

I have the following package references as well in my test projects:

<PackageReference Include="coverlet.collector" Version="1.0.1">
<PackageReference Include="Microsoft.NET.Test.Sdk" Version="16.1.0" />
<PackageReference Include="MSTest.TestAdapter" Version="1.3.2" />
<PackageRefernece Include="MSTest.TestFramework" Version="1.3.2" />

and <TargetFramework>netcoreapp2.2</TargetFramework>

Am I missing something or doing something wrong?

[MarcoRossignoli]

@vagisha-nidhi vstest is already supported? I remember #395 (comment)
got same issue on my simple local test

dotnet vstest bin\Debug\netcoreapp2.2\XUnitTestProject1.dll  --collect:"XPlat Code Coverage"
Microsoft (R) Test Execution Command Line Tool Version 16.2.0-preview-20190606-02
Copyright (c) Microsoft Corporation.  All rights reserved.

Starting test execution, please wait...
Data collection : Could not find data collector 'XPlat Code Coverage'

If not supported can you specify when will be supported?

[vagisha-nidhi]

In case of dotnet vstest, the coverlet.collector will be picked from the output directory(where the test dll resides). So in general, for dynamic code coverage also, we follow the pattern that first we do a dotnet publish which puts code coverage datacollector and the test dlls in the publish directory and then we run the tests from there.
The XPlat Code Coverage should run the same way.
Please try this. Run dotnet publish and check if the collector is present in the publish directory.
Then run the tests from the test dll present in this directory. Use the latest sdk. Do let me know if this doesn't work.

[MarcoRossignoli]

@vagisha-nidhi confirm that works.
@itsryankwon can you try to publish before?Something like:

C:\git\coverletissue\vstest
λ dotnet publish
Microsoft (R) Build Engine version 16.3.0-preview-19377-01+dd8019d9e for .NET Core
Copyright (C) Microsoft Corporation. All rights reserved.

  Restore completed in 408,06 ms for C:\git\coverletissue\vstest\vstest.csproj.
  You are using a preview version of .NET Core. See: https://aka.ms/dotnet-core-preview
  vstest -> C:\git\coverletissue\vstest\bin\Debug\netcoreapp3.0\vstest.dll
  vstest -> C:\git\coverletissue\vstest\bin\Debug\netcoreapp3.0\publish\

C:\git\coverletissue\vstest
λ dotnet vstest C:\git\coverletissue\vstest\bin\Debug\netcoreapp3.0\publish\vstest.dll --collect:"XPlat Code Coverage"
Microsoft (R) Test Execution Command Line Tool Version 16.3.0-preview-20190715-02
Copyright (c) Microsoft Corporation.  All rights reserved.

Starting test execution, please wait...

Attachments:
  C:\git\coverletissue\vstest\TestResults\f5fd42c2-77cc-4b82-ac0e-18c09f02cdc8\coverage.cobertura.xml
Test Run Successful.
Total tests: 1
     Passed: 1
 Total time: 1,3841 Seconds

[MarcoRossignoli]

@itsryankwon can you confirm that solution works for you?
If so feel free to close this issue.

[MarcoRossignoli]

@itsryankwon any news?
Feel free to close if resolved.

[MarcoRossignoli]

Close for stale feel free to re-open if needed.

[granadacoder]

I'll do a mini followup, since I just tackled this (with your suggestions! @marco)

For future readers.

I was getting the error:

Data collection: Could not find data collector 'XPlat Code Coverage"

....

(everything below is in the context of using a LINUX build machine/image, not windows....there is a difference !!)

So when you do a regular
dotnet build

you'll get an output like this:

\src\UnitTests\bin\Release\netcoreapp3.1\win-x64\MyProjectName.UnitTests.dll

At this point, if you were to look in the directory:

\src\UnitTests\bin\Release\netcoreapp3.1\win-x64\

you'll find NO "coverlet" dlls/files.

examples:

coverlet.core.dll
coverlet.collector.dll

What marco is saying here is "Yep, that's the issue, you don't have the magic dlls".
And to over come that...you "dotnet publish" your UnitTest project. ((<< this is a little bit contrary to what you usually do, which is publish your real-application layer.) (<<this is what was tripping me up initially).

So now if you "dotnet publish" your UNIT TEST (.csproj)....

You will get something like this: (emphasis on the last directory in the chain of "publish")

\src\UnitTests\bin\Release\netcoreapp3.1\win-x64\publish\MyProjectName.UnitTests.dll
Now to look in the directory

\src\UnitTests\bin\Release\netcoreapp2.1\win-x64\publish\
You should now see files like this:

\src\UnitTests\bin\Release\netcoreapp3.1\win-x64\publish\coverlet.core.dll
\src\UnitTests\bin\Release\netcoreapp3.1\win-x64\publish\coverlet.collector.dll

Don't focus on the exact complete paths, focus that the "publish" directory has the necessary coverlet files.

NOW, you run

dotnet test --collect:"XPlat Code Coverage" --logger:trx --ResultsDirectory:/MyTestResults "\src\UnitTests\bin\Release\netcoreapp3.1\win-x64\publish\MyProjectName.UnitTests.dll"
and NOW IT WORKS !!

Remember, "coverlet.core.dll" and "coverlet.collector.dll" won't show up by magic.
You need to add nuget references to your UnitTests.csproj (<<whatever your csproj name is)

I'll post my current (working) example below. Don't get too caught up in the versions (as long as they are not super old)...

  <ItemGroup>
    <PackageReference Include="FluentAssertions" Version="5.10.0" />
    <PackageReference Include="Microsoft.NET.Test.Sdk" Version="16.2.0" />
    <PackageReference Include="Moq" Version="4.13.1" />
    <PackageReference Include="MSTest.TestAdapter" Version="2.0.0" />
    <PackageReference Include="MSTest.TestFramework" Version="2.0.0" />
    <PackageReference Include="coverlet.collector" Version="1.2.0" />>
  </ItemGroup>

The FluentAssertions and Moq are not important to "coverlet" code coverage........but they were in my file, so I posted them.

The magic takeaway is

1. Have a PackageReference to "coverlet.collector" in your unit test csproj.
2. You can't get coverage from the results of "dotnet build", you need "dotnet publish" (to get the magic coverlet files in the output directory. "dotnet publish" of a unit test project is a little counter intuitive, but it is what you need here.
3. run your "dotnet test" on the myunittests.dll that is in the publish directory.

Thanks Marco ! Seeing coverage on the linux build image was awesome!

Final documentation note:

My "coverage" report came out at:

\MyTestResults\abcdabcd-abcd-abcd-abcd-abcdabcdabcd\coverage.cobertura.xml

where "abcdabcd-abcd-abcd-abcd-abcdabcdabcd" is SOME RANDOM GUID.

(the random guid subfolder is a different subject altogether, and is related to "dotnet test", NOT specifically this coverage collector.........and is outside the scope)

Final helpful breadcrumb (windows vs linux)

https://docs.microsoft.com/en-us/azure/devops/pipelines/ecosystems/dotnet-core?view=azure-devops

scroll down to “If you're building on Linux or macOS,”

This will give you some extra clues that "dotnet build", "dotnet test" and/or "dotnet publish" are not the exact same on linux.

[MarcoRossignoli]

@granadacoder thank's for your explanation....one more thing you should use last version of test sdk with collector greather than 1.0.0 because there is a bug in vstest repo for old version

<PackageReference Include="Microsoft.NET.Test.Sdk" Version="16.2.0" />

should be

<PackageReference Include="Microsoft.NET.Test.Sdk" Version="16.5.0" />

You don't see any issue because your code doesn't suffer of a particular timing issue https://github.com/tonerdo/coverlet/blob/master/Documentation/KnownIssues.md#2-upgrade-coverletcollector-to-version--100

Collector guide https://github.com/tonerdo/coverlet/blob/master/Documentation/VSTestIntegration.md

[RomanBadiornyi]

I wanted to share alternative to dotnet publish solution, which in some situations could be preferable, you can provide path to data collector as follows:
dotnet test "Build\**\*Tests.dll" --collect "XPlat Code Coverage" /testadapterpath:NUGET_PACKAGE_PATH\coverlet.collector\3.0.3\build\netstandard1.0

[mcartoixa]

FYI I am using MSBuild and I suffer from the same problem. I would build the tests before execution (using the MSBuild task) but the tests would not seem able to gather all the required files. But I happened to notice that for some reason it worked if I launched my script a second time without cleaning the outputs...
So I made sure sure that the projects were built twice in my scripts, the first time using an external process (with an Exec dotnet msbuild task because the MSBuild task would not do the trick). 🤷🏻‍♂️
So my guess is that if you are running the dotnet commands, running dotnet build before running the tests would work too (note that dotnet test compiles the projects in their default configuration so you should do the same).

[MarcoRossignoli]

msbuild integration uses this targets so it's also possible use part of it https://github.com/coverlet-coverage/coverlet/blob/master/src/coverlet.msbuild.tasks/coverlet.msbuild.targets

[RussKie]

I wanted to share alternative to dotnet publish solution, which in some situations could be preferable, you can provide path to data collector as follows: dotnet test "Build\**\*Tests.dll" --collect "XPlat Code Coverage" /testadapterpath:NUGET_PACKAGE_PATH\coverlet.collector\3.0.3\build\netstandard1.0

This should really be part of the documentation. After many wasted hours I arrived to the same solution, and I wish I found this thread sooner.

An additional tweak to this is to use GeneratePathProperty to avoid handcoding the name and the version of the package:

<PackageReference Include="coverlet.collector" GeneratePathProperty="true" />

Though it's worth noting, that the property may not be available immediately, and can only be accessed from a target, e.g.:

dotnet test D:\test\artifacts\bin\Sample.Tests\Debug\net7.0\Sample.Tests.dll --collect:"XPlat Code Coverage" --logger:trx --test-adapter-path:"$(Pkgcoverlet_collector)\build\netstandard1.0"

[captainjono]

@RussKie thanks for this info this should unblock me!

[Davidos533]

In my case, on PC, firstly i run tests projects with installed coverlet.collector in Visual Studio (from Test Explorer), and after running dotnet test --collect:"xplat code coverage"

[duyn9uyen]

I was missing the coverlet package reference in my .csproj in its entirety. Thanks @RussKie

<ItemGroup>
      <PackageReference Include="coverlet.collector" />
</ItemGroup>