coverlet.console works locally but generates NaN% in Cloud build

[agentq15]

We have a .NET Framework project.
When I run this powershell script locally everything works as expected, but in Azure Devops it only generates NaN%

$coverlet = "$pwd\coverlet.exe" & $coverlet $unitTestFile --target "dotnet" --targetargs "test $($unitTestFile) --no-build" --format "cobertura" --verbosity detailed gci -Recurse | ?{ $_.Name -eq "coverage.cobertura.xml"} | %{ &"$pwd\reportgenerator.exe" "-reports:$($_.FullName)" "-targetdir:reports" "-reportstypes:HTMLInline;HTMLChart" }

We don't use Deterministic Build

Result when run in Azure Hosted Agent:

Result when run in on premise Agent

Why isn't it working in the Cloud?

[MarcoRossignoli]

Likely you're hitting a known issue https://github.com/coverlet-coverage/coverlet/blob/master/Documentation/KnownIssues.md#1-vstest-stops-process-execution-earlydotnet-test
Can you try with collectors https://github.com/coverlet-coverage/coverlet/blob/master/Documentation/VSTestIntegration.md(it's not fully supported, there are some scenario where it fails for known reason)?

[agentq15]

Likely you're hitting a known issue https://github.com/coverlet-coverage/coverlet/blob/master/Documentation/KnownIssues.md#1-vstest-stops-process-execution-earlydotnet-test
Can you try with collectors https://github.com/coverlet-coverage/coverlet/blob/master/Documentation/VSTestIntegration.md(it's not fully supported, there are some scenario where it fails for known reason)?

Collector only works for .NET Core but we use .NET Framework. Any other suggestions?

[MarcoRossignoli]

Collector only works for .NET Core but we use .NET Framework. Any other suggestions?

It depends on project(sometimes it works thanks to same type forward between core/netfx), have you tried?
(Waiting to fix it microsoft/vstest#2278 (comment) but need some green light by vstest team, I don't want to risk to break everything)

[agentq15]

Tried now and have added the package but I get this
Data collection : Unable to find a datacollector with friendly name 'XPlat Code Coverage'. Data collection : Could not find data collector 'XPlat Code Coverage'
Don't know if I'm doing something wrong.

[MarcoRossignoli]

Can you attach the command line pls?

[agentq15]

This doesn't do anything
dotnet test --collect:"XPlat Code Coverage"

So this is what got me the message above:
dotnet vstest <path>\Test\bin\Debug\SDS.App.Test.dll --collect:"XPlat Code Coverage"

[MarcoRossignoli]

Are you using new project style or old one? https://docs.microsoft.com/en-us/visualstudio/msbuild/how-to-use-project-sdk?view=vs-2019

[agentq15]

Old style, unfortunately

[MarcoRossignoli]

Ah coverlet is fully supported only on new project style, apologize but for now you need to wait for this fix microsoft/vstest#2278 (comment)
There is a known issue where vstest plat kills test process early and won't let coverlet to gather the hits on files. The only way to fix it is to use collectors that are integrated in vstest pipeline an so vstest plat will wait collectors to complete it's job.
This is totally randomic, I mean it's a timing issue, could work on you host and not in others, all related to the speeed of tests, we got some issue with for instance for tests uses RabbitMQ, it slows down the process shutdown and lead to early test process kill.
To be sure of that we can enable logs https://github.com/coverlet-coverage/coverlet/blob/master/Documentation/Troubleshooting.md#coverlet-global-tool
If you see some Hits file:'C:\Users\Marco\AppData\Local\Temp\xxxxxx_703263e9-21f0-4d1c-9ce3-98ddeacecc01' not found for module: xxx mean that or there is no test for that assembly or the process has been killed before persist data(vstest behaviour out of our control).

[MarcoRossignoli]

Closed in #970