Fix coverage aggregation from multiple reports

[pavel-mikula-sonarsource]

Description

On sonar-dotnet build itself, we produce several coverage reports at the moment, each from a dedicated test project (here are the relevant fragments):

• coverage.DotNetCoverageNet7.net7.0-windows.xml.txt
• coverage.DotNetCoverageNet8.xml.txt
• coverage.DotNetCoverageTestFramework.xml.txt
• coverage.DotNetCoverageNet48.net48.xml.txt

These gets all imported into coverage sensor and processed via our aggregation logic. For the production code in SonarAnalyzer.TestFrameworkVerificationIssueValidationIssueLocationPair.cs and line 68

 else if (Expected is null)

this produces 2 out of 3 covered conditions that don't make sense.

We need to fix the aggregation.

Situation

The affected production code is part of our test framework. It is tested:

• Directly on all paths via coverage.DotNetCoverageTestFramework.xml.txt
• Indirectly, mostly only on happy paths via coverage.DotNetCoverageNet7.net7.0-windows.xml.txt and coverage.DotNetCoverageNet48.net48.xml.txt
• Not at all via coverage.DotNetCoverageNet8.xml.txt, because it doesn't reach this error state in the current set of tests

Coverage data

Coverage data contains expected values like this:

coverage.DotNetCoverageTestFramework.xml.txt

Indicates that there are 2 branches and both are visited

                <SequencePoint vc="70" uspid="198" ordinal="2" offset="54" sl="68" sc="14" el="68" ec="35" bec="2" bev="2" fileid="940" />
...
                <BranchPoint vc="52" uspid="204" ordinal="4" offset="60" sl="68" path="0" offsetend="65" fileid="940" />
                <BranchPoint vc="18" uspid="205" ordinal="5" offset="60" sl="68" path="1" offsetend="236" fileid="940" />

coverage.DotNetCoverageNet7.net7.0-windows.xml.txt and it's net48 counterpart

Indicates that there are 2 branches, and one of them is visited - this is expected.

                <SequencePoint vc="2" uspid="198" ordinal="2" offset="54" sl="68" sc="14" el="68" ec="35" bec="2" bev="1" fileid="940" />
...
                <BranchPoint vc="2" uspid="204" ordinal="4" offset="60" sl="68" path="0" offsetend="65" fileid="940" />
                <BranchPoint vc="0" uspid="205" ordinal="5" offset="60" sl="68" path="1" offsetend="236" fileid="940" />

coverage.DotNetCoverageNet8.xml.txt

Indicates that there are 2 branches and none of them is visited from this test run - this is also expected

                <SequencePoint vc="0" uspid="198" ordinal="2" offset="54" sl="68" sc="14" el="68" ec="35" bec="2" bev="0" fileid="940" />
...
                <BranchPoint vc="0" uspid="204" ordinal="4" offset="60" sl="68" path="0" offsetend="65" fileid="940" />
                <BranchPoint vc="0" uspid="205" ordinal="5" offset="60" sl="68" path="1" offsetend="236" fileid="940" />

[Jetski5822]

Seems I am having this issues too - with dotcover, when importing with net6.0 and net8.0 coverage goes to zero.

[andrei-epure-sonarsource]

The code coverage parsers and importers and aggregators are in https://github.com/SonarSource/sonar-dotnet/tree/master/sonar-dotnet-shared-library/src/main/java/org/sonar/plugins/dotnet/tests

Looking at https://sonarcloud.io/code?id=sonaranalyzer-dotnet&selected=sonaranalyzer-dotnet%3Aanalyzers%2Ftests%2FSonarAnalyzer.TestFramework%2FVerification%2FIssueValidation%2FIssueLocationPair.cs&line=68 at this moment in time, it appears to be fixed...

[sebastien-marichal]

#9018 could have improved coverage results.

[pavel-mikula-sonarsource]

#9018 could have improved coverage results.

I would say "It hid the problem again"

[Tim-Pohlmann]

The problem is not with the aggregation but with AltCover reporting differently for different framework versions:
SteveGilham/altcover#216

This issue is blocked until AltCover's behavior is changed (which might not happen; I'm not sure it's a bug) or we find a better way to match BranchPoints.