Unexpected Results for ThresholdStat=total

[leleSahil]

I am trying to use the /p:ThresholdStat option to specify that the total coverage not dip below a certain number and I am seeing weird results

+----------------------------------------+--------+--------+--------+
| Module | Line | Branch | Method |
+----------------------------------------+--------+--------+--------+
| Module 1 | 26.1% | 21.6% | 46.6% |
+----------------------------------------+--------+--------+--------+
| Module 2 | 68.4% | 54.2% | 70.8% |
+----------------------------------------+--------+--------+--------+

/Users/sahil/.nuget/packages/coverlet.msbuild/2.3.0/build/netstandard2.0/coverlet.msbuild.targets(25,5): error : 'Module 1' has a method coverage '46.6%' below specified threshold '58%'

Here is the command I am using to run this:
dotnet test *********.csproj /p:CollectCoverage=true /p:CoverletOutputFormat=cobertura /p:Threshold=58 /p:ThresholdType=method /p:ThresholdStat=total

The total method coverage definitely exceeds 58% is there something I am missing?

[leleSahil]

Additionally is there an option to exclude a module from coverage results and thresholding?

[MarcoRossignoli]

Additionally is there an option to exclude a module from coverage results and thresholding?

Not sure to understand well, but you can filter out module with filter https://github.com/tonerdo/coverlet#filters-1 with /p:Exclude

[leleSahil]

I am referring to this option from the documentation:

By default, Coverlet will validate the threshold value against the coverage result of each module. The /p:ThresholdStat option allows you to change this behaviour and can have any of the following values:

Minimum (Default): Ensures the coverage result of each module isn't less than the threshold
Total: Ensures the total combined coverage result of all modules isn't less than the threshold
Average: Ensures the average coverage result of all modules isn't less than the threshold
The following command will compare the threshold value with the overall total coverage of all modules:

[tonerdo]

What does the Total in the terminal summary output read?

[NicoDujde]

I am having a similar issue than @leleSahil .

How could we apply the threshold to a specific Module (knowing the only options at the moment are Total, EveryModule or Average)?

i.e.:
3 product code projects with one of them being common code:

• Module1
• Module2
• CommonModule

3 test projects:

• Module1_Tests
• Module2_Tests
• CommonModule_Tests

Each test project is being run sequentially.
I would like to only use the threshold on the module I am testing not the Total, not everyModule and not the average.

What would be a way to do so?

[MarcoRossignoli]

@NicoDujde at the moment we don't have a way to put threshold on single module...you could extract from console output(if you're using msbuild/dotnet tool) with regex #148 #284 and write a script to validate your numbers.

[NicoDujde]

Would Include="[Module1]*" set the Total/Threshold based on that Module only?

I have not been successful understanding the use of Include yet.

i.e.:
dotnet test -c Release Module1_Test.csproj /p:Include="[Module1]*" /p:Threshold="80" /p:ThresholdType="line" /p:ThresholdStat="total" /p:CollectCoverage=true

[MarcoRossignoli]

Include mean instrument only that module/types and in that case parameters should applies to only that module

[NicoDujde]

Thank you. I got it to work locally and I am getting the expected behavior.

Unfortunately, when run on Jenkins, the Include parameter seems to be ignored. I need to play with the Jenkins script and see if adding ' or " and escaping them work.

Thank you for the prompt responses

[MarcoRossignoli]

@NicoDujde some similar issue #182 let me know how you resolve that maybe I could add "jenkins" escaping to guide(we already have one for poweshell)

[MarcoRossignoli]

any news?

[MarcoRossignoli]

Close for stale discussion, feel free to reopen if needed.