Skip to content

Commit a3d8483

Browse files
Add NUnit Test handling for PSScriptAnalyzer results (#211)
This will now convert the output of `Invoke-ScriptAnalyzer` into an NUnitXml file that can then be published through the CI system so that we can fail on ScriptAnalyzer failures, as well as easily see them in the test output. Got the initial idea for this approach from @MathieuBuisson's [blog post](https://mathieubuisson.github.io/psscriptanalyzer-first-class-citizen/) (which directly converted the ScriptAnalyzer results into an NUnit XML file, but only had a single passing test for the success scenario). I then combined that idea with one from a [Dr. Scripto post](https://devblogs.microsoft.com/scripting/psscriptanalyzer-deep-dive-part-3-of-4/) which used Pester to invoke PSScriptAnalyzer, ensuring that there was a test created for each possible Rule. The problem with the Dr. Scripto approach was that multiple failures of the same rule still resulted in a single failure, and the test output lost the actual message from ScriptAnalyzer. I ended up implementing this differently than Mathieu, directly working with the XML class to build up the XML document, but really appreciated the initial ideas that he had with this approach. Sample build test result when there was a Script Analysis failure: https://dev.azure.com/ms/PowerShellForGitHub/_build/results?buildId=84441&view=ms.vss-test-web.build-test-results-tab
1 parent b9e1708 commit a3d8483

File tree

2 files changed

+492
-0
lines changed

2 files changed

+492
-0
lines changed

build/pipelines/templates/run-staticAnalysis.yaml

Lines changed: 14 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -11,4 +11,18 @@ steps:
1111
- powershell: |
1212
$results = Invoke-ScriptAnalyzer -Path ./ –Recurse
1313
$results | ForEach-Object { Write-Host "##vso[task.logissue type=$($_.Severity);sourcepath=$($_.ScriptPath);linenumber=$($_.Line);columnnumber=$($_.Column);]$($_.Message)" }
14+
15+
$null = New-Item -Path ..\ -Name ScriptAnalyzer -ItemType Directory -Force
16+
./build/scripts/ConvertTo-NUnitXml.ps1 -ScriptAnalyzerResult $results -Path ../ScriptAnalyzer/test-results.xml
17+
workingDirectory: '$(System.DefaultWorkingDirectory)'
1418
displayName: 'Run Static Code Analysis (PSScriptAnalyzer)'
19+
20+
- task: PublishTestResults@2
21+
displayName: 'Publish ScriptAnalyzer Test Results'
22+
inputs:
23+
testRunTitle: 'Windows Test Results for PSScriptAnalyzer'
24+
buildPlatform: 'Windows'
25+
testRunner: NUnit
26+
testResultsFiles: '../ScriptAnalyzer/test-results.xml'
27+
failTaskOnFailedTests: true # required to fail build when tests fail
28+
condition: succeededOrFailed()

0 commit comments

Comments
 (0)