PSScriptAnalyzer rule changes not sticky using PowerShell/vscode-powershell

[alx9r]

I understand that the fault may lie with PowerShell/vscode-powershell. I'm opening this bug here for the following reasons:

1. It reflects badly on PSScriptAnalyzer regardless of the cause.
2. PowerShell/vscode-powershell#899(comment) has already been asked and seems like should be straightforward for a developer familiar with the projects to answer. An answer to that question probably would have averted this issue. Despite this, that question on PowerShell/vscode-powershell has remained unanswered for more than 90 days.

Steps to reproduce

1. Fresh install of Visual Studio Code 1.22.2.
2. Install the PowerShell Extensions in Visual Studio Code.
3. Select "PowerShell: Select ScriptAnalyzer Rules" from the command pallette.
4. Note the state of the checkbox for the first item.
5. Toggle the state of the checkbox for the first item.
6. Click "Confirm".
7. Restart Visual Studio Code.
8. Select "PowerShell: Select ScriptAnalyzer Rules" from the command pallette, again.
9. Note the state of the checkbox for the first item.

Expected behavior

The item's checkbox state would stick across sessions.

Actual behavior

The item's checkbox state reverts back to the state immediately after installation of VS Code and PowerShell/vscode-powershell.

Environment data

> $PSVersionTable

Name                           Value                                      
----                           -----                                      
PSVersion                      5.1.14409.1012                             
PSEdition                      Desktop                                    
PSCompatibleVersions           {1.0, 2.0, 3.0, 4.0...}                    
BuildVersion                   10.0.14409.1012                            
CLRVersion                     4.0.30319.42000                            
WSManStackVersion              3.0                                        
PSRemotingProtocolVersion      2.3                                        
SerializationVersion           1.1.0.1                                    

> (Get-Module -ListAvailable PSScriptAnalyzer).Version | ForEach-Object { $_.ToString() }
1.16.1

ms-vscode.powershell
v1.6.0

VS Code
Version 1.22.2
Commit 3aeede733d9a3098f7b4bdc1f66b63b0f48c1ef9
Date 2018-04-12T17:28:16.777Z
Shell 1.7.12
Renderer 58.0.3029.110
Node 7.9.0
Architecture ia32

[bergmeister]

@alx9r Hi, thanks for the info. I was not aware that those VsCode settings do not get persisted and as you said it is a bug in the VsCode extension and not PSSA itself. However, I can offer you an alternative to save your settings (this is how I manage the settings myself): Just use the powershell.scriptAnalysis.settingsPath setting instead, which then gets saved in the .vscode/settings.json file. The main ReadMe gives an introduction to settings files and more examples are here. This is very neat because it means that everyone who opens your project in VsCode gets those settings applied. The path to the file is relative to the root where the .vscode folder is and if you name it PSScriptAnalyzerSettings.psd1 then those settings get applied implicitly if you run Invoke-ScriptAnalyzer from the root without having to specify the -Settings parameter. I hope I have explained it well enough but feel free to ask if I missed some details. I should probably use this features in this repo as well so that people can see actual examples.

[alx9r]

Hi @bergmeister. Thanks for helping me with this. I like the concept you described, but I haven't gotten it working.

I thought I followed your directions faithfully, but I'm seeing

when I select "PowerShell: Select ScriptAnalyzer Rules" from the command pallette.

Here is a minimal project folder that repros the problem: PssaVscode.zip Could you see if you see the same error using that project on your computer?

I'd like to get such a minimal project folder working so that I can use it as a reference for all my projects.

[bergmeister]

@alx9r Hi, yes, I have the same behaviour in my checkout and I think the extension is correct to display this error because the PSSA rules are now defined inside the settings file that you would need to modify instead if you want to enable/disable a rule. Modifying the file instead of using the menu is not as nice but should still be sufficient to solve your problem.
It seems that the implementation of the Select PSScriptAnalyzer rule is not fully finished in terms of persistence and being able to read/write to the settings file. The former developer @daviwil has now moved on to a different project (but still provides some help/maintenance in his free time), hence why this feature might have been left partially unfinished. I think the settings file option is sufficient for achieving the goal of being able to define the rules in VSCode.
I am sorry that this feature seems to be not fully integrated into the VSCode extension, therefore feel free to flag up the issues in the vscode extension repo.
I am sorry if you tried using this feature because I did demo it at psconf.eu, I have seen the feature myself, found it nice but was not aware of its limitations as I use a settings file myself.

[alx9r]

@bergmeister
Ok. I see how this fits together now. Thank you very much for the explanation.

...feel free to flag up the issues in the vscode extension repo.

I think I will. I'd like to suggest an improved message in that notification I posted above to avert this confusion. Would you be able to suggest some wording as a starting point?

I'm thinking that anyone who encounters that error should find a low-friction path to the same explanation you gave me.

I am sorry that this feature seems to be not fully integrated into the VSCode extension...

There's no apology necessary to me. I do think an improved error/warning message would assuage the downside of that feature being only partially implemented. Currently, it's a liability to productivity for anyone who tries to use it.

[bergmeister]

@alx9r Thanks. I will close the issue then if that is OK with you?