Secrets Management RFC

[SteveL-MSFT]

RFC to securely and simply manage secrets with PowerShell

[devblackops]

Are there any blockers to this being provided as a module itself and NOT a native addition to PS? I can see this capability being useful to lower versions of PS.

From the RFC, I'm not seeing anything that couldn't be provided via a module.

[thomasrayner]

Out of curiosity, what's the motivation for deferring certificates to a future release? Is it the complexity of storing certs in other, non-AKV stores? For my use cases, the ability to store pscredential and securestring secrets is only half a solution.

[SteveL-MSFT]

@thomasrayner intent was to scope this and incrementally improve over time. We may be able to implement cert support within PS7 timeframe, but it wasn't clear to us if certs is still being used heavily.

[SteveL-MSFT]

We can certainly see about making this module compatible with WinPS 5.1 and newer.

[codykonior]

If this was made, what I’d use is: local vaults, with a credential stored for the current user and other credentials shared between users.

[Jaykul]

This should clearly be an external module shipped through the gallery. Other than that, I have only minor nits to pick with this proposal (in-line).

[KirkMunro]

I'm hoping that one of the goals for this RFC is easy use of credentials in memory once they have been pulled from a (remote) credential store so that they don't have to be cached on a local disk at all. This allows for credentials to be managed in a cloud credential store and used in scripts run elsewhere by users who have access to those credentials.

But how is this even going to work?

Different systems have different definitions of "credentials", and you mentioned Get-Secret would return either SecureString or PSCredential. Those are very Microsoft-specific.

You also mentioned that Register-SecretsVaultExtension would take an -AccessToken or -Credential parameter. Those are also very Microsoft-specific.

What about AWS or other systems that don't use either of these?

In the AWSPowerShell.NetCore module, they use different sign-in methods. One that I use when I'm working with credentials that are only stored in memory is the common -AccessKey and -SecretKey strings that are common parameters in over 5800 of their cmdlets. These are both strings, and with them I can use permanent or temporary credentials without storing them locally on disk. But how would I retrieve them from a vault using the proposed design? I could force them into the PSCredential object, and then access them using -AccessKey $creds.UserName -SecretKey $creds.GetNetworkCredential().Password, but that's a hack, not a proper design. Similar challenges arise for vault extensions and the 'Credential' parameters that they are required to support.

Vaults used to store secrets should be able to be registered using whatever credentials they require.

Vaults should be able to store an encrypted version of whatever object containing the secret is required by the platform where the secret is used, and users should be able to retrieve those decrypted versions of those objects.

The current RFC does not properly support either of those. By limiting the definition of 'Credential' to SecureString or PSCredential, and by limiting registration to -AccessToken or -Credential, I don't think you're proposal properly considers how credentials are used more broadly for products and services outside of Microsoft, and how those types of credentials would be supported in the design. I also worry that defining this in a module with no support in PowerShell 7 itself at an SDK level, where, for instance, a module author could implement an interface used for dehydration/rehydration of a credential that was stored in a vault, or add an attribute that identifies a cmdlet as one that is used to store or retrieve secrets, and then supporting dynamic parameters on the Get-/Add-Secret command, will not be properly addressing the needs that this RFC tries to address.

[thedevopsmachine]

I think that the proposed module has over-simplified the complex process of secret management, and as such, limits its usefulness. It's still a good idea though.

Regarding the remote stores, I recommend working backwards from some common providers that exist today. Take a look at the existing Cmdlets and API's for AWS Secrets Manager, Thycotic Secret Server, Azure Key Vault etc. That's how people do "Remote" secret management today. Take note that secrets are never returned as a PSCredential; they're always a string or binary value. The plaintext value is also included as a property of an object rather than just a value; it usually also contains the secret ID, the version ID, and the last modified date as a bare minimum. It would make sense to return an object with similar properties, or better yet, define an ISecret interface that providers can implement, with some common properties like the ones listed above.

[StartAutomating]

Should be Register-SecretVault (pluralization in cmdlet names)

[KirkMunro]

Not true. Seems like you missed @SteveL-MSFT's reply about that. SecretsVault is singular.

[StartAutomating]

While there might only be one "SecretsVault", and it might actually contain multiple "Secrets", the point of the "No Plurals" rule is about how plurals translate. At this point in the ecosystem, it's also about a reasonable expectation.

I do not believe that any PowerShell user will assume that Register-SecretVault will register a vault and then make it secret (it kinda flies in the face of the cmdlet name). I do believe that, especially as we have "Add-Secret", "Register-SecretsVault" would end up being confusing in practice. Your brain would remember one or the other, and you'd be more likely to flub it.

The other rationale, past pure pluralization, is the desire for common noun roots and the easy memorization that comes with it.

[StartAutomating]

One other call out here, now that I see it, is that it shold support a -Parameter @{} ,which will be passed down to the undelying command. The scenario for this is that my cmdlet fits fine, but I want to supply default parameters to map to the underlying store.

Register-SecretVault -Name Azure Cmdlet Get-AzKeyVaultSecret -Module AZ.KeyVault -Version 1.0.0 -Parameter @{VaultName='MyVault;Location="West US"}

As I type out that bunch of code, I believe it would be nice to accept a command via pipeline, or to look up the command / assume defaults given the name. e.g.

  Get-Command Get-AzureKeyVaultSecret | Regster-SecretVault  # -Name would be AzureKeyVaultSecret, unless specified
  Register-SecretVault -Name Azure -Cmdlet Get-AzKeyVaultSecret # finds the command and supplies module / version, maybe complains or warns if there is no module

[StartAutomating]

One more thing is that the -Cmdlet parameter is somewhat nonstandard. In a quick check of PS 5.1, the only cmds with -Cmdlet are:

• Export-ModuleMember
• Import-Module
• New-Module

All of which actually need cmdlet.

Conversely, it's -Commad on:

• Get/Remove Job
• Invoke-Expression
• Trace-Command
• Get-PSBreakpoint /Set-PSBreakpoint

The last pair is especially instructive here, as I recall a discussion when naming that parameter on this topic. We wanted to avoid naming command breakpoints -Cmdlet because they could use commands that were not Cmdelts.

Unless you're saying this secret vault has to be a Cmdlet, the parameter should be -Command

[scrthq]

SecretsVault indicates a single vault containing multiple secrets. SecretVault indicates a single vault that itself is secret. I am with @KirkMunro on this one, the proposed noun is singular as the noun itself here is Vault while Secrets is describing what the vault itself is for.

[StartAutomating]

Not to beat the drum too much, but the Cmdlet Guidelines don't offer with that rationale.

"Plural parameter names should be used only in those cases where the value of the parameter is always a multiple-element value. In these cases, the cmdlet should verify that multiple elements are supplied, and the cmdlet should display a warning to the user if multiple elements are not supplied."

Are we registering N Secret(s)Vault(s) at a time? No. In fact we probably are only ever registering one or two.

Also, note the natural confusion in where this could be pluralized, and realize this is why people should not pluralize cmdlet names. We've got about a decade of establishment of this paradigm. Why start to break good habits now?

[StartAutomating]

One other call out here, now that I see it, is that it shold support a -Parameter @{} ,which will be passed down to the undelying command. The scenario for this is that my cmdlet fits fine, but I want to supply default parameters to map to the underlying store.

Register-SecretVault -Name Azure Cmdlet Get-AzKeyVaultSecret -Module AZ.KeyVault -Version 1.0.0 -Parameter @{VaultName='MyVault;Location="West US"}

As I type out that bunch of code, I believe it would be nice to accept a command via pipeline, or to look up the command / assume defaults given the name. e.g.

  Get-Command Get-AzureKeyVaultSecret | Regster-SecretVault  # -Name would be AzureKeyVaultSecret, unless specified
  Register-SecretVault -Name Azure -Cmdlet Get-AzKeyVaultSecret # finds the command and supplies module / version, maybe complains or warns if there is no module

[StartAutomating]

One more thing is that the -Cmdlet parameter is somewhat nonstandard. In a quick check of PS 5.1, the only cmds with -Cmdlet are:

• Export-ModuleMember
• Import-Module
• New-Module

All of which actually need cmdlet.

Conversely, it's -Commad on:

• Get/Remove Job
• Invoke-Expression
• Trace-Command
• Get-PSBreakpoint /Set-PSBreakpoint

The last pair is especially instructive here, as I recall a discussion when naming that parameter on this topic. We wanted to avoid naming command breakpoints -Cmdlet because they could use commands that were not Cmdelts.

Unless you're saying this secret vault has to be a Cmdlet, the parameter should be -Command

[scrthq]

Not to beat the drum too much, but the Cmdlet Guidelines don't offer with that rationale.

"Plural parameter names should be used only in those cases where the value of the parameter is always a multiple-element value. In these cases, the cmdlet should verify that multiple elements are supplied, and the cmdlet should display a warning to the user if multiple elements are not supplied."

Are we registering N Secret(s)Vault(s) at a time? No. In fact we probably are only ever registering one or two.

Also, note the natural confusion in where this could be pluralized, and realize this is why people should not pluralize cmdlet names. We've got about a decade of establishment of this paradigm. Why start to break good habits now?

The items you're quoting @StartAutomating are specific to parameter pluralization though, not pluralization of the noun in the cmdlet name itself. There are no guidelines specified in the linked document, that I can see, regarding ensuring cmdlet names are singular.

Regarding your assumption about "only ever registering one or two", you point out the possibility of having multiple secrets in a single vault. The cmdlet itself is scoped per vault, not per secret, so indicating that the vault can store and retrieve multiple secrets within it points at the value of SecretsVault supporting multiple secrets. I can think of a few vaults that I would maintain with these cmdlets (i.e. personal on local, AWS Secrets Manager per AWS account I manage, etc), but my expectation is that each vault would be represented by a single namespace regardless if it has 1 or 1000 secrets within it.

[StartAutomating]

There are no guidelines specified in the linked document, that I can see, regarding ensuring cmdlet names are singular.

The point is that plurals are frowned upon. Not that guidance is perfect.

I've embedded a little helper thought experiment for you.

$allCmds = Get-Command -CommandType Function, Cmdlet

function Get-CmdletWord
{
    param([Parameter(ValueFromPipelineByPropertyName=$true)][Alias('Name')][string]$text)
    process {            
    $r = New-Object Text.RegularExpressions.Regex "[a-z][A-Z]", "Multiline"
    $matches = @($r.Matches($text))
    $offset = 0
    foreach ($m in $matches) {
        $text = $text.Insert($m.Index + $offset + 1," ")
        $offset++
    }
    $text -split ' '
    }
}


$allPotentialPlurals = foreach ($cmd in $allCmds) {
    $cmdletWords = @($cmd | Get-CmdletWord)
    if ($cmdletWords -like '*s') {
    
        [PSCustomObject][Ordered]@{
            Command=$cmd;
            PotentialPlurals=
                 foreach ($_ in $cmdletWords -like '*s') {
                    $_ -replace '\S{1,}-',''
                 }
            } 
     }
}



 $uniquePotentialPlurals=  $allPotentialPlurals | Select-Object -ExpandProperty PotentialPlurals -Unique

$uniquePotentialPlurals

 $uniquePotentialPlurals.Count / $allCmds.Count

If you look, there are very few cmdlet names that even have something plural-like. The vast majority of those are not PowerShell team cmdlets, but Azure. A number of other ones have a better claim to "not being a plural", like "BIOS".

On my box, these unique plurals represent only about 1% of the cmdlet names. If you all for duplication, they only represent around 15% of all cmdlet names.

So, instead of arguing so hard to make this one a "Secrets" vault, ask why it should be the exception to the generally established rule.

[scrthq]

The exception to the rule is the same argument: a SecretsVault is still a singular Vault, the Vault being the actual noun within the Noun component of the cmdlet name. Secrets in the context is an adjective describing that singular noun, as the *-SecretsVault cmdlets are dealing with a single object, a Vault. The proposed cmdlets interacting with the secrets within the vault itself are also singular, *-Secret.

There aren't too many occurrences in the overall English language where you use a plural adjective to describe a singular noun, so I'm not surprised at the lack of matches that sample code returns. Using the singular form of Secret in this context turns the adjective into something other than intended, as the Vault itself isn't secret.

[scrthq]

Maybe a better command that satisfies all arguments here would be to drop Secrets from the Vault cmdlet names? Get-Vault, Register-Vault, Unregister-Vault

[KirkMunro]

Maybe a better command that satisfies all arguments here would be to drop Secrets from the Vault cmdlet names? Get-Vault, Register-Vault, Unregister-Vault

That's too generic, IMHO. SecretsVault is much more descriptive.

[StartAutomating]

The exception to the rule is the same argument

I think that there's some crosstalking going on here. I cede your point about the English language.

The primary point I am making is consistency with the way most things in PowerShell work, and with itself.

This point has a few facets:

• Any time you have paired commands with an inconsistent base noun it's annoying.
  ** It becomes harder for users to remember which one of them is inconsistent
  ** It makes tab completion more likely to be wonky (though in this case tab completion would be best served by having it be Add-Secret/Add-SecretVault)
• Commands should work in consistent families, and commands with similar functionality should be similarly named.
• Plurals aren't recommended. Period.

I think the short name of Get-Vault, Register-Vault, and Unregister-Vault would be fine, though I'd prever Get/Add/Remove (see the bit on tab completion below)

By the above rationale:

• While we're discussing these commands as a paired group, the reality is they are for distinctly different purposes. A user could run Register-Vault once, and Get-Secret everyday.
• The user has two distinct things to remember, with two distinct names. Both names are discoverable and sensible.
• The most common case would tab-complete better (if it was Remove-Secret and Register-SecretVault, Register-SecretVault would tab complete before Remove-Secret)
• Two distinct cmdlet families would exist, with no expectation of overlapping parameters.
• No plurals
• For slight bonus points, Vault tab-completes before VM.

To Kirk's comment about genericness, a few points:

1. How many other kinds of vaults can there be, especially in the context of computing?
2. If we can store a [byte[]] in these vaults, how would the term not fit for whatever other context?
3. Generic cmdlet names are fine, as long as the fit the most generic purpose. Nobody's mad that Get-Help is called Get-Help. Even if rebuttals can be found for 1 & 2, do you believe that there is a more appropriate use of the generic term vault?

[StartAutomating]

Maybe a better command that satisfies all arguments here would be to drop Secrets from the Vault cmdlet names? Get-Vault, Register-Vault, Unregister-Vault
As explained in more detail below, I'm pretty good with this. I'd prefer Get/Add/Remove.

[KirkMunro]

To Kirk's comment about genericness, a few points:

1. How many other kinds of vaults can there be, especially in the context of computing?
2. If we can store a [byte[]] in these vaults, how would the term not fit for whatever other context?
3. Generic cmdlet names are fine, as long as the fit the most generic purpose. Nobody's mad that Get-Help is called Get-Help. Even if rebuttals can be found for 1 & 2, do you believe that there is a more appropriate use of the generic term vault?

Backup and recovery vaults in cloud services.

As for which is more appropriate for a generic vault term, that's not a call that I can make. Personal experience says both are used/known, and specific nouns are better.

[thedevopsmachine]

The downside to this kind of implementation (as opposed to a C# implementation with standardized implementations) is that there's no consistency as to how things like error handling work.

Can we include a list of standard error messages that providers should use for a few common scenarios such as:

• Access Denied
• Not Found

If providers all implement the exact same error handling for these scenarios, then it will make switching between providers much easier, as error handling code won't have to be rewritten.

[StartAutomating]

As ideal as standardized error codes are, I do not believe this is something where one could guarantee underlying consistency. A web-based SecretVault will have a different return code under the covers compared to a C-api based SecretVault.

We could encourage people to use standard -ErrorIDs ( or at least a standard -ErrorCategory) to attempt to address the problem. Luckily, -ErrorCategory already contains the two we need: ObjectNotFound and PermissionDenied (or AuthenticationError).

As far as the need/want for a C# api, one can always use PowerShell's C# api to treat these cmdlets as a C# API. e.g.

PowerShell.Create().AddScript('Get-Secret MySecret').Invoke<string>();

Thus I believe not only isn't it in scope for this RFC, it shouldn't be for any future RFCs. I believe to do so opens a nasty pandora's box which would encourage other aspects of PowerShell to provide a C# api as well. This in turn de-emphasizes the need or want to ever write scripts.

Thus I believe we should specify standard error categories, but should not be in the habit of making special C# wrappers for PowerShell.

[thedevopsmachine]

Cool, the important part of my comment above was the standardization of errors. C# makes it easier to enforce standards, but having a section in the RFC stating that providers should implement the existing error categories/ID's to ensure consistency between different providers is the next best thing.

For example, if one had multiple remote providers and wanted to find a secret but wasn't certain which provider contained it (e.g. during a migration from one provider to another) then they could write a script that iterates through each installed remote provider and puts a "continue" in the catch block if it encounters an exception in the "ObjectNotFound" category. It would suck to have to handle each provider's error in a different way.

[SteveL-MSFT]

ErrorCategory is an enum, so not sure what needs to be explicitly stated here.

[StartAutomating]

I had a little language integration thought that could make this feature more interesting / a better draw to vNext:

It would be interesting to add a standard parameter name to look up and use these secrets:, say -SecretParameter @{NameOfParameter=NameOfSecret}

The reason I say this is that I have written a lot of commands wrapping the -SecureSetting commands (which do a similar thing). A number of these commands are written with a parameter that can either be the secret value or the name of the secret (for instance -ConnectionStringOrSetting in Select-Sql).

Essentially, having a -SecretParameter common parameter would enable this sort of scenario for any command without any authors work, and could act as an effective "hook" to encourage migration to vNext. I personally have seen secure scripting to be a killer feature for a long time, and see this sort of language integration as a great carrot to draw people into vNext.

Additionally, since one can hook and override common parameters, this wouldn't break if someone were to implement their own support for -SecretParameter downlevel.

[SteveL-MSFT]

@StartAutomating That's an interesting idea, however, it's out of scope for this initial version. I'll add it to the future considerations section.

[joeyaiello]

Just a few thoughts as I finally get around to reading all the comments here:

• I'm seeing a lot of comments about providing more generic parameters that might fit better with other providers. I don't think the goal here is to completely map all constructs of all types of secrets and credentials between each other. To that end, I think it's certainly possible that we could add new parameter sets that fit other providers better.
• I thought this discussion on the level of abstraction for the return of Get-Secret was really interesting (didn't want to unresolve it there). Would love to think more about the output of Get-Secret. I think we'd agree it shouldn't return the plaintext output in the default formatter unless the user is specifying -AsPlaintext, but it might also be worth having an explicit Get-SecretValue for that operation.
• In general, would just remind everyone that this is absolutely just a minimum viable module that we'll want to expand over time. I'd also very much like to start it out on the Gallery and make it work down to 5.1, but I haven't totally explored the feasibility of that with @SteveL-MSFT. Then, we could ship it into 7.x in a more stable and fully-featured state at some point down the line.
• I'm also on board with dropping the plural "Secrets" from everything, but only because I think I would just forget which cmdlets use the plural noun and which don't. If it's all "Secret", you never have to guess wrong. (And yes, tab completion and all that, but still....I like typing for disambiguity).

But yeah, on the whole, I think it's a good scoping for a good start. If we go with the Gallery approach for pre-release, we can also mess with the surface area of the objects if we find certain patterns to be more/less useful in practice.

[JamesWTruher]

I don't have anything blocking here, other than being sure that we get some solid usage/output examples.

[JamesWTruher]

network credential?

[vexx32]

Is that a secret of any kind? It has a plaintext password property.

[JamesWTruher]

Is it possible to add tags? It seems quite useful to add additional descriptive data (and it looks to be supported by the back end store). Also, I would like to have a description of all the parameters.
I assume this supports ShouldProcess, but want to be explicit.
Also, what is the nature of the object that is returned by some of these cmdlets?

[JamesWTruher]

this is pretty sketchy - would you please provide a demo.txt, or similar?

[JamesWTruher]

this is where I would love to have some sort of -Tag or similar

[JamesWTruher]

this seems, in any event, as additional code based on the ones described herein.

[daxian-dbw]

Can you please list the cmdlets exposed from the module? I find it hard to infer from rest of the RFC about exposed cmdlets.
It would be great if you can list all exposed cmdlets at the beginning of the specification section.

[daxian-dbw]

Maybe move this line to the previous line instead of using a line continuation?

[daxian-dbw]

I suggest to move this second to where the extension model is discussed. For someone who is not familiar with Az.KeyVault, he/she don't even know whether Register-SecretsVault is from that module or Microsoft.PowerShell.SecretsManagement.
Moving this to extension model discussion, it can serve as an example for that discussion.

[SteveL-MSFT]

Closing this as @SydneyhSmith has taken ownership of this RFC going forward and has published an updated one here: #234