fix: add a step to generate secrets.yaml from a template #4323
Conversation
It is common for the secrets you need to inject into a variety of charts to share common values. So the new 'jx step create install secrets' lets you define a template file 'template-secrets.yaml' which you can safely check into git which is then used to create your 'secrets.yaml' file using a more concise 'secrets-parameters.yaml' file. e.g. you may inject your pipeline user name and token from your git provider into various charts like Tekton and Prow using the same values. Or you may want to use template functions to extract values for secrets from specific locations in the parameters file or from Vault locations etc. So this templating mechanism lets us compose secret values from input parameters or arbitrary functions/locations in a very similar way to the use of go templates inside YAML files in helm charts themselves. Signed-off-by: James Strachan <[email protected]>
|
[APPROVALNOTIFIER] This PR is NOT APPROVED This pull-request has been approved by: If they are not already assigned, you can assign the PR to them by writing The full list of commands accepted by this bot can be found here. The pull request process is described here
Needs approval from an approver in each of these files:
Approvers can indicate their approval by writing |
Signed-off-by: James Strachan <[email protected]>
Codecov Report
@@ Coverage Diff @@
## master #4323 +/- ##
=========================================
- Coverage 43.33% 6.64% -36.7%
=========================================
Files 803 684 -119
Lines 100760 90516 -10244
=========================================
- Hits 43668 6013 -37655
- Misses 53470 84085 +30615
+ Partials 3622 418 -3204
Continue to review full report at Codecov.
|
Codecov Report
@@ Coverage Diff @@
## master #4323 +/- ##
===========================================
- Coverage 43.33% 30.96% -12.38%
===========================================
Files 803 684 -119
Lines 100760 90516 -10244
===========================================
- Hits 43668 28031 -15637
- Misses 53470 61119 +7649
+ Partials 3622 1366 -2256
Continue to review full report at Codecov.
|
| SecretsTemplateFileName = "template-secrets.yaml" | ||
|
|
||
| // SecretsParametersFileName the secret parameter files which are never checked into git and may come from, say, Vault | ||
| SecretsParametersFileName = "secrets-parameters.yaml" |
There was a problem hiding this comment.
Can we use in this file the vault's URI based secrets lookup? The purpose of URI based secrets lookup was exactly to avoid committing secrets in git and have a dynamic mechanism to resolve thme. The secrets are resolved and interpolated before applying the charts.
I think the existing vault URI based lookup can be extended for k8s secrets and local secrets as follows:
Vault:
vault:gitOps/jenkins-x/environment-tekton-mole-dev/feature-flag-api-key:token-passthrough
k8s secrets:
k8s:<namespace>/<secret-name>:<secret-value>
Local:
local:<yaml-file-path>/<secret-secret-path-in-yaml>:<secret-value>
There was a problem hiding this comment.
yeah - I did wonder about using a function in the template for this kinda thing - so {{ .Vault.gitOps/jenkins-x/environment-tekton-mole-dev/feature-flag-api-key:token-passthrough }} or something like that?
There was a problem hiding this comment.
Agree with @ccojocar.
- anything where there is a risk you might commit the actual secrets, we should avoid like the plauge
- the secrets plugin is odd and confusing, I think we should try to not use it at all
The URL approach is much better in my opinion and I think we can use the scheme Cosmin proposes to generalise it.
I'm not sure why we would need to use a function - is this when you are using non tillerless helm?
| var ( | ||
| createInstallSecretsLong = templates.LongDesc(` | ||
| Creates or updates the install/upgrade secrets.yaml file from a template you can safely check into git: 'template-secrets.yaml' and a local secret parameters file: 'secrets-parameters.yaml' | ||
|
|
There was a problem hiding this comment.
It would be great if we could be consistent with notation and with the secrets sources.
|
I'm 👎 on merging this PR. Instead we should extend the URLs and work out how to make them work with tillered helm |
|
@pmuir I initially thought of going all in on URLs - but then I hit things like maven config.json secrets.yaml.
e.g. compare the input for a specific user/install needs to provide: secrets-parameters.yaml to the current secrets.yaml thats generated by the generic template - so much simpler for the user to configure + the template hides the duplication inside Jenkins X / Apps and the composition of user/passwords into larger chunks of secret text like ingress passwords, maven The aim of the templating is to reduce the amount of user configuration required for secrets really - URLs don't help with templating |
|
/hold |
|
gonna try solve this in a different way...#4328 |
4 participants
It is common for the secrets you need to inject into a variety of charts to share common values. So the new
jx step create install secretslets you define a template filetemplate-secrets.yamlwhich you can safely check into git which is then used to create yoursecrets.yamlfile using a more concisesecrets-parameters.yamlfile which you do not check into git.You can then use the
.gitignorerule ofsecrets*.yamlto avoid ever checking in thesecrets-parameters.yamlor the generatedsecrets.yamlfilese.g. you may inject your pipeline user name and token from your git provider into various charts like Tekton and Prow using the same values. Or you may want to use template functions to extract values for secrets from specific locations in the parameters file or from Vault locations etc.
So this templating mechanism lets us compose secret values from input parameters or arbitrary functions/locations in a very similar way to the use of go templates inside YAML files in helm charts themselves.
Example
jx step create install secretsthen generates this secrets.yamlSigned-off-by: James Strachan [email protected]