Conversion webhooks in main

[amisevsk]

What does this PR do?

Moves the conversion webhooks used for DevWorkspaces and DevWorkspaceTemplates into the main controller deployment rather than the webhooks server. This allows OLM to generate appropriate CRD fields when an operator is installed, and so enables installing the operator to any namespace on the cluster with the appropriate OperatorGroup configuration.

As part of this process, the Operator SDK dependency is updated to v1.8.0, as conversion webhooks support was added after v1.7.2. This change is done in the first commit of this PR and results in a seemingly large line diff, but semantically the files are identical and all line changes are due to line wrapping long strings. This can be tested using e.g.

diff <(yq -Y -S 'walk(if type == "array" then sort else . end)' ${oldfile}) <(yq -Y -S 'walk(if type == "array" then sort else . end)' ${newfile})

Known issues

• Operator SDK v1.9.0 doesn't respect --output-dir when generating bundle.Dockerfile (Operator SDK v1.9.0 seems to ignore --output-dir when generating bundle.Dockerfile operator-framework/operator-sdk#5021), see also: PR fix: specify the right path of metadata in bundle.Dockerfile operator-framework/operator-sdk#5030
  • Avoided by updating to v1.8.0 instead, which does not have the problem
• Strange issue where Operator SDK v1.8.0 generated bundles fail to install on OpenShift 4.7. Still not clear on what's going on here, but it's worked-around by using yq to restore files to long lines (see the second commit in this PR)
  • Issue: Bundle yaml files generated by v1.8.0 and above fail to install on OpenShift 4.7 operator-framework/operator-sdk#5033
• The order of webhooks generated for the CSV in deploy/bundle/manifests is not stable and changes on repeated invocations of operator-sdk generate bundle. I will likely create a workaround commit for this soon, but hopefully it's not a huge issue for now. See issue Order of generated webhooks is not stable operator-framework/operator-sdk#5022 for details.

What issues does this PR fix or reference?

Closes #470

Is it tested? How?

For each of

1. Kubernetes install
2. OpenShift install (direct install via yaml -- i.e. make install)
3. OLM install
   • in the openshift-operators namespace
   • in a different namespace (e.g. devworkspace-operator)

test that

1. Operator installs and starts correctly
2. CRD conversion webhooks point at the controller service (get crd devworkspaces.workspace.devfile.io -o yaml | yq '.spec.conversion')
3. Conversion works:
   • Create a DevWorkspace from sample
   • Read DevWorkspace as v1alpha1: kc get devworkspaces.v1alpha1.workspace.devfile.io -o yaml
   • Read DevWorkspace as v1alpha2: kc get devworkspaces.v1alpha2.workspace.devfile.io -o yaml
   • Easiest way to check that conversion is functioning correctly is to check e.g. plugins -- on v1alpha1, it's .components.plugins[].plugin.name and on v1alpha2 it's .components.plugins[].name.

PR Checklist

• E2E tests pass (when PR is ready, comment /test v7-devworkspaces-operator-e2e, v7-devworkspace-happy-path to trigger)
  • v7-devworkspaces-operator-e2e: DevWorkspace e2e test
  • v7-devworkspace-happy-path: DevWorkspace e2e test

[sleshchenko]

Good job. Minor comments are left, some of them are even not really related to the current PR)

[sleshchenko]

/test v7-devworkspaces-operator-e2e, v7-devworkspace-happy-path

[JPinkney]

LGTM

[openshift-ci]

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: amisevsk, JPinkney, sleshchenko

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files:

• OWNERS [JPinkney,amisevsk,sleshchenko]

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

[amisevsk]

/test v7-devworkspaces-operator-e2e, v7-devworkspace-happy-path

[sleshchenko]

/test v7-devworkspace-happy-path

[sleshchenko]

Tests fail with https://gcsweb-ci.apps.ci.l2s4.p1.openshiftapps.com/gcs/origin-ci-test/pr-logs/pull/devfile_devworkspace-operator/474/pull-ci-devfile-devworkspace-operator-main-v7-devworkspace-happy-path/1412825297429991424/artifacts/devworkspace-happy-path/devworkspace-happy-path/artifacts/devworkspace-controller-info/devworkspace-controller-manager-66db66f88c-mkgnj-devworkspace-controller.log

{"level":"error","ts":1625682041.475059,"logger":"setup","msg":"failed to setup webhooks","error":"could not deploy webhooks server: environment variable WEBHOOK_SECRET_NAME is unset","stacktrace":"main.main\n\t/devworkspace-operator/main.go:140\nruntime.main\n\t/usr/lib/golang/src/runtime/proc.go:204"}

[openshift-ci]

@amisevsk: The following test failed, say /retest to rerun all failed tests:

Test name	Commit	Details	Rerun command
ci/prow/v7-devworkspace-happy-path	6248389	link	/test v7-devworkspace-happy-path

Full PR test history. Your PR dashboard.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository. I understand the commands that are listed here.

[amisevsk]

Happy path tests are failing because they use chectl to deploy templates, which ignores updates to templates from the PR being tested. I think fixing this is out of scope for the current PR.

Tested latest changes manually on OpenShift and the happy path test failure does not reproduce.

[sleshchenko]

+1 to merge without waiting for happy path fix, but I remember we already faced the same and Serhii faced it.
We need a separate issue for Maxim to take a look on that.

[sleshchenko]

OK, it seems that it's not possible to use the latest deployment files since Che Operator image embeds them on container image build, unless we build Che operator image with custom templates... Or we may install devworkspace operator with make install before installing Che with devworkspace support.

[amisevsk]

Forgot to mention on closing -- created a dedicated issue: #484

[amisevsk]

Or we may install devworkspace operator with make install before installing Che with devworkspace support.

Will Che overwrite existing devworkspace resources on install, or will it detect that DevWorkspace is already present? A relatively simple fix would be to oc apply -f deploy/templates/openshift/combined.yaml before/after installing the Che operator, since that's basically what chectl does.