Bundle yaml files generated by v1.8.0 and above fail to install on OpenShift 4.7

[amisevsk]

Bug Report

Apologies if this bug belongs in https://github.com/operator-framework/operator-lifecycle-manager instead, I wasn't sure.

What did you do?

1. operator-sdk generate bundle after updating to operator-sdk v1.8.0 from v1.7.2.
2. Build bundle image
3. opm index add <args>
4. register catalog source and install operator

Release v1.8.0 appears to change how bundle files are formatted, wrapping long strings (e.g. in descriptions) instead of leaving them on one line. Checking for semantic differences (i.e. ignoring formatting) the files are identical.

What did you expect to see?

Operator installs normally from OpenShift UI

What did you see instead? Under which circumstances?

Operator's InstallPlan hangs on UpgradePending, with conditions

  conditions:
  - message: bundle contents have not yet been persisted to installplan status
    reason: BundleNotUnpacked
    status: "True"
    type: BundleLookupNotPersisted
  - lastTransitionTime: "2021-06-30T18:59:47Z"
    message: unpack job not completed
    reason: JobIncomplete
    status: "True"
    type: BundleLookupPending

Environment

Operator type:

/language go

Kubernetes cluster type:
OpenShift 4.7

$ operator-sdk version
operator-sdk v1.8.0 and v1.9.0

Does not reproduce in v1.7.2

$ go version (if language is Go)
go version go1.16.4 linux/amd64

$ kubectl version
Client Version: version.Info{Major:"1", Minor:"19", GitVersion:"v1.19.4", GitCommit:"d360454c9bcd1634cf4cc52d1867af5491dc9c5f", GitTreeState:"clean", BuildDate:"2020-11-11T13:17:17Z", GoVersion:"go1.15.2", Compiler:"gc", Platform:"linux/amd64"}
Server Version: version.Info{Major:"1", Minor:"20", GitVersion:"v1.20.0+bd9e442", GitCommit:"bd9e4421804c212e6ac7ee074050096f08dda543", GitTreeState:"clean", BuildDate:"2021-02-11T23:05:38Z", GoVersion:"go1.15.5", Compiler:"gc", Platform:"linux/amd64"}

Possible Solution

If I force long strings back onto one line using

find . -name *.yaml -exec yq --indentless -i -y -w 1000000000 . {} \;

in the bundle/manifests dir, the install succeeds as normal.

Additional context

By no means a minimal reproducer, but if you register the following catalogsource and install the DevWorkspace Operator via UI, the issue should reproduce:

cat <<EOF | oc apply -f -
apiVersion: operators.coreos.com/v1alpha1
kind: CatalogSource
metadata:
  name: custom-devworkspace-operator-catalog
  namespace: openshift-marketplace
spec:
  sourceType: grpc
  image: quay.io/amisevsk/devworkspace-operator-index:failed-install
  publisher: amisevsk
  displayName: DevWorkspace Operator Catalog
EOF

[estroz]

@amisevsk this is likely a result of the dep upgrade in #4871. The produced YAML is still valid afaik. @njhale this sounds like a bug in the bundle unpacker (opm alpha bundle extract I'm guessing).

[amisevsk]

@estroz Yep, the YAML files are still valid, and I didn't see any errors while building the bundle/index.

Should I move the issue over to the operator-registry side?

[estroz]

@amisevsk I'd open this in operator-lifecycle-manager since that's where the bundle unpack code resides.

[amisevsk]

@estroz I got around to poking at it more (thanks for the tip on opm alpha bundle extract!) and it turns out I'm hitting

time="2021-07-02T20:55:07Z" level=error msg="File with size 406294 exceeded 1048576 limit, aboring" file=/bundle/manifests/workspace.devfile.io_devworkspacetemplates.yaml

(minor typo in the error message).

It's always a big file, but it turns out that wrapping the lines in our CRDs adds 212KB to the bundle, which just barely pushes it over the default limit 🙃.

Closing this issue as I think I remember there already being issues around bundle size and error message reporting elsewhere.