Design: CloudStorage API full support

[weshayutin]

As noted in our public doc:
https://docs.openshift.com/container-platform/4.15/backup_and_restore/application_backup_and_restore/installing/about-installing-oadp.html

The cloudstorage api is tech-preivew: We need this to be fully supported across aws, gcp and azure.

The CloudStorage API is a Technology Preview feature when you use a CloudStorage object and want OADP to use the CloudStorage API to automatically create an S3 bucket for use as a BackupStorageLocation.

The CloudStorage API supports manually creating a BackupStorageLocation object by specifying an existing S3 bucket. The CloudStorage API that creates an S3 bucket automatically is currently only enabled for AWS S3 storage.

When OADP uses the CloudStorage API we should have an option in DPA that defaults to true for automatic bucket creation.

https://issues.redhat.com/browse/OSDOCS-11734 WIF

https://docs.google.com/document/d/1iFNpyycby_rOY1wUew-yl3uPWlE00krTgr9XHDZOTNo/edit?tab=t.0

[weshayutin]

CloudStorage API for 1.4.0

1. Current status
   2. AWS supported
   3. GCP not
   4. Azure not
2. Cloud credential operator
   6. CCO doesn’t support WIF yet for standardized flow

High Level Breakdown of required work:

1. Background work
2. list out all related DPA config keys and value types
3. list out all the cloud specific keys and value types for configuration per cloud ( AWS, GCP, Azure )
4. Propose common user friendly variable names

CONTEXT

1. The cloud storage api main purpose to simply auto create a bucket if no buckets exists.
2. STS - auto create bucket
3. Uncouple from STS

The work:
AWS:

1. current status of bucket creation
2. Doc Review
3. Uncouple from STS

GCP:

1. current status of bucket creation
2. auto create a bucket
3. doc and test cases

Azure:

1. current status of bucket creation
2. Storage account, one level up from bucket. Extra steps here for azure
3. auto create bucket from customer provided storage account
4. Investigate STS support
5. Doc and test cases

Post scope

• investigate cloud credential operator

WORKFLOW from Customer:
STS enabled or no STS

1. user creates secret for cloud auth
2. user creates a dpa w/o BSL specificied but includes CloudStorage API - bucket key/value
   4. dpa.spec.backupLocation.velero - not filled out
   dpa.spec.backupLocation.bucket - user filled out
3. bucket auto created for user

[openshift-bot]

Issues go stale after 90d of inactivity.

Mark the issue as fresh by commenting /remove-lifecycle stale.
Stale issues rot after an additional 30d of inactivity and eventually close.
Exclude this issue from closing by commenting /lifecycle frozen.

If this issue is safe to close now please do so with /close.

/lifecycle stale

[kaovilai]

/lifecycle frozen

[weshayutin]

Tiger suggests https://github.com/kubernetes-sigs/container-object-storage-interface

[weshayutin]

ReReview on 04/25 - CloudStorage API for 1.5.0

1. Current status
   2. AWS supported
   3. GCP not
   4. Azure not
2. Cloud credential operator
   6. CCO does support WIF and Azure for standardized flow as of 4/2025

High Level Breakdown of required work:

work - Part: CCO

1. get credentials for azure, we have gcp wif, we have creds for aws.
2. put the current AWS cco code in an aws if statement
3. add GCP WIF if statement
4. add azure sts if statement
5. Validate w/ QE/Wes helping cco install from operator webui... verify that secrets exists and backup works.

work - Part: Cloud Storage API

1. aws bucket creation code is here
2. create code for gcp bucket creation
3. create code for azure bucket creation

work - Part: BSL Creation

1. Update BSL code to support azure and gcp

Uncouple CloudStorageAPI from CCO/standard workflow?

oadp-operator/internal/controller/cloudstorage_controller.go

Lines 126 to 135 in 338e8d5

	if common.CCOWorkflow() {
	// wait for the credential request to be processed and the secret to be created
	logger.Info(fmt.Sprintf("Following standardized STS workflow, waiting for for the credential request to be processed and provision the secret"))
	installNS := os.Getenv("WATCH_NAMESPACE")
	_, err = b.WaitForSecret(installNS, VeleroAWSSecretName)
	if err != nil {
	logger.Error(err, "unable to fetch secert created by CCO")
	return result, err
	}
	}

4. Test test test

CONTEXT

1. The cloud storage api main purpose to simply auto create a bucket if no buckets exists.
2. STS - auto create bucket
3. Uncouple from STS (maybe)

WORKFLOW from Customer:
STS enabled or no STS

1. user creates secret for cloud auth
2. user creates cloudstorage
3. bucket auto created for user
4. user creates a dpa with CloudStorage ref - bucket key/value

needs to think about encryption types, does that need to be added to:

• HERE https://github.com/openshift/oadp-operator/blob/0ff661cae03822fc278e9715af5515c7e89f680d/internal/controller/bsl.go#L152C44-L152C50