OADP-641: Enable AWS_CA_BUNDLE support for self-signed certificates in internal image backup

[kaovilai]

Issue Summary

OADP needs to support AWS_CA_BUNDLE environment variable for self-signed certificates in internal image backup scenarios, resolving OADP-641 that has been open for 2+ years.

Background

OADP-641 has tracked x509 certificate verification failures when using internal image backup with self-signed certificates. Investigation shows the technical solution is now available through dependency updates across the stack.

Problem Statement

Users encounter certificate validation errors like:

x509: certificate signed by unknown authority

When using:

• Internal S3-compatible storage with self-signed certificates
• Private container registries with custom CA certificates
• Air-gapped environments with enterprise PKI

Solution Status

The dependency chain now supports AWS_CA_BUNDLE:

OADP → openshift-velero-plugin → udistribution → openshift/docker-distribution

Progress tracker:

1. ✅ openshift/docker-distribution - PR #44 merged (Jan 2025)
2. 🔄 udistribution - update needed (migtools/udistribution#139)
3. 🔄 openshift-velero-plugin - update needed (openshift/openshift-velero-plugin#340)
4. 🔄 oadp-operator - this issue

Required OADP Actions

1. Dependency Updates

• Update openshift-velero-plugin dependency once it includes the udistribution fix
• Ensure OADP container images include updated dependencies

2. Configuration Support

Ensure OADP supports passing AWS_CA_BUNDLE environment variable to Velero pods:

apiVersion: oadp.openshift.io/v1alpha1
kind: DataProtectionApplication
metadata:
  name: dpa-sample
spec:
  configuration:
    velero:
      podConfig:
        env:
        - name: AWS_CA_BUNDLE
          value: /path/to/ca-bundle.pem

3. Documentation Updates

Add documentation for both methods:

Method 1: AWS_CA_BUNDLE Environment Variable

# Set in Velero deployment
AWS_CA_BUNDLE=/path/to/ca-bundle.pem

Method 2: OpenShift Certificate Injection (existing)

metadata:
  labels:
    config.openshift.io/inject-trusted-cabundle: "true"

4. Testing Scenarios

Test with:

• MinIO with self-signed certificates
• Noobaa with custom CA
• External S3-compatible storage with enterprise PKI
• Both AWS_CA_BUNDLE and OpenShift injection methods

Expected User Experience

After implementation, users with self-signed certificates can:

1. Configure CA bundle in DPA:

   spec:
     configuration:
       velero:
         podConfig:
           env:
           - name: AWS_CA_BUNDLE
             value: /etc/ssl/certs/ca-bundle.pem

2. Use certificate injection (current method):

   metadata:
     labels:
       config.openshift.io/inject-trusted-cabundle: "true"

Success Criteria

• OADP passes AWS_CA_BUNDLE to Velero pods
• Internal image backup works with self-signed certificates
• Documentation includes both configuration methods
• End-to-end testing validates the solution
• OADP-641 can be resolved

Impact

This resolves:

• OADP-641: 2+ year old certificate validation issues
• Air-gapped deployments: Enables OADP in disconnected environments
• Enterprise PKI: Supports custom certificate authorities
• User experience: Provides clear configuration options

Related Issues

• JIRA: OADP-641 - Support self-signed certificate for internal image backup
• Dependencies:
  • migtools/udistribution#139
  • openshift/openshift-velero-plugin#340
• Upstream:
  • openshift/docker-distribution#44
  • distribution/distribution#3841
  • distribution/distribution#4190

Documentation Reference

Updated OpenShift networking guide: https://docs.redhat.com/en/documentation/openshift_container_platform/4.19/html/networking/configuring-network-settings#configuring-a-custom-pki

[kaovilai]

Cross-Reference: Related OADP-641 Issues

This issue is part of a coordinated effort to resolve OADP-641 across the entire dependency chain:

Dependency Chain:

OADP → openshift-velero-plugin → udistribution → openshift/docker-distribution

Related GitHub Issues:

• 🔄 Upstream dependencies:
  • migtools/udistribution#139 - Update docker-distribution dependency
  • openshift/openshift-velero-plugin#340 - Update udistribution dependency
• 🔄 This issue: openshift/oadp-operator#1834 - Enable AWS_CA_BUNDLE support in OADP

Progress Status:

1. ✅ openshift/docker-distribution - PR #44 merged
2. 🔄 udistribution - migtools/udistribution#139 (upstream dependency)
3. 🔄 openshift-velero-plugin - openshift/openshift-velero-plugin#340 (direct dependency)
4. 🔄 oadp-operator - this issue (final integration)

Dependencies:

• ⏳ Blocked by:
  • migtools/udistribution#139 must be completed first
  • openshift/openshift-velero-plugin#340 must be completed second
• 🎯 Resolves: OADP-641 - 2+ year old certificate validation issue

Coordination Note:
This is the final integration point where all upstream dependency updates come together to enable AWS_CA_BUNDLE support for OADP users with self-signed certificates. Success here resolves the long-standing OADP-641 issue.