feat: add dual-compression (zstd:chunked) to buildah task

Add COMPRESSION_FORMAT (gzip|zstd-chunked|dual) and FORCE_COMPRESSION
params. In dual mode, push gzip and zstd:chunked variants bundled in a
per-arch OCI index (gzip first). The IMAGES result exposes both child
manifest refs for Chains provenance. SBOM generation, signing, and SBOM
attachment run for each variant.

Default stays gzip; dual requires BUILDAH_FORMAT=oci.

Ref #1264
Ref https://github.com/konflux-ci/architecture/blob/main/ADR/0070-dual-compression-for-container-builds.md

Signed-off-by: Amanyd <amaninsane139@gmail.com>

Author: Amanyd

pipelines/docker-build-multi-platform-oci-ta/README.md

@@ -65,11 +65,13 @@ This pipeline is pushed as a Tekton bundle to [quay.io](https://quay.io/reposito
 |BUILD_TIMESTAMP| Defines the single build time for all buildah builds in seconds since UNIX epoch. Conflicts with SOURCE_DATE_EPOCH.| ""| |
 |CACHI2_ARTIFACT| The Trusted Artifact URI pointing to the artifact with the prefetched dependencies.| ""| '$(tasks.prefetch-dependencies.results.CACHI2_ARTIFACT)'|
 |COMMIT_SHA| The image is built from this commit.| ""| '$(tasks.clone-repository.results.commit)'|
+|COMPRESSION_FORMAT| Compression format: gzip (default), zstd-chunked, or dual. In dual mode both gzip and zstd:chunked variants are bundled in a per-arch OCI index (gzip first). dual requires BUILDAH_FORMAT=oci.| gzip| |
 |CONTEXT| Path to the directory to use as context.| .| '$(params.path-context)'|
 |CONTEXTUALIZE_SBOM| Determines if SBOM will be contextualized.| true| |
 |DOCKERFILE| Path to the Dockerfile to build.| ./Dockerfile| '$(params.dockerfile)'|
 |ENTITLEMENT_SECRET| Name of secret which contains the entitlement certificates| etc-pki-entitlement| |
 |ENV_VARS| Array of --env values ("env=value" strings)| []| |
+|FORCE_COMPRESSION| Recompress all layers including base image layers, not just layers created by this build. Only applies to zstd-chunked and dual modes.| false| |
 |HERMETIC| Determines if build will be executed without network access.| false| '$(params.hermetic)'|
 |HTTP_PROXY| HTTP/HTTPS proxy to use for the buildah pull and build operations. Will not be passed through to the container during the build process.| ""| '$(tasks.init.results.http-proxy)'|
 |ICM_KEEP_COMPAT_LOCATION| Whether to keep compatibility location at /root/buildinfo/ for ICM injection| true| |
@@ -291,6 +293,7 @@ This pipeline is pushed as a Tekton bundle to [quay.io](https://quay.io/reposito
 ### buildah-remote-oci-ta:0.10 task results
 |name|description|used in params (taskname:taskrefversion:taskparam)
 |---|---|---|
+|IMAGES| Newline-separated image manifest references. In dual mode lists the gzip and zstd child manifest references for Chains provenance; otherwise the single manifest reference.| |
 |IMAGE_DIGEST| Digest of the image just built| |
 |IMAGE_REF| Image reference of the built image| build-image-index:0.3:IMAGES|
 |IMAGE_URL| Image repository and tag where the built image was pushed| |

pipelines/docker-build-oci-ta-min/README.md

@@ -56,11 +56,13 @@ This pipeline is pushed as a Tekton bundle to [quay.io](https://quay.io/reposito
 |BUILD_TIMESTAMP| Defines the single build time for all buildah builds in seconds since UNIX epoch. Conflicts with SOURCE_DATE_EPOCH.| ""| |
 |CACHI2_ARTIFACT| The Trusted Artifact URI pointing to the artifact with the prefetched dependencies.| ""| '$(tasks.prefetch-dependencies.results.CACHI2_ARTIFACT)'|
 |COMMIT_SHA| The image is built from this commit.| ""| '$(tasks.clone-repository.results.commit)'|
+|COMPRESSION_FORMAT| Compression format: gzip (default), zstd-chunked, or dual. In dual mode both gzip and zstd:chunked variants are bundled in a per-arch OCI index (gzip first). dual requires BUILDAH_FORMAT=oci.| gzip| |
 |CONTEXT| Path to the directory to use as context.| .| '$(params.path-context)'|
 |CONTEXTUALIZE_SBOM| Determines if SBOM will be contextualized.| true| |
 |DOCKERFILE| Path to the Dockerfile to build.| ./Dockerfile| '$(params.dockerfile)'|
 |ENTITLEMENT_SECRET| Name of secret which contains the entitlement certificates| etc-pki-entitlement| |
 |ENV_VARS| Array of --env values ("env=value" strings)| []| |
+|FORCE_COMPRESSION| Recompress all layers including base image layers, not just layers created by this build. Only applies to zstd-chunked and dual modes.| false| |
 |HERMETIC| Determines if build will be executed without network access.| false| '$(params.hermetic)'|
 |HTTP_PROXY| HTTP/HTTPS proxy to use for the buildah pull and build operations. Will not be passed through to the container during the build process.| ""| '$(tasks.init.results.http-proxy)'|
 |ICM_KEEP_COMPAT_LOCATION| Whether to keep compatibility location at /root/buildinfo/ for ICM injection| true| |
@@ -231,6 +233,7 @@ This pipeline is pushed as a Tekton bundle to [quay.io](https://quay.io/reposito
 ### buildah-oci-ta-min:0.10 task results
 |name|description|used in params (taskname:taskrefversion:taskparam)
 |---|---|---|
+|IMAGES| Newline-separated image manifest references. In dual mode lists the gzip and zstd child manifest references for Chains provenance; otherwise the single manifest reference.| |
 |IMAGE_DIGEST| Digest of the image just built| |
 |IMAGE_REF| Image reference of the built image| |
 |IMAGE_URL| Image repository and tag where the built image was pushed| build-image-index:0.3:IMAGES|

pipelines/docker-build-oci-ta/README.md

@@ -64,11 +64,13 @@ This pipeline is pushed as a Tekton bundle to [quay.io](https://quay.io/reposito
 |BUILD_TIMESTAMP| Defines the single build time for all buildah builds in seconds since UNIX epoch. Conflicts with SOURCE_DATE_EPOCH.| ""| |
 |CACHI2_ARTIFACT| The Trusted Artifact URI pointing to the artifact with the prefetched dependencies.| ""| '$(tasks.prefetch-dependencies.results.CACHI2_ARTIFACT)'|
 |COMMIT_SHA| The image is built from this commit.| ""| '$(tasks.clone-repository.results.commit)'|
+|COMPRESSION_FORMAT| Compression format: gzip (default), zstd-chunked, or dual. In dual mode both gzip and zstd:chunked variants are bundled in a per-arch OCI index (gzip first). dual requires BUILDAH_FORMAT=oci.| gzip| |
 |CONTEXT| Path to the directory to use as context.| .| '$(params.path-context)'|
 |CONTEXTUALIZE_SBOM| Determines if SBOM will be contextualized.| true| |
 |DOCKERFILE| Path to the Dockerfile to build.| ./Dockerfile| '$(params.dockerfile)'|
 |ENTITLEMENT_SECRET| Name of secret which contains the entitlement certificates| etc-pki-entitlement| |
 |ENV_VARS| Array of --env values ("env=value" strings)| []| |
+|FORCE_COMPRESSION| Recompress all layers including base image layers, not just layers created by this build. Only applies to zstd-chunked and dual modes.| false| |
 |HERMETIC| Determines if build will be executed without network access.| false| '$(params.hermetic)'|
 |HTTP_PROXY| HTTP/HTTPS proxy to use for the buildah pull and build operations. Will not be passed through to the container during the build process.| ""| '$(tasks.init.results.http-proxy)'|
 |ICM_KEEP_COMPAT_LOCATION| Whether to keep compatibility location at /root/buildinfo/ for ICM injection| true| |
@@ -288,6 +290,7 @@ This pipeline is pushed as a Tekton bundle to [quay.io](https://quay.io/reposito
 ### buildah-oci-ta:0.10 task results
 |name|description|used in params (taskname:taskrefversion:taskparam)
 |---|---|---|
+|IMAGES| Newline-separated image manifest references. In dual mode lists the gzip and zstd child manifest references for Chains provenance; otherwise the single manifest reference.| |
 |IMAGE_DIGEST| Digest of the image just built| |
 |IMAGE_REF| Image reference of the built image| |
 |IMAGE_URL| Image repository and tag where the built image was pushed| build-image-index:0.3:IMAGES|

pipelines/docker-build/README.md

@@ -63,11 +63,13 @@ This pipeline is pushed as a Tekton bundle to [quay.io](https://quay.io/reposito
 |BUILD_ARGS_FILE| Path to a file with build arguments, see https://www.mankier.com/1/buildah-build#--build-arg-file| ""| '$(params.build-args-file)'|
 |BUILD_TIMESTAMP| Defines the single build time for all buildah builds in seconds since UNIX epoch. Conflicts with SOURCE_DATE_EPOCH.| ""| |
 |COMMIT_SHA| The image is built from this commit.| ""| '$(tasks.clone-repository.results.commit)'|
+|COMPRESSION_FORMAT| Compression format: gzip (default), zstd-chunked, or dual. In dual mode both gzip and zstd:chunked variants are bundled in a per-arch OCI index (gzip first). dual requires BUILDAH_FORMAT=oci.| gzip| |
 |CONTEXT| Path to the directory to use as context.| .| '$(params.path-context)'|
 |CONTEXTUALIZE_SBOM| Determines if SBOM will be contextualized.| true| |
 |DOCKERFILE| Path to the Dockerfile to build.| ./Dockerfile| '$(params.dockerfile)'|
 |ENTITLEMENT_SECRET| Name of secret which contains the entitlement certificates| etc-pki-entitlement| |
 |ENV_VARS| Array of --env values ("env=value" strings)| []| |
+|FORCE_COMPRESSION| Recompress all layers including base image layers, not just layers created by this build. Only applies to zstd-chunked and dual modes.| false| |
 |HERMETIC| Determines if build will be executed without network access.| false| '$(params.hermetic)'|
 |HTTP_PROXY| HTTP/HTTPS proxy to use for the buildah pull and build operations. Will not be passed through to the container during the build process.| ""| '$(tasks.init.results.http-proxy)'|
 |ICM_KEEP_COMPAT_LOCATION| Whether to keep compatibility location at /root/buildinfo/ for ICM injection| true| |
@@ -272,6 +274,7 @@ This pipeline is pushed as a Tekton bundle to [quay.io](https://quay.io/reposito
 ### buildah:0.10 task results
 |name|description|used in params (taskname:taskrefversion:taskparam)
 |---|---|---|
+|IMAGES| Newline-separated image manifest references. In dual mode lists the gzip and zstd child manifest references for Chains provenance; otherwise the single manifest reference.| |
 |IMAGE_DIGEST| Digest of the image just built| |
 |IMAGE_REF| Image reference of the built image| |
 |IMAGE_URL| Image repository and tag where the built image was pushed| build-image-index:0.3:IMAGES|

pipelines/fbc-builder/README.md

@@ -63,11 +63,13 @@ This pipeline is pushed as a Tekton bundle to [quay.io](https://quay.io/reposito
 |BUILD_TIMESTAMP| Defines the single build time for all buildah builds in seconds since UNIX epoch. Conflicts with SOURCE_DATE_EPOCH.| ""| |
 |CACHI2_ARTIFACT| The Trusted Artifact URI pointing to the artifact with the prefetched dependencies.| ""| '$(tasks.prefetch-dependencies.results.CACHI2_ARTIFACT)'|
 |COMMIT_SHA| The image is built from this commit.| ""| '$(tasks.clone-repository.results.commit)'|
+|COMPRESSION_FORMAT| Compression format: gzip (default), zstd-chunked, or dual. In dual mode both gzip and zstd:chunked variants are bundled in a per-arch OCI index (gzip first). dual requires BUILDAH_FORMAT=oci.| gzip| |
 |CONTEXT| Path to the directory to use as context.| .| '$(params.path-context)'|
 |CONTEXTUALIZE_SBOM| Determines if SBOM will be contextualized.| true| |
 |DOCKERFILE| Path to the Dockerfile to build.| ./Dockerfile| '$(params.dockerfile)'|
 |ENTITLEMENT_SECRET| Name of secret which contains the entitlement certificates| etc-pki-entitlement| |
 |ENV_VARS| Array of --env values ("env=value" strings)| []| |
+|FORCE_COMPRESSION| Recompress all layers including base image layers, not just layers created by this build. Only applies to zstd-chunked and dual modes.| false| |
 |HERMETIC| Determines if build will be executed without network access.| false| '$(params.hermetic)'|
 |HTTP_PROXY| HTTP/HTTPS proxy to use for the buildah pull and build operations. Will not be passed through to the container during the build process.| ""| '$(tasks.init.results.http-proxy)'|
 |ICM_KEEP_COMPAT_LOCATION| Whether to keep compatibility location at /root/buildinfo/ for ICM injection| true| |
@@ -216,6 +218,7 @@ This pipeline is pushed as a Tekton bundle to [quay.io](https://quay.io/reposito
 ### buildah-remote-oci-ta:0.10 task results
 |name|description|used in params (taskname:taskrefversion:taskparam)
 |---|---|---|
+|IMAGES| Newline-separated image manifest references. In dual mode lists the gzip and zstd child manifest references for Chains provenance; otherwise the single manifest reference.| |
 |IMAGE_DIGEST| Digest of the image just built| |
 |IMAGE_REF| Image reference of the built image| build-image-index:0.3:IMAGES|
 |IMAGE_URL| Image repository and tag where the built image was pushed| |

task/buildah-oci-ta-min/0.10/README.md

@@ -20,11 +20,13 @@ When prefetch-dependencies task is activated it is using its artifacts to run bu
 |BUILD_TIMESTAMP|Defines the single build time for all buildah builds in seconds since UNIX epoch. Conflicts with SOURCE_DATE_EPOCH.|""|false|
 |CACHI2_ARTIFACT|The Trusted Artifact URI pointing to the artifact with the prefetched dependencies.|""|false|
 |COMMIT_SHA|The image is built from this commit.|""|false|
+|COMPRESSION_FORMAT|Compression format: gzip (default), zstd-chunked, or dual. In dual mode both gzip and zstd:chunked variants are bundled in a per-arch OCI index (gzip first). dual requires BUILDAH_FORMAT=oci.|gzip|false|
 |CONTEXT|Path to the directory to use as context.|.|false|
 |CONTEXTUALIZE_SBOM|Determines if SBOM will be contextualized.|true|false|
 |DOCKERFILE|Path to the Dockerfile to build.|./Dockerfile|false|
 |ENTITLEMENT_SECRET|Name of secret which contains the entitlement certificates|etc-pki-entitlement|false|
 |ENV_VARS|Array of --env values ("env=value" strings)|[]|false|
+|FORCE_COMPRESSION|Recompress all layers including base image layers, not just layers created by this build. Only applies to zstd-chunked and dual modes.|false|false|
 |HERMETIC|Determines if build will be executed without network access.|false|false|
 |HTTP_PROXY|HTTP/HTTPS proxy to use for the buildah pull and build operations. Will not be passed through to the container during the build process.|""|false|
 |ICM_KEEP_COMPAT_LOCATION|Whether to keep compatibility location at /root/buildinfo/ for ICM injection|true|false|
@@ -64,6 +66,7 @@ When prefetch-dependencies task is activated it is using its artifacts to run bu
 ## Results
 |name|description|
 |---|---|
+|IMAGES|Newline-separated image manifest references. In dual mode lists the gzip and zstd child manifest references for Chains provenance; otherwise the single manifest reference.|
 |IMAGE_DIGEST|Digest of the image just built|
 |IMAGE_REF|Image reference of the built image|
 |IMAGE_URL|Image repository and tag where the built image was pushed|