Propose OCI-packaging changes

[Silvanoc]

Description

I'd like to propose some changes in the OCI-packaging format implemented in #

Container image packaging looks like this:

---
  config:
    class:
      hideEmptyMembersBox: true
---
classDiagram
    namespace ContainerImages {
        class mai["Multi-Arch Container Image"]:::ociIndex {
            <<OCI index>>
            manifests
        }

        class x86i["x86-64 Container Image"]:::ociManifest {
            <<OCI manifest>>
            config
            layers
        }

        class armi["ARM64 Container Image"]:::ociManifest {
            <<OCI manifest>>
            config
            layers
        }    
    }

%%    namespace Compositions {
%%        class ci["Multi-Arch Composition"]:::ociIndex {
%%           <<OCI index>>
%%            subject
%%            manifests
%%        }

%%        class cx86m["x86-64 Composition"]:::ociManifest {
%%            <<OCI manifest>>
%%            subject
%%            layers
%%        }

%%        class compx86["x86-64 compose.yaml"] {
%%            <<OCI blob>>
%%        }

%%        class carmm["ARM64 Composition"]:::ociManifest {
%%            <<OCI manifest>>
%%            subject
%%            layers
%%        }

%%        class comparm["ARM64 compose.yaml"] {
%%            <<OCI blob>>
%%        }
%%    }

    namespace Tags {
        class latestIm["latest"]:::ociTag {
            <<OCI tag>>
        }

%%        class latestComp["latest-composition"]:::ociTag {
%%            <<OCI tag>>
%%        }        
    }

    classDef ociTag fill:#09f,stroke:#444,stroke-width:4px;
    classDef ociIndex fill:#f9f,stroke:#444,stroke-width:4px;
    classDef ociManifest fill:#f9a,stroke:#444,stroke-width:2px;

    latestIm --> mai : digest
    mai o-- x86i : manifest
    mai o-- armi : manifest
%%    latestComp --> ci : digest
%%    ci o-- cx86m : manifest
%%    ci o-- carmm : manifest
%%    ci --> mai : subject
%%    cx86m o-- compx86 : layer
%%    cx86m --> x86i : subject
%%    carmm o-- comparm : layer
%%    carmm --> armi : subject

Loading

---

‼️ IMPORTANT ‼️

Following proposal enables:

1. "attaching" a compose.yaml to an existing container image without modifying any of its OCI index or manifests
2. composing multiple container images copying them to a single OCI repository, will be called "bundled composition" and is similar to a phone app
3. composing multiple container images contained in other OCI repositories, will be called "loose composition" and is similar to a Helm Chart

---

Bundled Composition

This would be the case when the presence of all the needed parts needs to be guaranteed by the OCI registry. But also when "attaching" a compose.yaml to an existing container image.

It modifies the OCI-reference to the container images, but not the images themselves. Therefore it is changing the compose.yaml file.

It provides better guarantees that other container-based packages like "Helm Charts" and therefore it can be seen as the "appification" of a composition. The availability of the OCI repository guarantees the availability of the application.

Here is where the beauty of the subject jumps in:

---
  config:
    class:
      hideEmptyMembersBox: true
---
classDiagram
    namespace ContainerImages {
        class mai["Multi-Arch Container Image"]:::ociIndex {
            <<OCI index>>
            manifests
        }

        class x86i["x86-64 Container Image"]:::ociManifest {
            <<OCI manifest>>
            config
            layers
        }

        class armi["ARM64 Container Image"]:::ociManifest {
            <<OCI manifest>>
            config
            layers
        }    
    }

    namespace Compositions {
%%        class ci["Multi-Arch Composition"]:::ociIndex {
%%           <<OCI index>>
%%            subject
%%            manifests
%%        }

        class cx86m["x86-64 Composition"]:::ociManifest {
            <<OCI manifest>>
            subject
            layers
        }

        class compx86["x86-64 compose.yaml"] {
            <<OCI blob>>
        }

%%        class carmm["ARM64 Composition"]:::ociManifest {
%%            <<OCI manifest>>
%%            subject
%%            layers
%%        }

%%        class comparm["ARM64 compose.yaml"] {
%%            <<OCI blob>>
%%        }
    }

    namespace Tags {
        class latestIm["latest"]:::ociTag {
            <<OCI tag>>
        }

%%        class latestComp["latest-composition"]:::ociTag {
%%            <<OCI tag>>
%%        }        
    }

    classDef ociTag fill:#09f,stroke:#444,stroke-width:4px;
    classDef ociIndex fill:#f9f,stroke:#444,stroke-width:4px;
    classDef ociManifest fill:#f9a,stroke:#444,stroke-width:2px;

    latestIm --> mai : digest
    mai o-- x86i : manifest
    mai o-- armi : manifest
%%    latestComp --> ci : digest
%%    ci o-- cx86m : manifest
%%    ci o-- carmm : manifest
%%    ci --> mai : subject
    cx86m o-- compx86 : layer
    cx86m --> x86i : subject
%%    carmm o-- comparm : layer
%%    carmm --> armi : subject

Loading

You wouldn't even need to add a tag to the compose (though you could). The referrers API (or its fallback) would let you "discover" the composition querying for the container image.

But what if you want to provide multi-arch compositions for a multi-arch image? You simply apply the same pattern:

---
  config:
    class:
      hideEmptyMembersBox: true
---
classDiagram
    namespace ContainerImages {
        class mai["Multi-Arch Container Image"]:::ociIndex {
            <<OCI index>>
            manifests
        }

        class x86i["x86-64 Container Image"]:::ociManifest {
            <<OCI manifest>>
            config
            layers
        }

        class armi["ARM64 Container Image"]:::ociManifest {
            <<OCI manifest>>
            config
            layers
        }    
    }

    namespace Compositions {
        class ci["Multi-Arch Composition"]:::ociIndex {
           <<OCI index>>
            subject
            manifests
        }

        class cx86m["x86-64 Composition"]:::ociManifest {
            <<OCI manifest>>
            subject
            layers
        }

        class compx86["x86-64 compose.yaml"] {
            <<OCI blob>>
        }

        class carmm["ARM64 Composition"]:::ociManifest {
            <<OCI manifest>>
            subject
            layers
        }

        class comparm["ARM64 compose.yaml"] {
            <<OCI blob>>
        }
    }

    namespace Tags {
        class latestIm["latest"]:::ociTag {
            <<OCI tag>>
        }

%%        class latestComp["latest-composition"]:::ociTag {
%%            <<OCI tag>>
%%        }        
    }

    classDef ociTag fill:#09f,stroke:#444,stroke-width:4px;
    classDef ociIndex fill:#f9f,stroke:#444,stroke-width:4px;
    classDef ociManifest fill:#f9a,stroke:#444,stroke-width:2px;

    latestIm --> mai : digest
    mai o-- x86i : manifest
    mai o-- armi : manifest
%%    latestComp --> ci : digest
    ci o-- cx86m : manifest
    ci o-- carmm : manifest
    ci --> mai : subject
    cx86m o-- compx86 : layer
    cx86m --> x86i : subject
    carmm o-- comparm : layer
    carmm --> armi : subject

Loading

No matter which image reference you have (tag to the index, index digest, arch-specific manifest digest,...) you can always discover the corresponding composition.

Finally a tag can be added also to the composition, providing two different entry points:

1. only for the container image
2. for the container image with a compose.yaml providing some help to run a container using the image

---
  config:
    class:
      hideEmptyMembersBox: true
---
classDiagram
    namespace ContainerImages {
        class mai["Multi-Arch Container Image"]:::ociIndex {
            <<OCI index>>
            manifests
        }

        class x86i["x86-64 Container Image"]:::ociManifest {
            <<OCI manifest>>
            config
            layers
        }

        class armi["ARM64 Container Image"]:::ociManifest {
            <<OCI manifest>>
            config
            layers
        }    
    }

    namespace Compositions {
        class ci["Multi-Arch Composition"]:::ociIndex {
           <<OCI index>>
            subject
            manifests
        }

        class cx86m["x86-64 Composition"]:::ociManifest {
            <<OCI manifest>>
            subject
            layers
        }

        class compx86["x86-64 compose.yaml"] {
            <<OCI blob>>
        }

        class carmm["ARM64 Composition"]:::ociManifest {
            <<OCI manifest>>
            subject
            layers
        }

        class comparm["ARM64 compose.yaml"] {
            <<OCI blob>>
        }
    }

    namespace Tags {
        class latestIm["latest"]:::ociTag {
            <<OCI tag>>
        }

        class latestComp["latest-composition"]:::ociTag {
            <<OCI tag>>
        }        
    }

    classDef ociTag fill:#09f,stroke:#444,stroke-width:4px;
    classDef ociIndex fill:#f9f,stroke:#444,stroke-width:4px;
    classDef ociManifest fill:#f9a,stroke:#444,stroke-width:2px;

    latestIm --> mai : digest
    mai o-- x86i : manifest
    mai o-- armi : manifest
    latestComp --> ci : digest
    ci o-- cx86m : manifest
    ci o-- carmm : manifest
    ci --> mai : subject
    cx86m o-- compx86 : layer
    cx86m --> x86i : subject
    carmm o-- comparm : layer
    carmm --> armi : subject

Loading

Loose Composition

In this case the presence of the needed container images is not guaranteed by the OCI registry. It is equivalent to a "Helm Chart" in that sense. If an OCI repository hosting one of the required container images is deleted, the composition is broken.

It does not change the compose.yaml file in any way.

The references to the container images are not explicit in the OCI artifact, but "embedded" in the compose.yaml file and in the image-digests.yaml file (which is used to ensure the integrity of the images).

---
  config:
    class:
      hideEmptyMembersBox: true
---
classDiagram
    namespace Image1 {
        class mai["Multi-Arch Container Image"]:::ociIndex {
            <<OCI index>>
            manifests
        }

        class x86i["x86-64 Container Image"]:::ociManifest {
            <<OCI manifest>>
            config
            layers
        }

        class armi["ARM64 Container Image"]:::ociManifest {
            <<OCI manifest>>
            config
            layers
        }    

        class latestIm["latest"]:::ociTag {
            <<OCI tag>>
        }
    }

    namespace CompositionRepository {
        class ci["Multi-Arch Composition"]:::ociIndex {
           <<OCI index>>
            subject
            manifests
        }

        class cx86m["x86-64 Composition"]:::ociManifest {
            <<OCI manifest>>
            subject
            layers
        }

        class compx86["x86-64 compose.yaml"] {
            <<OCI blob>>
        }

        class idx86["x86-64 image-digests.yaml"] {
            <<OCI blob>>
        }

        class carmm["ARM64 Composition"]:::ociManifest {
            <<OCI manifest>>
            subject
            layers
        }

        class comparm["ARM64 compose.yaml"] {
            <<OCI blob>>
        }

        class idarm["ARM64 image-digests.yaml"] {
            <<OCI blob>>
        }

        class latestComp["latest"]:::ociTag {
            <<OCI tag>>
        }        
    }

    classDef ociTag fill:#09f,stroke:#444,stroke-width:4px;
    classDef ociIndex fill:#f9f,stroke:#444,stroke-width:4px;
    classDef ociManifest fill:#f9a,stroke:#444,stroke-width:2px;

    latestIm --> mai : digest
    mai o-- x86i : manifest
    mai o-- armi : manifest
    latestComp --> ci : digest
    ci o-- cx86m : manifest
    ci o-- carmm : manifest
    cx86m o-- compx86 : layer
    cx86m o-- idx86 : layer
    carmm o-- comparm : layer
    carmm o-- idarm : layer
    compx86 ..> x86i
    idx86 ..> x86i
    comparm ..> armi
    idarm ..> armi

Loading

[ndeloof]

about multi-arch: why refer by subject to platform specific images and not the multi-plaform index-list ? Doing so you don't need platform-specific compose.yaml (which would have the exact same content)

"Loose Composition" you describe is what we have when a compose file is published without the --app flag, but relying on index-list for multi-platform

"Bundled Composition" you describe allows a compose OCI artifact to be subject of an image but not a set of images, as required by a compose application. Could use a top-level index-list to group such images, but then you basically get the same solution as publish --app with subject relation reverted.

The main issue I can see with your approach is that the application tag doesn't target the actual compose OCI artifact. Then there's no way to refer to it for a consumer to use your "bundle" application.