Reference deck.yaml / k8s resources between private repositories

[buschNT]

Hello!

I just started to write a deck.yaml to start integrating a new service into an existing Micro-service architecture. However, I am not sure how to "combine" decks and k8s resource definition files spread across different private git repositories.

Initially, we wrote a deck.yaml within the repository of the new service. Furthermore we added the required k8s resources within the same repo, including the Depolyment/Service/Ingress definitions. The deck.yaml uses the sources type "file" to point to the definitions and everything works great.

    sources:
      - type: file
        ref: ./k8s/service.yaml

Now, I would like to integrate this new service (let's call it "service N") into an existing service ("service E"), which is part of a micro-service architecture. This overall system has a helm-charts repository which I can use to create a development cluster. But how can I combine the new service into this cluster? What is the smartest approach? I tried to combine the services type helm with a file type. However I always get an error:

[ERROR] Error loading files from http 404 Client Error:

The deck.yaml file sources structure within the repo of service E looks something like the following, where I do not manage to provide a valid ref path to the k8s resources located in the repository of service N (both repos are private).

    sources:
      - type: helm
        ref: [email protected]:.../....git
...
      - type: file
        ref: ???????

I would like to avoid to copy files between repos, as it would make it hard to maintain everything. Instead I would prefer to reference the deck.yaml or resources of the service N within my deck.yaml of service E.

Suggestions and hints would be great! I think with more and more services contributing to an overall system architecture, it would be great to "select/pick" all connected services required to develop/test one specific service within its deck.yaml (without having to duplicate files between repos).

Best regards
Chris

[Schille]

Indeed, we don't want to copy files across repositories. I found the missing piece in the code base:

getdeck/getdeck/sources/file.py

Lines 111 to 112 in 5af100c

	def fetch_git(self, **kwargs) -> List[K8sSourceFile]:
	raise NotImplementedError

Although git-location is documented, it is currently not supported. If you have some spare time, will you create a PR to make this work?
I suppose we would need to find a special notation to reference a file in a git repository or add a path attribute to it.

[Schille]

We could support something like this for the ref attribute: "https://github.example.com/myuser/myrepository.git?path=/example/folder#v1.2.3"

[Schille]

Or "[email protected]:myuser/myrepository.git?path=/example/folder#v1.2.3" with:

• # used for the revision (tag, branch, commit)
• path used to get only a file or directory from that repo

[buschNT]

The error I got is raised by the fetch_https function. If I provide the link to the raw file "https://raw.githubusercontent.com/.../main/k8s/service.yaml?token=..." it seems to work shortly. But I guess the token is not valid for very long. The fetch_git might be a better/cleaner solution. I guess the implementation should be similar to the fetch_git function form the ToolerFetcher.

[buschNT]

I'm working on the fetch_git implementation (see PR #47 ).