Creating new status fields for RDS DBInstance CRD

[davirobin]

What Is Your Question?
Is it possible to create new status fields for the RDS DBInstance CRD through ack-generate code generation?

What Is The Context For This Question?
I am currently investigating the integration of Redhat Service Binding Operator (SBO) with ACK RDS, more specifically the DBInstance CRD, to enhance native binding. Going through the code-generator documentation and the community docs for determining status fields, there is a lot of material on how the custom code generator works using existing Create Operation’s Output shape. I need to create fields outside of the Output scope.

For example, to integrate the ACK RDS DBInstance with SBO I need to turn the DBInstance into a Provisioned Service via adding status.binding.name.

Any Additional Info?
This question is separate, but adjacent to the ACK K8s Native Application Binding proposal. FieldExport allows the user to export fields from the status to secrets whereas I need to include a new field in status to point to an existing secret.

[RedbackThomson]

This issue we have from the Crossplane team asks something similar: #881

The current answer I have for you is that we currently don't have a configuration for the code-generator to create an arbitrarily shaped field. We do support creating custom lists or maps using existing AWS SDK shapes and for setting the shape of a field.

The problem with adding status.binding.name to the RDS CRDs is that it is only applicable to SBO. The team at Crossplane maintain their own separate set of CRDs for every service, and in that they inject new fields that are only applicable to them and then also write custom code to initialize those fields. I understand it's annoying to maintain your own code, but putting custom SBO code into ACK controllers makes for a confusing experience for those who aren't using SBO.

With that said, if you are interested in maintaining a set of the RDS CRDs which can integrate with SBO, we can work on a solution for getting fields into the right place with custom generator code.

[davirobin]

Yes, I am interested in maintaining the RDS CRDs which can integrate with SBO. I understand the cause for concern with adding the code as a new feature regarding the confusion for non-SBO users. This question was more about understanding how ack-generate creates its CR fields and being able to create custom fields locally or maintaining custom RDS CRDs and generator code.

[RedbackThomson]

Okay cool. In that case, we still don't currently support adding arbitrary structs to the either the spec or status. This is mainly because our code-gen relies on using the AWS SDK models to build methods, so we kind of need an SDK model associated with everything. The way we support creating custom lists or maps is by creating a new SDK shape and injecting it into the model set before generation (see the code, here). I imagine this could be extended, whereby we accept arbitrary shapes either in the generator YAML file (maybe passed in through a file path) and take the same actions. This isn't supported out of the box at the moment, though.

[sadlerap]

Something I think might be relevant to this conversation: .status.binding.name comes from a specification upstream of SBO, so this wouldn't just be applicable to only SBO. For instance, there's an implementation from VMWare for use with Tanzu (source here), and there's also a work-in-progress reference implementation of the specification.

[baijum]

The FieldExport API provides native bindings for all kinds of developers. Both application development and infrastructure code can use FieldExport. However, it is a cumbersome solution for application development. Contrary to DevOps engineers who work with infrastructure code, application developers only require a minimal set of fields that is part of an AWS service. The application needs to connect to the service to perform its primary purpose. For example, database connectivity is the only requirement for application in the RDS service. The DBInstance custom resource part of RDS could become a Provisioned Service with a Secret resource containing all the fields required for database connectivity. Similar to RDS, other services can also provide Provisioned Service APIs.

Sometime back, I wrote a blog post about Service Binding spec focusing on Provisioned Service. Please read it here:
https://muthukadan.net/kubernetes/binding/support-service-binding-specification-for-kubernetes/

[baijum]

I have created a proof of concept support for Provisioned Service in the ACK RDS controller. The demo video is here: https://www.youtube.com/watch?v=AXXWv7N12JM
I intend to demonstrate the end-user experience of the ACK RDS controller with support for Provisioned Service. If there is interest, I would be happy to work on a proposal to submit here: https://github.com/aws-controllers-k8s/community/tree/main/docs/design/proposals

[RedbackThomson]

I have created a proof of concept support for Provisioned Service in the ACK RDS controller. The demo video is here: https://www.youtube.com/watch?v=AXXWv7N12JM I intend to demonstrate the end-user experience of the ACK RDS controller with support for Provisioned Service. If there is interest, I would be happy to work on a proposal to submit here: https://github.com/aws-controllers-k8s/community/tree/main/docs/design/proposals

This is awesome! Often we are only thinking about the experience for the DevOps teams that use and maintain these operators, so I'm glad you are able to understand how developers would interact with the final result. It'd be great to see a proposal. Before you submit one though, would you mind just summarising what those changes would be?

[baijum]

It'd be great to see a proposal. Before you submit one though, would you mind just summarising what those changes would be?

I modified the generated code in the RDS controller for the POC. The changes are here: baijum/ack-rds-controller@98da817

1. Updated the DBInstanceStatus to make the DBIntance a Provisioned Service (.status.binding.name attribute)
2. Updated the ClusterRole with create and update permissions
3. Updated the controller code to create the Secret resource
   To create a Secret resource, I have also modified the runtime package.
   In the runtime package, I updated the Reconciler interface with a new method with a signature like this: CreateSecret(context.Context, *corev1.Secret) error

I intended to demonstrate the developer experience if the DBInstance becomes a Provisioned Service.

Note: An alternative solution would be to use the Direct Secret Reference feature of the Service Binding Specification.

[baijum]

I have sent a PR with proposal: #1539