Skip to content

Commit 4c4d5cb

Browse files
Add FieldExport CRD and field export reconciler (#75)
Issue #, if available: aws-controllers-k8s/community#740 Description of changes: Creates a new `FieldExport` CRD as outlined [this proposal](aws-controllers-k8s/community#1183). Also creates a new reconciler that reconciles changes to `FieldExport` and changes to any ACK resource recognised by the current controller. The path for reconciling a `FieldExport`: 1. Ensure the `FieldExport` `Source` resource group matches the current controller's group 2. Mark as managed (add finalizer) 3. Describe the source object 3a. If no source object is found, stop reconciling 4. Use the [`gojq` library](https://github.com/itchyny/gojq) to parse the unstructured source object 5. Write to `ConfigMap`/`Secret` The path when any ACK resource is updated: 1. Ensure the resource has the synced condition set to true 2. List and filter all `FieldExport` types in the namespace that reference the current resource as its `Source` 3. For each matching `FieldExport` do steps 3 and 4 from the previous path The reconciler converts all primitive types into strings (ConfigMap is a string-string map, Secret is a string-byte[] map). If the jq query returns a list of results, we will only take the first result (this PR does not support exporting slices/maps). If the jq query returns a struct, we will not take any action, treating the query as a failure. The reconciler will not delete any values from the ConfigMap or Secret. If the referenced field is removed from the object, the reconciler will no-op, rather than overwriting the data with an empty string. When the `FieldExport` is deleted, the ConfigMap or Secret will also be left untouched - meaning it contains whatever was last written to it. Upon creating a `FieldExport` CR, the reconciler will initially attempt to export the field. If it fails, it will not attempt to retry. After the first sync, the reconciler will only wait until the `ResourceVersion` of the source object changes for it to trigger a new update. By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.
1 parent 02010de commit 4c4d5cb

38 files changed

+2654
-135
lines changed

ATTRIBUTION.md

Lines changed: 25 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -26,6 +26,7 @@ License version 2.0, we include the full text of the package's License below.
2626
* `github.com/go-logr/logr`
2727
* `github.com/google/go-cmp`
2828
* `github.com/iancoleman/strcase`
29+
* `github.com/itchyny/gojq`
2930
* `github.com/mitchellh/go-homedir`
3031
* `github.com/pkg/errors`
3132
* `github.com/spf13/cobra`
@@ -339,6 +340,30 @@ LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
339340
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
340341
SOFTWARE.
341342

343+
### github.com/itchyny/gojq
344+
345+
The MIT License (MIT)
346+
347+
Copyright (c) 2019-2022 itchyny
348+
349+
Permission is hereby granted, free of charge, to any person obtaining a copy
350+
of this software and associated documentation files (the "Software"), to deal
351+
in the Software without restriction, including without limitation the rights
352+
to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
353+
copies of the Software, and to permit persons to whom the Software is
354+
furnished to do so, subject to the following conditions:
355+
356+
The above copyright notice and this permission notice shall be included in all
357+
copies or substantial portions of the Software.
358+
359+
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
360+
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
361+
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
362+
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
363+
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
364+
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
365+
SOFTWARE.
366+
342367
### github.com/mitchellh/go-homedir
343368

344369
The MIT License (MIT)

Makefile

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -27,7 +27,7 @@ mocks: install-mockery ## Build mocks
2727
@bin/mockery --quiet --name=Object --case=underscore --output=mocks/apimachinery/pkg/apis/meta/v1 --dir="$(K8S_APIMACHINERY_DIR)/pkg/apis/meta/v1"
2828
@echo "ok."
2929
@echo -n "building mocks for k8s.io/apimachinery/runtime ... "
30-
@bin/mockery --quiet --name=Object --case=underscore --output=mocks/apimachinery/pkg/runtime --dir="$(K8S_APIMACHINERY_DIR)/pkg/runtime"
30+
@bin/mockery --quiet --name="(Object|UnstructuredConverter)" --case=underscore --output=mocks/apimachinery/pkg/runtime --dir="$(K8S_APIMACHINERY_DIR)/pkg/runtime"
3131
@echo "ok."
3232
@echo -n "building mocks for k8s.io/apimachinery/runtime/schema ... "
3333
@bin/mockery --quiet --name=ObjectKind --case=underscore --output=mocks/apimachinery/pkg/runtime/schema --dir="$(K8S_APIMACHINERY_DIR)/pkg/runtime/schema"

apis/core/v1alpha1/adopted_resource.go

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -20,7 +20,7 @@ import (
2020
// AdoptedResourceSpec defines the desired state of the AdoptedResource.
2121
type AdoptedResourceSpec struct {
2222
// +kubebuilder:validation:Required
23-
Kubernetes *TargetKubernetesResource `json:"kubernetes"`
23+
Kubernetes *ResourceWithMetadata `json:"kubernetes"`
2424
// +kubebuilder:validation:Required
2525
AWS *AWSIdentifiers `json:"aws"`
2626
}

apis/core/v1alpha1/common.go

Lines changed: 10 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -21,3 +21,13 @@ type AWSAccountID string
2121

2222
// AWSResourceName represents an AWS Resource Name (ARN)
2323
type AWSResourceName string
24+
25+
// FieldExportOutputType represents all types that can be produced by a field
26+
// export operation
27+
// +kubebuilder:validation:Enum=configmap;secret
28+
type FieldExportOutputType string
29+
30+
const (
31+
FieldExportOutputTypeConfigMap FieldExportOutputType = "configmap"
32+
FieldExportOutputTypeSecret = "secret"
33+
)

apis/core/v1alpha1/field_export.go

Lines changed: 62 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,62 @@
1+
// Copyright Amazon.com Inc. or its affiliates. All Rights Reserved.
2+
//
3+
// Licensed under the Apache License, Version 2.0 (the "License"). You may
4+
// not use this file except in compliance with the License. A copy of the
5+
// License is located at
6+
//
7+
// http://aws.amazon.com/apache2.0/
8+
//
9+
// or in the "license" file accompanying this file. This file is distributed
10+
// on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either
11+
// express or implied. See the License for the specific language governing
12+
// permissions and limitations under the License.
13+
14+
package v1alpha1
15+
16+
import (
17+
metav1 "k8s.io/apimachinery/pkg/apis/meta/v1"
18+
)
19+
20+
// FieldExportTarget provides the values necessary to identify the
21+
// output path for a field export.
22+
type FieldExportTarget struct {
23+
Name *string `json:"name"`
24+
// Namespace is marked as optional, so we cannot compose `NamespacedName`
25+
Namespace *string `json:"namespace,omitempty"`
26+
Kind FieldExportOutputType `json:"kind"`
27+
}
28+
29+
// FieldExportSpec defines the desired state of the FieldExport.
30+
type FieldExportSpec struct {
31+
From *ResourceFieldSelector `json:"from"`
32+
To *FieldExportTarget `json:"to"`
33+
}
34+
35+
// FieldExportStatus defines the observed status of the FieldExport.
36+
type FieldExportStatus struct {
37+
// A collection of `ackv1alpha1.Condition` objects that describe the various
38+
// recoverable states of the field CR
39+
Conditions []*Condition `json:"conditions"`
40+
}
41+
42+
// FieldExport is the schema for the FieldExport API.
43+
// +kubebuilder:object:root=true
44+
// +kubebuilder:subresource:status
45+
type FieldExport struct {
46+
metav1.TypeMeta `json:",inline"`
47+
metav1.ObjectMeta `json:"metadata,omitempty"`
48+
Spec FieldExportSpec `json:"spec,omitempty"`
49+
Status FieldExportStatus `json:"status,omitempty"`
50+
}
51+
52+
// FieldExportList defines a list of FieldExports.
53+
// +kubebuilder:object:root=true
54+
type FieldExportList struct {
55+
metav1.TypeMeta `json:",inline"`
56+
metav1.ListMeta `json:"metadata,omitempty"`
57+
Items []FieldExport `json:"items"`
58+
}
59+
60+
func init() {
61+
SchemeBuilder.Register(&FieldExport{}, &FieldExportList{})
62+
}

apis/core/v1alpha1/identifiers.go

Lines changed: 24 additions & 8 deletions
Original file line numberDiff line numberDiff line change
@@ -13,6 +13,10 @@
1313

1414
package v1alpha1
1515

16+
import (
17+
metav1 "k8s.io/apimachinery/pkg/apis/meta/v1"
18+
)
19+
1620
// AWSIdentifiers provide all unique ways to reference an AWS resource.
1721
type AWSIdentifiers struct {
1822
// ARN is the AWS Resource Name for the resource. It is a globally
@@ -26,14 +30,26 @@ type AWSIdentifiers struct {
2630
AdditionalKeys map[string]string `json:"additionalKeys,omitempty"`
2731
}
2832

29-
// TargetKubernetesResource provides all the values necessary to identify a given ACK type
30-
// and override any metadata values when creating a resource of that type.
31-
type TargetKubernetesResource struct {
32-
// +kubebuilder:validation:Required
33-
Group string `json:"group"`
34-
// +kubebuilder:validation:Required
35-
Kind string `json:"kind"`
36-
Metadata *PartialObjectMeta `json:"metadata,omitempty"`
33+
// NamespacedResource provides all the values necessary to identify an ACK
34+
// resource of a given type (within the same namespace as the custom resource
35+
// containing this type).
36+
type NamespacedResource struct {
37+
metav1.GroupKind `json:""`
38+
Name *string `json:"name"`
39+
}
40+
41+
// ResourceWithMetadata provides the values necessary to create a
42+
// Kubernetes resource and override any of its metadata values.
43+
type ResourceWithMetadata struct {
44+
metav1.GroupKind `json:""`
45+
Metadata *PartialObjectMeta `json:"metadata,omitempty"`
46+
}
47+
48+
// ResourceFieldSelector provides the values necessary to identify an individual
49+
// field on an individual K8s resource.
50+
type ResourceFieldSelector struct {
51+
Resource NamespacedResource `json:"resource"`
52+
Path *string `json:"path"`
3753
}
3854

3955
// AWSResourceReferenceWrapper provides a wrapper around *AWSResourceReference

0 commit comments

Comments
 (0)