Skip to content

Commit 9fd4de9

Browse files
KidswissKidswiss
authored
Merge pull request #1045 from danielpodwysocki/master
implement filtering PVCs/preBackupPods using labelSelectors
2 parents de8151c + 548c29f commit 9fd4de9

File tree

17 files changed

+504
-11
lines changed

17 files changed

+504
-11
lines changed

‎Makefile‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -75,7 +75,7 @@ deploy: kind-load-image install ## Deploy controller in the configured Kubernete
7575
--set image.registry=$(E2E_REGISTRY) \
7676
--set image.repository=$(E2E_REPO) \
7777
--set image.tag=$(E2E_TAG) \
78-
--values ./e2e/definitions/operator/deploy.yaml \
78+
--values ./e2e/definitions/operator/values.yaml \
7979
--wait $(deploy_args)
8080

8181
.PHONY: generate

‎api/v1/backup_types.go‎

Lines changed: 5 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -40,6 +40,11 @@ type BackupSpec struct {
4040

4141
// Tags is a list of arbitrary tags that get added to the backup via Restic's tagging system
4242
Tags []string `json:"tags,omitempty"`
43+
44+
// LabelSelectors is a list of selectors that we filter for.
45+
// When defined, only PVCs and PreBackupPods matching them are backed up.
46+
// +optional
47+
LabelSelectors []metav1.LabelSelector `json:"labelSelectors,omitempty"`
4348
}
4449

4550
type BackupTemplate struct {

‎api/v1/zz_generated.deepcopy.go‎

Lines changed: 7 additions & 0 deletions
Some generated files are not rendered by default. Learn more about customizing how changed files appear on GitHub.

‎config/crd/apiextensions.k8s.io/v1/k8up.io_backups.yaml‎

Lines changed: 52 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -448,6 +448,58 @@ spec:
448448
449449
Deprecated: Use FailedJobsHistoryLimit and SuccessfulJobsHistoryLimit respectively.
450450
type: integer
451+
labelSelectors:
452+
description: |-
453+
LabelSelectors is a list of selectors that we filter for.
454+
When defined, only PVCs and PreBackupPods matching them are backed up.
455+
items:
456+
description: |-
457+
A label selector is a label query over a set of resources. The result of matchLabels and
458+
matchExpressions are ANDed. An empty label selector matches all objects. A null
459+
label selector matches no objects.
460+
properties:
461+
matchExpressions:
462+
description: matchExpressions is a list of label selector requirements.
463+
The requirements are ANDed.
464+
items:
465+
description: |-
466+
A label selector requirement is a selector that contains values, a key, and an operator that
467+
relates the key and values.
468+
properties:
469+
key:
470+
description: key is the label key that the selector applies
471+
to.
472+
type: string
473+
operator:
474+
description: |-
475+
operator represents a key's relationship to a set of values.
476+
Valid operators are In, NotIn, Exists and DoesNotExist.
477+
type: string
478+
values:
479+
description: |-
480+
values is an array of string values. If the operator is In or NotIn,
481+
the values array must be non-empty. If the operator is Exists or DoesNotExist,
482+
the values array must be empty. This array is replaced during a strategic
483+
merge patch.
484+
items:
485+
type: string
486+
type: array
487+
required:
488+
- key
489+
- operator
490+
type: object
491+
type: array
492+
matchLabels:
493+
additionalProperties:
494+
type: string
495+
description: |-
496+
matchLabels is a map of {key,value} pairs. A single {key,value} in the matchLabels
497+
map is equivalent to an element of matchExpressions, whose key field is "key", the
498+
operator is "In", and the values array contains only "value". The requirements are ANDed.
499+
type: object
500+
type: object
501+
x-kubernetes-map-type: atomic
502+
type: array
451503
podConfigRef:
452504
description: |-
453505
PodConfigRef describes the pod spec with wich this action shall be executed.

‎config/crd/apiextensions.k8s.io/v1/k8up.io_schedules.yaml‎

Lines changed: 52 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -1756,6 +1756,58 @@ spec:
17561756
17571757
Deprecated: Use FailedJobsHistoryLimit and SuccessfulJobsHistoryLimit respectively.
17581758
type: integer
1759+
labelSelectors:
1760+
description: |-
1761+
LabelSelectors is a list of selectors that we filter for.
1762+
When defined, only PVCs and PreBackupPods matching them are backed up.
1763+
items:
1764+
description: |-
1765+
A label selector is a label query over a set of resources. The result of matchLabels and
1766+
matchExpressions are ANDed. An empty label selector matches all objects. A null
1767+
label selector matches no objects.
1768+
properties:
1769+
matchExpressions:
1770+
description: matchExpressions is a list of label selector
1771+
requirements. The requirements are ANDed.
1772+
items:
1773+
description: |-
1774+
A label selector requirement is a selector that contains values, a key, and an operator that
1775+
relates the key and values.
1776+
properties:
1777+
key:
1778+
description: key is the label key that the selector
1779+
applies to.
1780+
type: string
1781+
operator:
1782+
description: |-
1783+
operator represents a key's relationship to a set of values.
1784+
Valid operators are In, NotIn, Exists and DoesNotExist.
1785+
type: string
1786+
values:
1787+
description: |-
1788+
values is an array of string values. If the operator is In or NotIn,
1789+
the values array must be non-empty. If the operator is Exists or DoesNotExist,
1790+
the values array must be empty. This array is replaced during a strategic
1791+
merge patch.
1792+
items:
1793+
type: string
1794+
type: array
1795+
required:
1796+
- key
1797+
- operator
1798+
type: object
1799+
type: array
1800+
matchLabels:
1801+
additionalProperties:
1802+
type: string
1803+
description: |-
1804+
matchLabels is a map of {key,value} pairs. A single {key,value} in the matchLabels
1805+
map is equivalent to an element of matchExpressions, whose key field is "key", the
1806+
operator is "In", and the values array contains only "value". The requirements are ANDed.
1807+
type: object
1808+
type: object
1809+
x-kubernetes-map-type: atomic
1810+
type: array
17591811
podConfigRef:
17601812
description: |-
17611813
PodConfigRef describes the pod spec with wich this action shall be executed.

‎docs/modules/ROOT/examples/usage/operator.txt‎

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -11,6 +11,7 @@ OPTIONS:
1111
--annotation value the annotation to be used for filtering (default: "k8up.io/backup") [$BACKUP_ANNOTATION]
1212
--backupcommandannotation value set the annotation name that identify the backup commands on Pods (default: "k8up.io/backupcommand") [$BACKUP_BACKUPCOMMANDANNOTATION]
1313
--fileextensionannotation value set the annotation name where the file extension is stored for backup commands (default: "k8up.io/file-extension") [$BACKUP_FILEEXTENSIONANNOTATION]
14+
--global-backoff-limit value set the backoff limit for all backup jobs (default: 6) [$BACKUP_GLOBAL_BACKOFF_LIMIT]
1415
--global-failed-jobs-history-limit value set the number of old, failed jobs to keep when cleaning up, applies to all job types (default: 3) [$BACKUP_GLOBAL_FAILED_JOBS_HISTORY_LIMIT]
1516
--global-successful-jobs-history-limit value set the number of old, successful jobs to keep when cleaning up, applies to all job types (default: 3) [$BACKUP_GLOBAL_SUCCESSFUL_JOBS_HISTORY_LIMIT]
1617
--global-concurrent-archive-jobs-limit value set the limit of concurrent archive jobs (default: unlimited) [$BACKUP_GLOBAL_CONCURRENT_ARCHIVE_JOBS_LIMIT]

‎docs/modules/ROOT/pages/how-tos/backup.adoc‎

Lines changed: 67 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -219,3 +219,70 @@ spec:
219219
podConfigRef:
220220
name: podconfig
221221
----
222+
223+
== Target specific PVCs or PreBackupPods
224+
225+
An optional labelSelectors field can be specified to target PVCs or PreBackupPods matching those expressions.
226+
You can specify multiple selectors - as long as at least one matches, the PVC/PreBackupPod will be included in the backup.
227+
228+
Keep in mind that does NOT apply to terms within an individual labelSelector - this will be processed as usual, as we use the standard K8s API for this.
229+
To find out how selectors themselves work, you can consult the upstream Kubernetes documentation: https://kubernetes.io/docs/concepts/overview/working-with-objects/labels/
230+
231+
Below you can find a practical example. It would back up any resources with "my-label-key" defined OR "another-label-key" with one of the accepted values.
232+
233+
[source,yaml]
234+
----
235+
apiVersion: k8up.io/v1
236+
kind: Backup
237+
metadata:
238+
name: backup-test
239+
spec:
240+
labelSelectors:
241+
- matchExpressions:
242+
- key: my-label-key
243+
operator: Exists
244+
- matchExpressions:
245+
- key: another-label-key
246+
operator: In
247+
values:
248+
- acceptable-value
249+
- another-acceptable-value
250+
failedJobsHistoryLimit: 2
251+
successfulJobsHistoryLimit: 2
252+
backend:
253+
repoPasswordSecretRef:
254+
name: backup-repo
255+
key: password
256+
s3:
257+
endpoint: http://minio:9000
258+
bucket: backups
259+
accessKeyIDSecretRef:
260+
name: minio-credentials
261+
key: username
262+
secretAccessKeySecretRef:
263+
name: minio-credentials
264+
key: password
265+
266+
----
267+
268+
If you'd like to only target entities having both labels, you can use a single labelSelector.
269+
This change will cause the backup to select only entities matching both conditions.
270+
271+
[source,yaml]
272+
----
273+
apiVersion: k8up.io/v1
274+
kind: Backup
275+
metadata:
276+
name: backup-test
277+
spec:
278+
labelSelectors:
279+
- matchExpressions:
280+
- key: my-label-key
281+
operator: Exists
282+
- key: another-label-key
283+
operator: In
284+
values:
285+
- acceptable-value
286+
- another-acceptable-value
287+
...
288+
----

‎docs/modules/ROOT/pages/references/api-reference.adoc‎

Lines changed: 4 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -257,6 +257,8 @@ KeepJobs is used property is not specified.
257257
information about the snapshots to. This is in addition to the prometheus
258258
pushgateway.
259259
| *`tags`* __string array__ | Tags is a list of arbitrary tags that get added to the backup via Restic's tagging system
260+
| *`labelSelectors`* __link:https://kubernetes.io/docs/reference/generated/kubernetes-api/v1.20/#labelselector-v1-meta[$$LabelSelector$$] array__ | LabelSelectors is a list of selectors that we filter for.
261+
When defined, only PVCs and PreBackupPods matching them are backed up.
260262
|===
261263

262264

@@ -299,6 +301,8 @@ KeepJobs is used property is not specified.
299301
information about the snapshots to. This is in addition to the prometheus
300302
pushgateway.
301303
| *`tags`* __string array__ | Tags is a list of arbitrary tags that get added to the backup via Restic's tagging system
304+
| *`labelSelectors`* __link:https://kubernetes.io/docs/reference/generated/kubernetes-api/v1.20/#labelselector-v1-meta[$$LabelSelector$$] array__ | LabelSelectors is a list of selectors that we filter for.
305+
When defined, only PVCs and PreBackupPods matching them are backed up.
302306
|===
303307

304308

‎e2e/definitions/backup/backup-selectors.yaml‎

Lines changed: 34 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,34 @@
1+
---
2+
apiVersion: k8up.io/v1
3+
kind: Backup
4+
metadata:
5+
name: k8up-backup-selectors
6+
namespace: k8up-e2e-subject
7+
spec:
8+
labelSelectors:
9+
- matchExpressions:
10+
- key: exists
11+
operator: Exists
12+
- matchExpressions:
13+
- key: specific-values
14+
operator: In
15+
values:
16+
- specific-value-1
17+
- specific-value-2
18+
failedJobsHistoryLimit: 1
19+
successfulJobsHistoryLimit: 1
20+
backend:
21+
repoPasswordSecretRef:
22+
name: backup-repo
23+
key: password
24+
s3:
25+
endpoint: http://minio.minio-e2e.svc.cluster.local:9000
26+
bucket: backup
27+
accessKeyIDSecretRef:
28+
name: backup-credentials
29+
key: username
30+
secretAccessKeySecretRef:
31+
name: backup-credentials
32+
key: password
33+
podSecurityContext:
34+
runAsUser: $ID

‎e2e/definitions/prebackup/prebackup-match-labels.yaml‎

Lines changed: 38 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,38 @@
1+
---
2+
apiVersion: k8up.io/v1
3+
kind: PreBackupPod
4+
metadata:
5+
name: prebackup-label-specific-value-1
6+
namespace: k8up-e2e-subject
7+
labels:
8+
specific-values: specific-value-1
9+
spec:
10+
backupCommand: sh -c 'echo hello there'
11+
pod:
12+
spec:
13+
containers:
14+
- image: busybox
15+
command:
16+
- 'sleep'
17+
- 'infinity'
18+
imagePullPolicy: Always
19+
name: specific-label-value-1
20+
---
21+
apiVersion: k8up.io/v1
22+
kind: PreBackupPod
23+
metadata:
24+
name: prebackup-label-exists
25+
namespace: k8up-e2e-subject
26+
labels:
27+
exists: arbitrary
28+
spec:
29+
backupCommand: sh -c 'echo whatup'
30+
pod:
31+
spec:
32+
containers:
33+
- image: busybox
34+
command:
35+
- 'sleep'
36+
- 'infinity'
37+
imagePullPolicy: Always
38+
name: arbitrary-label-value

0 commit comments

Comments
 (0)