Add k8s name related formats #384

jpbetz · 2023-04-12T16:38:03Z

This adds the most heavily used name formats in k8s:

k8s.io/short-name (represents the DNS 1123 label format)
k8s.io/long-name (represents the DNS 1123 subdomain format)

Note: These formats implemented in k8s in ways that do not exactly match the RFCs (kubernetes/kubernetes#71140,
kubernetes/kubernetes#79351). But, intend to use these formats to communicate the existing Kubernetes validation rules via OpenAPI, so we want to keep them as-is.

Benefits:

Easier for CRDs to use the same name format already used in k8s.
Useful for publication of OpenAPI from Declarative Validation

xref: https://json-schema.org/draft/2020-12/json-schema-validation#name-custom-format-attributes, https://spec.openapis.org/registry/format/

liggitt · 2023-04-12T16:48:31Z

are non-standard format strings supposed to be namespaced or not?

what are the implications for CRDs created using these format strings prior to this, allowing in data that ignores those format requirements, then auto-tightening validation when upgrading to a control plane with this support added?

jpbetz · 2023-04-12T20:33:25Z

are non-standard format strings supposed to be namespaced or not?

I have not found any mention of namespacing in the specs. xref https://datatracker.ietf.org/doc/html/draft-bhutton-json-schema-validation-01#name-custom-format-attributes

I am open to naming suggestions for these formats though. I had briefly considered something like "kubernetes-name" but couldn't think of anything that aided rather than harmed clarity on what the format actually is.

what are the implications for CRDs created using these format strings prior to this, allowing in data that ignores those format requirements, then auto-tightening validation when upgrading to a control plane with this support added?

Yes, exactly.

Today, supported formats is used strip out unsupported
formats when used in older version of the kube-apiserver. We could extend this to have
two lists: supported and storage. All new formats will be added to storage for
at least one minor release of Kubernetes before the format is added to supported so
that any usages of the new formats written to CRDs downgrade safely.

jpbetz · 2023-04-26T16:21:38Z

/assign @apelisse

Would you be willing to review?

apelisse

looks good thanks

pkg/validation/strfmt/default.go

apelisse · 2023-04-26T17:55:28Z

Thanks, looks better to me :-)

liggitt · 2023-04-27T19:06:59Z

pkg/validation/strfmt/default.go

@@ -124,6 +124,48 @@ func IsEmail(str string) bool {
 	return e == nil && addr.Address != ""
 }

+const dns1123LabelFmt string = "[a-z0-9]([-a-z0-9]*[a-z0-9])?"


if we're copying/pasting from apimachinery validation, are we inheriting these issues:

kubernetes/kubernetes#71140

This requires things to be lowercase, which is more strict than the RFC

kubernetes/kubernetes#79351 (comment)

This tolerates individual labels exceeding 63 chars, which is not as strict as the RFC

Should we fix those issues before adopting those formats here?

We intend to publish OpenAPI the in-tree types using these formats. So I think we're stuck inheriting the it all.

That said, we shouldn't using the RFC identifiers for formats that are not exactly compliant with the RFC. I'll propose some alternative names.

liggitt · 2023-04-27T19:07:41Z

pkg/validation/strfmt/default.go

+	dns1123subdomain := DNS1123Subdomain("")
+	Default.Add("dns1123subdomain", &dns1123subdomain, IsDNS1123Subdomain)
+
+	dns1123label := DNS1123Label("")
+	Default.Add("dns1123label", &dns1123label, IsDNS1123Label)
+
+	dns1035label := DNS1035Label("")
+	Default.Add("dns1035label", &dns1035label, IsDNS1035Label)


if these get added to the default set of recognized formats here, how do we roll them out in a controlled way in kubernetes?

This was held until CRD Ratcheting was promoted to beta in 1.30, we'll leverage that capability.

I'll update this code so we can manage the newly added formats separate from the pre-existing ones.

liggitt · 2023-04-27T19:20:55Z

what are the implications for CRDs created using these format strings prior to this, allowing in data that ignores those format requirements, then auto-tightening validation when upgrading to a control plane with this support added?

Yes, exactly.

Today, supported formats is used strip out unsupported formats when used in older version of the kube-apiserver. We could extend this to have two lists: supported and storage. All new formats will be added to storage for at least one minor release of Kubernetes before the format is added to supported so that any usages of the new formats written to CRDs downgrade safely.

I'm not sure I understand how that makes existing persisted CR data safe. However long we take to roll out support for the format, at some point, would we start treating existing persisted data that didn't match the specified format as invalid?

jpbetz · 2023-05-04T21:51:29Z

I'm not sure I understand how that makes existing persisted CR data safe. However long we take to roll out support for the format, at some point, would we start treating existing persisted data that didn't match the specified format as invalid?

We could solve this together with the more general problem of CRD validation ratcheting. I.e. the case where a CRD author adds a format to a CRD, and there already persisted CR data which then immediately becomes invalid. The case we are discussing is similar except that instead of the CRD author adding a format, the CRD author already had a non-validating format in the CRD and then Kubernetes changed the format to be validating.

Ratcheting rule would be: if any CR data fails validation but is unchanged from the persisted value, we ignore the validation failure and allow the update.

EDIT: This is also what we need for rollback. Note that because any format is allowed on any CRD, this would allow for rollback to arbitrary old versions.

liggitt · 2023-05-09T18:16:21Z

We could solve this together with the more general problem of CRD validation ratcheting.

I agree a solution to that would resolve the issue with this. Shouldn't we get that solution in place before adding this? This would be the first example I can think of where we made a change that would reinterpret existing persisted CRDs in ways that would invalidate existing data

jpbetz · 2023-05-09T19:04:18Z

We could solve this together with the more general problem of CRD validation ratcheting.

I agree a solution to that would resolve the issue with this. Shouldn't we get that solution in place before adding this? This would be the first example I can think of where we made a change that would reinterpret existing persisted CRDs in ways that would invalidate existing data

SGTM. I'll poke around and see who might be able to implement.

jpbetz · 2023-07-21T01:22:47Z

/close
When we circle back on this we'll either reopen or create a new PR.

k8s-ci-robot · 2023-07-21T01:22:52Z

@jpbetz: Closed this PR.

In response to this:

/close
When we circle back on this we'll either reopen or create a new PR.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

alexzielenski · 2023-08-18T23:18:30Z

We could solve this together with the more general problem of CRD validation ratcheting.
I agree a solution to that would resolve the issue with this. Shouldn't we get that solution in place before adding this?

Just updating progress on this front was made with CRD Ratcheting Alpha added in 1.28: https://kep.k8s.io/4008

Will need to wait for it to mature before using it for name formats, but in the interim we can use inline OpenAPI schemas to describe the regex/cel expressions used for most formats.

jpbetz · 2024-06-28T17:38:20Z

/reopen

CRD Ratcheting was promoted to beta in 1.30, freeing us up to resume progress on this.

k8s-ci-robot · 2024-06-28T17:38:26Z

@jpbetz: Reopened this PR.

In response to this:

/reopen

CRD Ratcheting was promoted to beta in 1.30, freeing us up to resume progress on this.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository.

liggitt · 2025-03-20T19:42:05Z

Now that we have CRD ratcheting, it would be good to give guidance around when expanded openapi formats are the preferred way to model a validation, and when a CEL library is.

xref kubernetes/kubernetes#124490 (comment)

jpbetz · 2025-03-21T01:51:30Z

Now that we have CRD ratcheting, it would be good to give guidance around when expanded openapi formats are the preferred way to model a validation, and when a CEL library is.

xref kubernetes/kubernetes#124490 (comment)

We are missing well documented guidance. Let's get that going. I like the idea of CEL "supplementing" OpenAPI formats only were clearly needed.

Related- We've fallen behind on supporting formats. kubernetes/kubernetes#130639 describes where we haveCEL support for formats that SHOULD have an OpenAPI format.

To help catch up, we've added emulation version support to formats: kubernetes/kubernetes#130783. Combined with ratcheting, this should give us all the machinery we need to add the formats we agree should be added.

jpbetz · 2025-03-21T01:53:57Z

cc @thockin for the Declarative Validation aspect. We anticipate needing OpenAPI formats for the built-in formats that we validate with hand written code so that we can publish the declarative validation rules through OpenAPI.

thockin · 2025-03-21T23:34:37Z

Is there an extension-mechanism for format names? Like x-something or example.com/something?

The ultimate goal, I think is unification. CRDs can be written saying "this field is validated by format=x-k8s-dns-label" and we will do that server-side; Builtin types can express the same thing in a different way; apiserver publishes this in a way that an "enlightened" client can handle if they wanted to to do shift-left validation (e.g. as a git pre-commit).

jpbetz · 2025-03-25T15:03:25Z

This PR has been refreshed and is ready for review.

I've gone with "k8s.io/" as the prefix after consulting a few LLMs about widely used conventions . The main feedback was that standard bodies are steering away from "x-" prefixes, and that the charset in this prefix is within norms.

thockin

A couple thoughts on names. Over time we have used 3 different DNS formats. But what we use is not really DNS compatible, it's DNS-ish.

Should we continue with the "DNS" nomenclature? This looks like a significant move towards publishing this terminology so this may be our last chance to name the formats better. Or it might be too late already.
Should we nominate one of the 3 as the "normal" format (e.g. k8s.io/dns-label) and let the others be wierd?

jpbetz · 2025-03-25T16:36:52Z

A couple thoughts on names. Over time we have used 3 different DNS formats. But what we use is not really DNS compatible, it's DNS-ish.

Should we continue with the "DNS" nomenclature? This looks like a significant move towards publishing this terminology so this may be our last chance to name the formats better. Or it might be too late already.

Should we nominate one of the 3 as the "normal" format (e.g. k8s.io/dns-label) and let the others be wierd?

Since DNS 1123 subdomain is the type we use most often for resource name validation, how about a format name that emphasizes that? "k8s.io/basic-name" or something?

jpbetz · 2025-03-25T16:55:16Z

How about:

k8s.io/qualified-identifier (dns 1123 subdomain)
k8s.io/identifier (dns 1123 label)
k8s.io/strict-identifier (dns 1035 label)

thockin · 2025-03-28T23:22:42Z

I am terrible at naming things, but a few quick thoughts:

DNS 1123 Subdomain and Label are most common, 1035 less so, and it looks like 952 has been weeded out. So 1123 is "obviously" the default.
We already use the word "qualified" in "qualified name". I hate it there, too.
We should avoid "simple" in names.
AFAICT Service is the last and only resource which demands 1035. Maybe we can ratchet it to 1123? Can we do the same for other users of 1035?

Then the name could just be "dns-label" or "dns-style-label" or something? Would love alternatives to "qualified name" too :)

jpbetz · 2025-03-29T00:33:41Z

We already use the word "qualified" in "qualified name". I hate it there, too.

Maybe we just stick with subdomain then? We could consider "dns-full-name" or something. But if we're avoiding "qualified", is "full name" any better?

We should avoid "simple" in names.

+1

AFAICT Service is the last and only resource which demands 1035. Maybe we can ratchet it to 1123? Can we do the same for other users of 1035?

Dropping 1035 from the format list SGTM.

Then the name could just be "dns-label" or "dns-style-label" or something? Would love alternatives to "qualified name" too :)

Curious what value to the user using the term "DNS" adds? That aside, how about:

k8s.io/dns-subdomain (dns 1123 subdomain)
k8s.io/dns-subdomain-prefix
k8s.io/dns-label (dns 1123 label)
k8s.io/dns-label-prefix

?

jpbetz · 2025-03-29T00:39:28Z

My current concerns:

Not sure what value the term "DNS" really adds to the users of these formats (they don't conform to DNS specs.. even though they ALMOST conform)
The user of the term "label" is pulling in a term from DNS feels like it could confuse anyone that is thinking in k8s terms and immediately starts thinking of k8s labels when they see it.

liggitt · 2025-03-31T15:10:06Z

I agree that invoking DNS is a bit confusing, especially if we invoke a particular detailed RFC which we then don't quite implement correctly (waves vaguely at all the issues complaining our 1123 validation... isn't)

I also agree that "label" is likely to confuse people in the context of Kubernetes. Perhaps "segment" would be clearer?

Taking a step back, I'm a little fuzzy on which of these goals we're trying to accomplish:

create formats that exactly match all currently used in-tree validations (with their corresponding bugs / inconsistencies / variants) for use by CRDs
create formats that exactly match all currently used in-tree validations (with their corresponding bugs / inconsistencies / variants) so we can publish openapi for in-tree types that can be translated into CRDs that will use identical validation
create formats that exactly match all currently used in-tree validations (with their corresponding bugs / inconsistencies / variants) so we can switch in-tree types to use generated validation
create formats that are based on in-tree validations but fix bugs / inconsistencies in them
evolve / change (either tightening or relaxing) validation of in-tree types

Example bugs:

IsDNS1123Subdomain allows individual label segments to exceed 63 characters (overall length is restricted, but the regex for the subdomain doesn't constraint the length of individual labels)

Example inconsistencies / variants:

allowing or disallowing underscores
allowing or disallowing mixed-case
allowing or disallowing leading digits in segments
allowing all-numeric segments

thockin · 2025-04-01T16:26:16Z

Jordan, I think this was aiming at:

create formats that exactly match all currently used in-tree validations (with their corresponding bugs / inconsistencies / variants) so we can publish openapi for in-tree types that can be translated into CRDs that will use identical validation

create formats that exactly match all currently used in-tree validations (with their corresponding bugs / inconsistencies / variants) so we can switch in-tree types to use generated validation

If we can find a better name to use in this that doesn't reference DNS and doesn't abuse the word "label" (which I never really recognized before but now cannot unsee), then that's great.

If we don't need to handle Service, perhaps we can use 1.34 to alpha-gate service name ratcheting to the 1123 -ish spec. 1.34 would tolerate services with 1123-style names, but not allow new ones. 1.35 would allow new usage. Unless we can lean on compat-version to save the alpha-release? @danwinship @aojea

Then we only have to name one thing instead of two.

"kubernetes short name" vs "... long name" ?

jpbetz · 2025-04-08T14:18:45Z

I've updated this to use k8s.io/short-name and k8s.io/long-name

danwinship · 2025-04-08T19:52:10Z

If we don't need to handle Service

I think either way, we should probably ignore Service for purposes of this PR; either we manage to fix Service and it becomes just a k8s.io/short-name, or we don't, and we can add k8s.io/service-name later.

thockin · 2025-04-16T20:11:36Z

I asked OAI/learn.openapis.org#128 to see if there is a real norm here. I like the k8s.io/something extension mechanism, but from reading a lot of the issues around this, I wonder if just k8s-something is more "regular"?

k8s-short-name + k8s-long-name ?

k8s-shortname + k8s-longname ?

k8s-name + k8s-long-name ?

k8s-shortname + k8s-name

jpbetz · 2025-04-16T20:17:17Z

I wonder if just k8s-something is more "regular"?

We've gone full circle. This is what I'd proposed originally.

thockin · 2025-04-20T19:53:04Z

We have two open questions in this thread:

1) Overall notation

Almost full circle - after looking at all the openapi and JSON schema got repos and specs I figured:

Nothing in the spec prevents us from using k8s.io/something
Nobody else seems to use any notation other than lower-case-with-dashes for format names
It seems conventional to use a "sufficiently unique" prefix for locally defined format names

As such, I think the lowest risk and most non-contentious path is to declare a prefix (eg. k8s-) and use it on all of our custom format names. We should use k8s-something-name style, even though it is not the same syntax as label names.

I think this should extend into declarative validation, and all our custom formats should be "k8s-something"

Agree or discuss more?

2) Rename "DNS label" and "DNS subdomain"

I can't come up with anything more semantically meaningful that "short" and "long". Maybe just more descriptive. We want to eschew the words "DNS" and "label" in the name. "object name" isn't right because we use them other places. Even "name" is questionable. The short form is similar to a hostname, but we should avoid that, too since it is not exact.

When it comes to object names we use subdomain more than label (sadly) but for other fields we seem to use label more (totally haphazard estimate by line-counting calls to each func).

So I am fine with "k8s-short-name" and "k8s-long-name" or maybe with dropping either "long" or "short" (making that one more default-ish).

Or I am fine with "k8s-name" (or "k8s-short-name") and "k8s-dotted-name" or something similarly descriptive.

You are closest to the consequences of this decision, Joe, so I defer.

jpbetz · 2025-05-08T01:29:05Z

Apologies for the delay, I lost track of this one and am just circling back now.

Agree or discuss more?

Agree.

So I am fine with "k8s-short-name" and "k8s-long-name" or maybe with dropping either "long" or "short" (making that one more default-ish).

Or I am fine with "k8s-name" (or "k8s-short-name") and "k8s-dotted-name" or something similarly descriptive.

I also could live with any of these variations.

I'm going to go with k8s-short-name and k8s-long-name.

While I'm tempted to have just k8s-name instead of k8s-short-name, we use k8s-long-name to identify resources, so I don't think crowing k8s-short-name as "the k8s name format" is quite right.

jpbetz · 2025-05-08T01:32:50Z

@thockin @liggitt this is ready

liggitt · 2025-05-13T16:10:23Z

pkg/validation/strfmt/kubernetes-extensions.go

+
+// KubernetesExtensions is the formats registry for JSON Schema formats
+// extensions defined by the Kubernetes project.
+var KubernetesExtensions = NewSeededFormats(nil, nil)


This is peer to the existing Default var? looks like we have a lot of things using that other var... will they need to switch to this one?

I just reviewed the usages. I think it's best to just collapse these into the Default set. We still have kubernetes/kubernetes#130783 for ompat-version guided enablement.

liggitt · 2025-05-13T16:11:15Z

pkg/validation/strfmt/format.go

+	case string:
+		*r = T(v)
+	default:
+		return fmt.Errorf("cannot sql.Scan() strfmt.StringFormat from: %#v", v)


🤨 are the sql.Scan errors load-bearing or can we stop outputting that?

For some reason most of the string formats in kube-openapi support sql.Scanner and output errors like this. But a few don't. I'll just not support it and open an issue to drop it from the other string formats.

pkg/validation/strfmt/kubernetes-extensions.go

thockin · 2025-05-13T17:53:38Z

If the API validation functions were exposed in a low-deps library, would we take a dep on it here?

/lgtm

k8s-ci-robot added the cncf-cla: yes Indicates the PR's author has signed the CNCF CLA. label Apr 12, 2023

k8s-ci-robot requested review from apelisse and liggitt April 12, 2023 16:38

k8s-ci-robot added the size/L Denotes a PR that changes 100-499 lines, ignoring generated files. label Apr 12, 2023

k8s-ci-robot assigned apelisse Apr 26, 2023

apelisse reviewed Apr 26, 2023

View reviewed changes

pkg/validation/strfmt/default.go Outdated Show resolved Hide resolved

liggitt reviewed Apr 27, 2023

View reviewed changes

k8s-ci-robot closed this Jul 21, 2023

k8s-ci-robot reopened this Jun 28, 2024

jpbetz force-pushed the k8s-formats branch from 73727bd to efcb7bf Compare June 28, 2024 18:36

jpbetz closed this Aug 27, 2024

liggitt mentioned this pull request Mar 20, 2025

Wishlist of CEL libraries kubernetes/kubernetes#124490

Open

7 tasks

thockin reviewed Mar 25, 2025

View reviewed changes

jpbetz force-pushed the k8s-formats branch from 4e46223 to 18c3235 Compare April 4, 2025 20:36

k8s-ci-robot added size/L Denotes a PR that changes 100-499 lines, ignoring generated files. and removed size/XL Denotes a PR that changes 500-999 lines, ignoring generated files. labels Apr 4, 2025

thockin mentioned this pull request Apr 20, 2025

WIP: Add the idea of "cohorts" for subfield validation jpbetz/kubernetes#142

Closed

jpbetz force-pushed the k8s-formats branch from 18c3235 to 15861bc Compare May 8, 2025 01:30

liggitt mentioned this pull request May 13, 2025

Custom Resource Definition field format is not validated kubernetes/kubernetes#131082

Open

liggitt reviewed May 13, 2025

View reviewed changes

pkg/validation/strfmt/kubernetes-extensions.go Outdated Show resolved Hide resolved

Add k8s-short-name and k8s-long-name formats.

e7b662c

jpbetz force-pushed the k8s-formats branch from ac14af2 to e7b662c Compare May 13, 2025 16:40

k8s-ci-robot assigned thockin May 13, 2025

k8s-ci-robot added the lgtm "Looks good to me", indicates that a PR is ready to be merged. label May 13, 2025

Add k8s name related formats #384

Are you sure you want to change the base?

Add k8s name related formats #384

Conversation

jpbetz commented Apr 12, 2023 • edited Loading

liggitt commented Apr 12, 2023

jpbetz commented Apr 12, 2023 • edited Loading

jpbetz commented Apr 26, 2023

apelisse left a comment

Choose a reason for hiding this comment

apelisse commented Apr 26, 2023

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

liggitt commented Apr 27, 2023

jpbetz commented May 4, 2023 • edited Loading

liggitt commented May 9, 2023

jpbetz commented May 9, 2023

jpbetz commented Jul 21, 2023

k8s-ci-robot commented Jul 21, 2023

alexzielenski commented Aug 18, 2023

jpbetz commented Jun 28, 2024

k8s-ci-robot commented Jun 28, 2024

liggitt commented Mar 20, 2025

jpbetz commented Mar 21, 2025 • edited Loading

jpbetz commented Mar 21, 2025

thockin commented Mar 21, 2025

jpbetz commented Mar 25, 2025 • edited Loading

thockin left a comment

Choose a reason for hiding this comment

jpbetz commented Mar 25, 2025

jpbetz commented Mar 25, 2025 • edited Loading

thockin commented Mar 28, 2025

jpbetz commented Mar 29, 2025 • edited Loading

jpbetz commented Mar 29, 2025

liggitt commented Mar 31, 2025

thockin commented Apr 1, 2025

jpbetz commented Apr 8, 2025

danwinship commented Apr 8, 2025

thockin commented Apr 16, 2025

jpbetz commented Apr 16, 2025

thockin commented Apr 20, 2025

1) Overall notation

2) Rename "DNS label" and "DNS subdomain"

jpbetz commented May 8, 2025

jpbetz commented May 8, 2025

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

thockin commented May 13, 2025

jpbetz commented Apr 12, 2023 •

edited

Loading

jpbetz commented Apr 12, 2023 •

edited

Loading

jpbetz commented May 4, 2023 •

edited

Loading

jpbetz commented Mar 21, 2025 •

edited

Loading

jpbetz commented Mar 25, 2025 •

edited

Loading

jpbetz commented Mar 25, 2025 •

edited

Loading

jpbetz commented Mar 29, 2025 •

edited

Loading