x509pop grouping selector

[kfox1111]

Problem Statement

When deploying a the spire-iidentity-exchange alongside a spire-agent inside a standard Kubernetes Deployment (rather than a DaemonSet), each individual pod's spire-agent must obtain a unique SPIFFE ID via a node attestation method like x509pop to avoid kicking each other out.

However, because these agents act on behalf of a singular, logical "identity exchange" service, a SPIRE administrator needs a clean way to register downstream workload entries using a single, static parentID (e.g., spiffe://example.org/spire-identity-exchange). Currently, there is a missing link to bridge the gap between unique per-agent SPIFFE IDs and a common identity/selector that a Node Alias can target.

Without this, managing individual registration entries for every scaled replica of an identity-exchange pod becomes unmaintainable.

Proposed Solution Architecture

The goal is to deploy spire-identity-exchange as a standard Kubernetes Deployment.

The host-level agents run standard node attestation (e.g., k8s_psat).

The specialized Deployment pods use the SPIFFE CSI driver to receive an SVID, then use x509pop to attest to the SPIRE server.

The server needs to generate a non-unique, common selector based on the attested agent's path metadata so a Node Alias can target all instances of this deployment.

Considered Options

Option 1: Introduce Path/Parent Selector Logic to x509pop (Preferred)

Modify the x509pop server node attestor to generate a selector based on a "parent" or "trimmed" path derived from the configured agent_path_template.

Example Structure:

SVID provided to agent: spiffe://spire-exchange/k8s/clusterfoo/spire-identity-exchange/abcdefg

Generated Agent SVID: spiffe://spire/agent/x509pop/k8s/clusterfoo/spire-identity-exchange/abcdefg

New Auto-Generated Selector:
x509pop:agent_path_parent:spiffe://spire/agent/x509pop/k8s/clusterfoo/spire-identity-exchange (analogous to a dirname operation on the path template).

A single Node Alias entry can then target x509pop:agent_path_parent:spiffe://spire/agent/x509pop/k8s/clusterfoo/spire-identity-exchange and project the target spiffe://example.org/spire-identity-exchange parent ID to all instances seamlessly.

Option 2: Static Group Config for x509pop

Introduce an option such as extra_groups or static selectors into the x509pop configuration that allows adding generic classification selectors during attestation.

Example Config: config.x509pop.extraGroups = ["spire-identity-exchange"]

Resulting Selector: x509pop:static_group:spire-identity-exchange

[amartinezfayo]

Thanks for laying out the options, @kfox1111. After digging into this, I think the gap is narrower than it looks, and I'd like to suggest a direction that I believe fits SPIRE's model a bit more naturally than either option here.

The grouping half of this is already a well-supported pattern: a Node Alias that matches a stable, non-unique selector. Most attestors emit those by design, and k8s_psat already produces cluster, agent_ns, agent_sa, and agent_pod_label, so a single alias on something like k8s_psat:agent_sa:<sa> would project a common parent to every replica. x509pop is the outlier: a stock SPIRE SVID gives it no usable group selector, since the CN is empty and ca:fingerprint is the whole trust domain. If I'm reading your credential composer right, that's exactly what it works around, stamping a shared CN onto the SVIDs so x509pop emits a shared subject:cn selector to group on. That's the plumbing you'd like to retire.

I think the deeper root cause is that k8s_psat derives a node-scoped agent ID (k8s_psat/<cluster>/<nodeUID>), so two identity-exchange pods on the same node collide on that ID and evict each other, which is what pushes you onto x509pop and the composer in the first place. The grouping selectors are already there. The only thing missing is a unique-per-pod agent ID.

So rather than add a new selector to x509pop, I wonder if the cleaner fix is to let k8s_psat's agent path be templated, the same way aws_iid, gcp_iit, and x509pop already support agent_path_template. Setting something like agent_path_template = "/{{ .PluginName }}/{{ .Cluster }}/{{ .PodUID }}" would give a unique ID per pod, and the existing agent_sa (optionally scoped with agent_ns) or agent_pod_label selectors would carry the grouping with no new machinery and no composer. The pod UID is already attested through the TokenReview path, and per-pod IDs model reality better anyway, since each pod genuinely runs its own agent and is really a distinct node. The node-scoped path bakes in a one-agent-per-node assumption that doesn't hold for a Deployment.

My main hesitation with Option 1 is that agent_path_parent would be a bespoke selector coupled to the configured agent_path_template, so changing the template would silently break every alias built on it. Two caveats with the templating approach: the default template has to reproduce today's path exactly so existing agent IDs don't orphan, and per-pod IDs mean every pod becomes its own attested node entry, though that churn is exactly what the current x509pop spiffe-mode approach already pays.

On the security side, I'd limit the templatable fields to the k8s-vouched identifiers (pod UID, pod name, namespace, service account, node UID/name) and keep free-form labels and annotations out of the path, so the ID-composition surface stays as trustworthy as it is today. Steering toward pod UID for uniqueness also matters here, since k8s_psat sets CanReattest and has no attest-once fallback if two agents collide on an ID.

Curious what you and others think.

[kfox1111]

Hi @amartinezfayo

Thats a very interesting idea. I do think that is worth doing, but it assumes a bit that attesting with a spiffeid was done because the k8s attestor wasnt capable/available. But its a problem that exists outside of k8s, and the choice to use spiffe x509pop was more specific then an unavailable other node attestor.

It is done in the bottom turtle implementations of the spire agent as well. where you have an agent outside of k8s, and then want to chain trust into the agent within k8s.

But I think that is the key piece of the puzzle. And an important one. Chaining trust.

Say I have a tpm on the host. The host's spire-agent is the only one that should talk to the tpm and attest with it. Otherwise, a different agent would be able to attest with it and there would be no way for a spire server to tell agents on that node apart. So one agent needs to be a bottom turtle of agents when multiple agents are involved.

Then, how do you use that established trust to launch other spire-agents? thats where the spiffe x509pop attestor comes in. The bottom turtle agent can workload attest a particular agent, and give it a svid, and then the spire-server can use that svid to establish the other spire-agent's identity cleanly. The bottom turtle spire-agent's trust is used to establish the child spire-agents trust.

In a k8s cluster with a bottom turtle setup, the k8s node attestor is a weaker attestation mechanism to the spiffe x509pop one because of this chain of trust. It can prove a given pod to a child k8s-spire-agent to a bottom turtle agent to a tpm to the spire-server without just believing k8s's api server. Its all done on the node itself attesting at each step in the chain.

So, I do think its useful to be able to enhance the x509pop plugin to be able to do this sort of grouping thing. As these more advanced, chained trust type models are used, I think we will continue to hit this issue.

---

Separate to this, it would still be great if we could stack attestors too. We're having to make a choice in security... can we chain trust with spiffe x509pop, or use k8s attestor to vouch for a spire-agent. But ideally, you would do both. Be able to chain trust via the bottom turtle spire-agent AND ensure its actually running in k8s. With the lack of stacking, we're leaving some really good security options off the table.

[matheuscscp]

Hey @kfox1111, I think @amartinezfayo's suggestion is essentially a template-based version of what I proposed in #7116.

I think a k8s_psat-based solution is superior than the one proposed in #7002 because:

• It is a mechanism built into the SPIRE server that is capable of assigning unique SPIFFE IDs to the Deployment-based agents. It does not depend on an external mechanism to ensure the Deployment Pods will get unique SPIFFE IDs to avoid having them kicking each other out. ClusterSPIFFEID is an external dependency, relying on it to ensure the Deployment agents get unique SPIFFE IDs is very bad in my opinion. SPIRE should solve this natively, with built-in features.
• It is a direct evolution of today's solution for DaemonSet-based agents that entirely replaces the coarse-grained node-UID-based SPIFFE IDs. I can see from the Git history that Pods were originally the atomic workload unit before, and the k8s node attestor evolved to today's node-based approach so as to align with SPIRE's general node-based architecture for other platforms. In my opinion, this was a regression, as in Kubernetes Pods are the atomic workload unit, and SPIRE agents running as DaemonSet Pods are still pods. If a SPIRE user opts into giving DaemonSet agents Pod-based SPIFFE IDs, they remain unique, which is the required property for SPIRE agents. And, on top of that, multiple SPIRE agents running on the same node become supported. All because we are leveraging Kubernetes' golden paths rather than forcing an agnostic model that tries to fit all platforms.
• It's very hard to reason about the selector proposed in Add an agent_id_parent selector to x509pop in spiffe mode #7002, it feels very niche and hard to justify.

TLDR:

• k8s_psat has no extra dependencies, it's a built-in solution.
• k8s_psat is a Kubernetes-native solution, things flow naturally.
• PR Add an agent_id_parent selector to x509pop in spiffe mode #7002 does not look good.

If this comes from "SPIFFE's proof of possession is always superior to PSAT", I think this example does a very good job showing this is not always true. The DaemonSet agents are already bootstrapped via PSAT. Going from PSAT to PoP for child agents does not improve anything, as the bootstrap was already done via PSAT anyway, so any PoP stemming from it is inherently grounded in the PSAT bootstrap. Basically, if you don't trust the PSAT bootstrap, then you don't trust SPIRE for Kubernetes at all.

If there was a native way to bootstrap the DaemonSet pods via PoP instead of PSAT, then the story here would be different. In that case, maintaining PoP for child agents would be valuable. This is unfortunately not the case.

[kfox1111]

No, I dont think your listening. I can not be using k8s_psat for agents on my own clusters until node attestor stacking is supported because it can not safely be trust chained all the way to a hardware TPM. so it is not a viable solution for me. That means, this PR stands on its own. Feel free to implement k8s_psat support. Like, I said, I support implementing that. But it is not sufficient to solve my problem. So lets not tie the two together as an either/or. Both is an option. And considering how little code I'm proposing, not a burdensome ask.

I think a k8s_psat-based solution is superior than the one proposed in #7002 because:

It is a mechanism built into the SPIRE server that is capable of assigning unique SPIFFE IDs to the Deployment-based agents.

k8s_psat is not capable of that today. Not sure your point.

It does not depend on an external mechanism to ensure the Deployment Pods will get unique SPIFFE IDs to avoid having them kicking each other out.

What does that mean? Neither should x509pop, which is the whole point of this pr. Also, see below. I think there is a logic error here that I'll explain in a minute.

ClusterSPIFFEID is an external dependency, relying on it to ensure the Deployment agents get unique SPIFFE IDs is very bad in my opinion. SPIRE should solve this natively, with built-in features.

It can be "naitively" solved via registration entry as well.

It is a direct evolution of today's solution for DaemonSet-based agents that entirely replaces the coarse-grained node-UID-based SPIFFE IDs.

I can see from the Git history that Pods were originally the atomic workload unit before, and the k8s node attestor evolved to today's node-based approach so as to align with SPIRE's general node-based architecture for other platforms. In my opinion, this was a regression, as in Kubernetes Pods are the atomic workload unit, and SPIRE agents running as DaemonSet Pods are still pods. If a SPIRE user opts into giving DaemonSet agents Pod-based SPIFFE IDs, they remain unique, which is the required property for SPIRE agents. And, on top of that, multiple SPIRE agents running on the same node become supported.

This is the biggest hole in the argument.

No, your pushing the problem somewhere else, not solving it. it makes it significantly harder to reason about what workloads should be bound to a k8s_psat based agent pod uuid when there are multiple per node. Which ones go to the daemonset ones as a normal agent. which ones go to deployment 123 pod uid abcd because they are spire-identity-exchange, and which ones go to deployment 124 pod uid defg because its for flux?

This requires clusterspiffeid's to now template out somehow the unknown node pod uid in the parent id. So, not only is it leaning on clusterspiffeids, its greatly expanding the logic on them.. This is much more problematic then adding a single, regular, clusterspiffeid representing the extra agent.

It will require an unknown complex solution on the spire-controller-manager to solve the above.

The agent path grouping solves this problem, today with a few lines of code, once and for all. Its an optimization of something already working reliably on some of my clusters, today with an external plugin. I will continue to ship the solution until this optimization is made. I already have a working soluiton. This pr just makes my life slightly easier and that of users wanting to do the same thing.

I suggest you should implement a total, end to end, working solution before proposing strongly that an already working, end to end solution is not the right solution. I think you will likely find its the cleanest solution in the end. Such solutions in spire tend to be very complex and hard to reason about til the end of a working solution in my experience. :/ You often find that a simple sounding optimization in one place, makes the overall solution much harder in another place.