Split the driver and retain only the individual mode

[ffromani]

We discussed on April 7, 2026 in both the "native resources in DRA" and then in SIG-Node weekly (slides).

The idea

We plan to split the grouped mode support and the memory integration and all the other enhancements we planned into a grouped-mode-only driver, and repurpose and consolidate this driver as individual-mode-only.

Essentially:
keep the existing DRA CPU driver codebase and target individual mode only

• remove the grouped mode
• featurewise, we are already done: polish, optimize, stabilize
• while this is not a commitment or a promise, the release 1.0 GA of such a refocused driver is realistic by end of 2026

create a new dra-driver-cpumem (final name TBD) grouped mode only

• remove the individual mode
• merge the memory support
• keep evolving with the ecosystem

Rationale

We currently have two very different resource representations packed in one driver.
We didn't set a goal about canonical resource representation, and different resource drivers (e.g. vendor-specific CPU drivers) can expose different resource representations. This variety brings flexibility and is almost universally seen as plus. The question is however if a single driver should expose very different resource representations.

Currently, the driver represents resources in “individual mode” and “grouped mode”, which are very different and incompatible representations. These two different modes are very different operational mode stitched together because the initial explorationatory and evolutionary nature of the project. This is unavoidable and actually fine, but we can now stop and review.

Two very different modes bundled together suggest unclear scope, and create unnecessary UX friction: "why should a driver expose so different resource models?" "which one is the best for me?" are very legit and very natural questions.

There's also a timeline consideration: featurewise, the individual mode is “done” bar stabilization.
If we keep the two modes together, it's harder to call GA or to distill clear GA criterias, because the grouped mode dilutes the GA potential; we can call a subset of features GA but this sidesteps the core tension.
In addition, the memory support composes badly with the individual mode, because memory is inherently pooled and anonymous; but the main argument stand even without considering the memory support.

Timeline

The initial idea is to split before 0.2.0 if consensus is reached, to minimize the disruption for early adopters. If we split, the sooner the better

[ffromani]

/cc @pravk03 @catblade @fmuyassarov @AutuSnow @johnbelamaric @SergeyKanzhelev

[ffromani]

Naming

the individual mode driver should at least initially retain the current name.

The grouped mode driver could be called cpumem but a review is in order if we should hint in the name that driver wants to manage core resources.
Here are the kubernetes core resources:

// ResourceName is the name identifying various resources in a ResourceList.
type ResourceName string

// Resource names must be not more than 63 characters, consisting of upper- or lower-case alphanumeric characters,
// with the -, _, and . characters allowed anywhere, except the first or last character.
// The default convention, matching that for annotations, is to use lower-case names, with dashes, rather than
// camel case, separating compound words.
// Fully-qualified resource typenames are constructed from a DNS-style subdomain, followed by a slash `/` and a name.
const (
	// CPU, in cores. (500m = .5 cores)
	ResourceCPU ResourceName = "cpu"
	// Memory, in bytes. (500Gi = 500GiB = 500 * 1024 * 1024 * 1024)
	ResourceMemory ResourceName = "memory"
	// Volume size, in bytes (e,g. 5Gi = 5GiB = 5 * 1024 * 1024 * 1024)
	ResourceStorage ResourceName = "storage"
	// Local ephemeral storage, in bytes. (500Gi = 500GiB = 500 * 1024 * 1024 * 1024)
	ResourceEphemeralStorage ResourceName = "ephemeral-storage"
)


const (
	// Default namespace prefix.
	ResourceDefaultNamespacePrefix = "kubernetes.io/"
	// Name prefix for huge page resources (alpha).
	ResourceHugePagesPrefix = "hugepages-"
	// Name prefix for storage resource limits
	ResourceAttachableVolumesPrefix = "attachable-volumes-"
)

I don't think we will have more core resources anytime soon.
About the existing resources, CPU is obviously in scope; memory and hugepages are a planned addition for the grouped mode driver, what's left is storage, ephemeral-storage and attachable-volumes. which are a hard fit for DRA since CSI exists.

While I won't mind using a name hinting at core resources, I think that finding a compact and intuitive name seems not trivial, and just using cpumem seems quite defensible.

[AutuSnow]

I have a few concerns:

1. If we want to split it, we need to do it quickly and start it quickly, because we are not sure how many users are actually using it. Of course, based on my understanding, there may be very few, but there will definitely be a few. If the current users are using both modes at the same time, how should they migrate
2. Do users who need to use two modes after splitting require two drivers?

[ffromani]

I have a few concerns:

1. If we want to split it, we need to do it quickly and start it quickly, because we are not sure how many users are actually using it. Of course, based on my understanding, there may be very few, but there will definitely be a few. If the current users are using both modes at the same time, how should they migrate

Yes, this is a decision which will affect 0.2.0 planning. We need to reach a conclusion because if we do NOT split before 0.2.0, the next practical window to split would likely be much later.

The general feedback from the communities I reached was in favor or neutral towards the split

2. Do users who need to use two modes after splitting require two drivers?

They will. In practice this would translate in fetching an extra image.
During the conversations in the commuinity around this topic, it emerged that using the same DeviceClass for the two very different existing modes was already a questionable and likely awkward decision.

[SergeyKanzhelev]

I strongly support GA-ing the split driver. I think there is enough demand for it and we can start getting feedback to shape the future rollout of more features.

[ffromani]

@SergeyKanzhelev @mrunalp @dchen1107 @haircommander would sig-node leads endorse the creation of a new repo of a combined grouped-mode-only driver? After consensus (which seems to be forming?) the repo and the naming are the only open points blocking the split and the implementation of this plan.

[haircommander]

+1 from me to split

[johnbelamaric]

+1 from me to split as well

[fmuyassarov]

Unfortunately I could not join the meeting today, but I don't see a reason not to split. This will also affect some of the open and planned patches (not drastically but few bits will need to be cut off I think).

The question I have is do we need to define a criteria for accepting/rejecting future resource representations? Without that, I'm afraid the new driver risks accumulating the same ambiguity over time. That decision what to include and what not should probably be written down somewhere.

[ffromani]

Folks, let's set a tentative deadline with lazy consensus till April 14 at least to enable people to surface thoughts, concerns, objections and to think about a good name for the grouped mode driver.

@fmuyassarov good point, thanks for raising. Let me gather my thoughts and I'll answer with my proposal for guidelines.

[catblade]

The main question we have is can we use hugepages from the grouped driver and still use this one to do individual cores? Will that be an issue? If not, no issue with the split.

[pravk03]

It would be a problem if two drivers published ResourceSlice based on the same underlying resource. The easy fix is to have a configuration option to manage only CPU/mem or both in the cpumem driver.

[catblade]

Can we make sure that is captured somewhere?

It would be a problem if two drivers published ResourceSlice based on the same underlying resource. The easy fix is to have a configuration option to manage only CPU/mem or both in the cpumem driver.

[ffromani]

The question I have is do we need to define a criteria for accepting/rejecting future resource representations? Without that, I'm afraid the new driver risks accumulating the same ambiguity over time. That decision what to include and what not should probably be written down somewhere.

I spent quite some time thinking about this and I believe... I don't have a clear cut binary decision framework just yet.
But I can offer some thoughts and guidelines.

From my POV the main deciding factor is the resource model the driver exposes, which in turn riverberates in the DeviceClass and ResourceSlice representation. I think a driver should expose just one of these (and this is the reason why this whole conversation started). That single model can be enhanced, refined, extended, but the changes should have a clear path to the same root. In a way, I see this reminiscent of the Liskov principle: if we plan somehow an extension of a resource model but still the base one is functional in the same context, albeit degraded, then that extension fit in the same driver; otherwise, I believe it is best served using a different driver.

Another parallel is the kubelet cpumanager/topologymanager options. These can alter even very significantly the behavior of the base component, but still the fundamental core remains the same, and the basic use cases are covered: a cpumanager with all the extra options added over time is still functional and can still serve the usecases the options address and improve, but in a worse way.

Therefore, in the future, the basic guiding principle to accept/reject new resource models or extensions thereof can be derived by this question: "is the new proposed way still fundamentally compatible? Can the Liskov principle apply?"

In our specific case the individual and grouped mode are very far, so we have a basis to split.
Or, and this can be probably another good gating question, from the other angle: if we had two completely separate drivers to begin with, one individual-mode only and one grouped-mode only, would we have a case to merge them?
I for one I'd have a quite hard time to find good reasons to merge them.

[ffromani]

The main question we have is can we use hugepages from the grouped driver and still use this one to do individual cores? Will that be an issue? If not, no issue with the split.

Thanks for raising this point. This is very important because it is about composability between drivers.

Straight answer first: yes, we must enable this usecase.

Having a rich ecosystem of drivers, some of them possibly providing different representation for the same hardware, was so far universally preached in the community (caveat emptor: AFAICT) as upside of DRA. The classic example is to have community-provided and vendor-provided hardware drivers, but now we are thinking about two community-driven drivers for the same hardware!
All this variety is pointless if drivers can't be composed as users want, so we need to make all the efforts to make the community-provided drivers composable, to set the example and the precedent.

The other emerging point is about combining drivers.
There's no technical requirement to merge the memory support in the grouped-only CPU driver we are discussing. Point in case: the memory support merge proposal vastly predates this current split proposal.

There's a trend in the community towards combining drivers, e.g. for core resources. The grouped mode leans naturally in this area because memory is inherently anonymous and grouped - you can't ask for this specific hugepage or that specific memory range. You may ask for memory in a specific NUMA node though, which is the core principle of the grouped mode.
The other main reasons for combined drivers are saving connections to the API server, especially at scale, and the existing best practices in the resource allocators: NRI drivers and the lessons learned from the kubelet herd of managers. The consensus trends in the direction that combining allocations (e.g. NOT treat them as separated resources, but as interrelated resources) leads to better result.

There also a challenge we currently have which we sidestep using a combined driver: lack of consensus and standard attribute about the NUMA node locality. Arguably, NUMA node locality is the main if not the only attribute we can use to bridge cpu/device allocation with memory allocation. While this is an open issue, it's also an issue for any non-combined driver. We can workaround this problem in the short/mid term kinda like we did with dranet, and we will need a solution of some form in the long term anyway.

Finally, I think we can and should rearrange the codebase of the community-driven combined driver such as is possible to integrate pulling packages in a common frontend. This is a purely engineering problem/constraint, so it's doable at implementation stage. This will add more work, but it will make easy for any interested part to pull the parts and produce their own combined drivers. And it will set a golden standard for DRA drivers. Just an idea though, the main point is that we must enable integration.