Skip to content

[Do Not Merge] POC for Mixed CPU Allocation#149

Draft
pravk03 wants to merge 4 commits into
kubernetes-sigs:mainfrom
pravk03:mixed-allocation-poc
Draft

[Do Not Merge] POC for Mixed CPU Allocation#149
pravk03 wants to merge 4 commits into
kubernetes-sigs:mainfrom
pravk03:mixed-allocation-poc

Conversation

@pravk03
Copy link
Copy Markdown
Contributor

@pravk03 pravk03 commented May 20, 2026

POC for #103

This PR introduces Mixed Allocation Mode to the CPU DRA driver, enabling workloads to request both exclusive and shared CPUs.

Key Principles:

  • Any request that goes in the standard container spec is considered as a request for shared CPUs.
  • Any claim request is considered as a request for guaranteed exclusive CPUs.
  • Relies on KEP-5517 to allow for correct scheduler accounting.

Flags:

  • Added --mixed-allocation-mode flag to enable or disable mixed allocation mode.
  • Added --disable-node-allocatable-mapping flag to toggle native KEP-5517 scheduler accounting.

DRA Hook:

  • Adds a native NodeAllocatable resource mapping to published ResourceSlices. This allows kube-scheduler to track and subtract DRA based CPU requests directly from node allocatable capacity.

NRI Hook:

  • Mixed Allocation Mode Implementation:
    • Configured the NRI plugin to inspect container CPU shares at runtime.
    • Containers with shares > 2 are automatically identified as running in mixed mode. Kubelet sets container shares to exactly 2 when standard CPU requests are omitted, as defined in helpers_linux.go.
    • Configures the CPU affinity mask of mixed containers to the union of their dedicated exclusive CPUs and the node's shared CPU pool.
    • Shrinks the shared CPU pool of other standard containers by subtracting exclusive CPUs.

@k8s-ci-robot
Copy link
Copy Markdown
Contributor

Skipping CI for Draft Pull Request.
If you want CI signal for your change, please convert it to an actual PR.
You can still manually trigger a test run with /test all

@k8s-ci-robot k8s-ci-robot added the do-not-merge/work-in-progress Indicates that a PR should not merge because it is a work in progress. label May 20, 2026
@k8s-ci-robot
Copy link
Copy Markdown
Contributor

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: pravk03

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Details Needs approval from an approver in each of these files:

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

@k8s-ci-robot k8s-ci-robot added approved Indicates a PR has been approved by an approver from all required OWNERS files. size/XXL Denotes a PR that changes 1000+ lines, ignoring generated files. cncf-cla: yes Indicates the PR's author has signed the CNCF CLA. labels May 20, 2026
@pravk03 pravk03 force-pushed the mixed-allocation-poc branch 8 times, most recently from 4cdc990 to 944aa73 Compare May 20, 2026 05:09
pravk03 added 4 commits May 20, 2026 05:11
Remove flags that are on by default
This enables kube-scheduler and the Kubernetes control plane to track
DRA claims directly against node allocatable CPU resources, this ensures
1. coordinated scheduling between DRA and non-DRA workload on the same node.
2. We no longer have to duplicate dra resources in the pod's standard
   request
@pravk03 pravk03 force-pushed the mixed-allocation-poc branch from 944aa73 to 50df449 Compare May 20, 2026 05:37
@ffromani
Copy link
Copy Markdown
Contributor

/cc

I'm aware this it is a POC and about all the caveats, still very interested in this work! thanks for posting

@k8s-ci-robot k8s-ci-robot requested a review from ffromani May 20, 2026 07:11
@swatisehgal
Copy link
Copy Markdown
Contributor

/cc

@k8s-ci-robot k8s-ci-robot requested a review from swatisehgal May 20, 2026 13:16
@Tal-or
Copy link
Copy Markdown

Tal-or commented May 20, 2026

/cc

@k8s-ci-robot k8s-ci-robot requested a review from Tal-or May 20, 2026 14:14
@AutuSnow
Copy link
Copy Markdown
Contributor

/cc

@k8s-ci-robot k8s-ci-robot requested a review from AutuSnow May 20, 2026 16:09
@k8s-ci-robot k8s-ci-robot added the needs-rebase Indicates a PR cannot be merged because it has merge conflicts with HEAD. label May 21, 2026
@k8s-ci-robot
Copy link
Copy Markdown
Contributor

PR needs rebase.

Details

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository.

@pravk03
Copy link
Copy Markdown
Contributor Author

pravk03 commented May 26, 2026

I will continue exploring this, but the progress is likely going to be slow as the v1.37 deadlines approach. Additionally, this work depends on KEP-5517 reaching Alpha 2 (hopefully in 1.37) so that Kubelet can set safe pod-level cgroup boundaries and container level limits to include DRA.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

approved Indicates a PR has been approved by an approver from all required OWNERS files. cncf-cla: yes Indicates the PR's author has signed the CNCF CLA. do-not-merge/work-in-progress Indicates that a PR should not merge because it is a work in progress. needs-rebase Indicates a PR cannot be merged because it has merge conflicts with HEAD. size/XXL Denotes a PR that changes 1000+ lines, ignoring generated files.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

6 participants