MVP support for serving workloads running as LeaderWorkerSet #3232
Description
What would you like to be added:
MVP support for LeaderWorkerSet in Kueue. It does not need to be ideal, but we want to have some support to unblock users and collect users' feedback.
The idea is to base the support on StatefulSets, so the integration would also use Pod Groups, similarly as for regular StatefulSets. Each LeaderWorkerGroup creates a new Pod Group. I a single pod group we will have:
- Leader pod, controller by Leader’s STS
- Worker pods, controller by unique, dedicated STS
The size of the group will be taken from LeaderWorkerSet.Spec.LeaderWorkerTemplate.Size and increased by 1 (to include the leader).
This is a follow up to #2717.
Why is this needed:
We want to support serving primitives in Kueue as there is an increasing demand among users to run clusters mixing AI training and inference who want to manage the expensive GPU resources.
LeaderWorkerSet is a new serving API which is gaining popularity as a primitive to host AI/ML inference.