Skip to content

Update kvcache v1alpha1 api spec #1055

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
Jeffwan merged 1 commit into from
May 9, 2025
Merged

Update kvcache v1alpha1 api spec #1055

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
99 changes: 34 additions & 65 deletions api/orchestration/v1alpha1/kvcache_types.go
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -21,116 +21,85 @@ import (
metav1 "k8s.io/apimachinery/pkg/apis/meta/v1"
)

// ServiceConfig holds all service configuration about KvCache public facing service
type ServiceConfig struct {
// ServiceSpec holds all service configuration about KvCache public facing service
type ServiceSpec struct {
// Type defines the type of service (e.g., ClusterIP, NodePort, LoadBalancer).
// +kubebuilder:validation:Optional
// +kubebuilder:default:="ClusterIP"
Type corev1.ServiceType `json:"type,omitempty"`

// service port
// +kubebuilder:validation:Optional
// +kubebuilder:default:=9600
Port int32 `json:"port,omitempty"`

// NodePort specifies the port on each node on which this service is exposed when using NodePort type.
// +kubebuilder:validation:Optional
NodePort *int32 `json:"nodePort,omitempty"`
// Ports defines the list of exposed ports
// +kubebuilder:validation:MinItems=1
Ports []corev1.ServicePort `json:"ports"`
}

// MetadataConfig holds the configuration about the kv cache metadata service
type MetadataConfig struct {
Redis *RedisConfig `json:"redis,omitempty"`
Etcd *EtcdConfig `json:"etcd,omitempty"`
}
// ExternalConnectionConfig holds config for connecting to external metadata service
type ExternalConnectionConfig struct {
// Address to connect to (host:port)
Address string `json:"address,omitempty"`

// RedisConfig provides the configuration fields for deploying Redis.
type RedisConfig struct {
Image string `json:"image"`
Replicas int32 `json:"replicas"`
Resources corev1.ResourceRequirements `json:"resources,omitempty"`
Storage *MetadataStorage `json:"storage,omitempty"`
// Optional secret reference for password or credential
PasswordSecretRef string `json:"passwordSecretRef,omitempty"`
}

// EtcdConfig provides the configuration fields for deploying etcd.
type EtcdConfig struct {
Image string `json:"image"`
// +kubebuilder:validation:Optional
// +kubebuilder:default:=1
Replicas int32 `json:"replicas"`
Resources corev1.ResourceRequirements `json:"resources,omitempty"`
Storage *MetadataStorage `json:"storage,omitempty"`
// MetadataConfig provides the configuration fields for deploying Redis.
type MetadataConfig struct {
ExternalConnection *ExternalConnectionConfig `json:"externalConnection,omitempty"`
Runtime *RuntimeSpec `json:"runtime,omitempty"`
}

// MetadataStorage configures the persistent storage used by the metadata service.
type MetadataStorage struct {
Size string `json:"size"`
// MetadataSpec holds deployment or external connection config for metadata services
type MetadataSpec struct {
Redis *MetadataConfig `json:"redis,omitempty"`
Etcd *MetadataConfig `json:"etcd,omitempty"`
}

type CacheSpec struct {
type RuntimeSpec struct {
// Replicas is the number of kvcache pods to deploy
// +kubebuilder:validation:Optional
// +kubebuilder:default:=3
Replicas int `json:"replicas,omitempty"`
// +kubebuilder:default:=1
Replicas int32 `json:"replicas,omitempty"`

// represent the kvcache's image
// +kubebuilder:validation:Optional
// +kubebuilder:default:="aibrix/kvcache:20241120"
// +kubebuilder:validation:Required
Image string `json:"image,omitempty"`

// the policy about pulling image
// +kubebuilder:validation:Optional
// +kubebuilder:default:="IfNotPresent"
ImagePullPolicy string `json:"imagePullPolicy,omitempty"`

// shared memory size for kvcache
// +kubebuilder:validation:Optional
// +kubebuilder:default:=""
SharedMemorySize string `json:"sharedMemorySize,omitempty"`

// kvcache environment configuration
// +kubebuilder:validation:Optional
// +kubebuilder:default:={}
Env []corev1.EnvVar `json:"env,omitempty"`

// the memory resources of kvcache container
// the resources of kvcache container
// +kubebuilder:validation:Optional
// +kubebuilder:default:="2Gi"
Memory string `json:"memory,omitempty"`

// the cpu resources of kvcache container
// +kubebuilder:validation:Optional
// +kubebuilder:default:="1"
CPU string `json:"cpu,omitempty"`
Resources corev1.ResourceRequirements `json:"resources,omitempty"`
}

// KVCacheSpec defines the desired state of KVCache
type KVCacheSpec struct {
// Replicas is the number of kv cache pods to deploy
// +kubebuilder:validation:Required
// +kubebuilder:default:=1
Replicas int32 `json:"replicas,omitempty"`

// EtcdReplicas describe the etcd replicas
// +kubebuilder:validation:Optional
// +kubebuilder:default:=1
EtcdReplicas int32 `json:"etcdReplicas,omitempty"`
// +kubebuilder:default:=distributed
Mode string `json:"mode,omitempty"` // centralized | distributed

// Metadata configuration for kv cache service
// +kubebuilder:validation:Optional
// +kubebuilder:default:={etcd: {image: "", replicas: 1, storage: {size: "10Gi"}}}
Metadata *MetadataConfig `json:"metadata,omitempty"`
Metadata *MetadataSpec `json:"metadata,omitempty"`

// kvcache dataplane container configuration
// +kubebuilder:validation:Optional
//nolint: lll
// +kubebuilder:default:={image: "aibrix/kvcache:20241120", imagePullPolicy: "IfNotPresent"}
Cache CacheSpec `json:"cacheSpec,omitempty"`
Cache RuntimeSpec `json:"cache,omitempty"`
Copy link
Contributor

@vie-serendipity vie-serendipity May 9, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

RuntimeSpec's fields are a subset of the podSpec+replicas. I want to know the standard for exposing which fields from the podSpec. I think using podSpec directly would be more appropriate.

Copy link
Collaborator Author

@Jeffwan Jeffwan May 9, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

yeah, using podSpec is more flexible. We saw a few users meet problem using the right arguments so this api wrap lots of logic and only expose most common changes.. We plan to rollout v0.3.0 pretty soon, I think we can get some feedback from users and gradually improve this part. Feel free to leave more feedbacks. really appreicate it

Copy link
Contributor

@vie-serendipity vie-serendipity May 9, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Great, thanks for the response, looking forward to the 0.3.0 release.


// kvcache watcher pod for member registration
// +kubebuilder:validation:Optional
Watcher *RuntimeSpec `json:"watcher,omitempty"`

// cache's service
// +kubebuilder:validation:Optional
// +kubebuilder:default:={type: "ClusterIP", port: 9600}
Service ServiceConfig `json:"service,omitempty"`
Service ServiceSpec `json:"service,omitempty"`
}

// KVCacheStatus defines the observed state of KVCache
Expand Down
113 changes: 52 additions & 61 deletions api/orchestration/v1alpha1/zz_generated.deepcopy.go
View file
Open in desktop

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

5 changes: 5 additions & 0 deletions cmd/kvcache-watcher/main.go
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -379,6 +379,11 @@ func syncPods(
}
}

if len(validPods) == 0 {
klog.Warningf("No valid KVCache pods found after filtering for cluster %v", kvClusterId)
return nil
}

nodeSlots := calculateSlotDistribution(validPods, consistentHashingTotalSlots, consistentHashingVirtualNodeCount)
currentNodes := make([]NodeInfo, 0)
for _, pod := range validPods {
Expand Down
Loading
Loading