Skip to content

Deployer: Evaluate Inference Extension Support #10878

@danehans

Description

@danehans

When #10684 merges, the Deployer will support managing k8s objects, e.g. Deployment, to run the endpoint picker inference extension. The following questions should be answered to improve supporting future extensions:

  1. Should Kgateway continue to support managing the infra for inference extensions? The answer to this question may depend on the results of this upstream issue.
  2. If Kgateway continues to support managing k8s objects for an inference extension, should an extension point be added to the Deployer?

Metadata

Metadata

Assignees

Labels

Area: InferenceActivities related to Gateway API Inference Extension support.

Type

No type

Projects

Status

Done

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions