Skip to content

feat: support ai-gateway v0.2.1 #478

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Draft
wants to merge 1 commit into
base: main
Choose a base branch
from
Draft
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
6 changes: 3 additions & 3 deletions chart/Chart.lock
Original file line number Diff line number Diff line change
Expand Up @@ -13,6 +13,6 @@ dependencies:
version: v1.4.0
- name: ai-gateway-helm
repository: oci://registry-1.docker.io/envoyproxy
version: v0.1.5
digest: sha256:d4ca67d95b86b66af01991f1a2eab307db36ebd9a627f06581c0292e10cad259
generated: "2025-06-17T11:08:57.411227+08:00"
version: v0.2.1
digest: sha256:4570f7225628789c1485224dfedac251373ce77c1b28abaae9fc8721043fff0a
generated: "2025-07-05T09:46:42.466286422+08:00"
6 changes: 5 additions & 1 deletion chart/Chart.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -38,6 +38,10 @@ dependencies:
repository: "oci://registry-1.docker.io/envoyproxy"
condition: envoy-gateway.enabled
- name: ai-gateway-helm
version: v0.1.5
version: v0.2.1
repository: "oci://registry-1.docker.io/envoyproxy"
condition: envoy-ai-gateway.enabled
- name: ai-gateway-crds-helm
version: v0.2.1
repository: "oci://registry-1.docker.io/envoyproxy"
condition: envoy-ai-gateway.enabled
21 changes: 16 additions & 5 deletions docs/examples/envoy-ai-gateway/basic-vllm.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -62,18 +62,29 @@ spec:
value: qwen3-0--6b
backendRefs:
- name: qwen3-0--6b
modelsOwnedBy: "llmaz"
timeouts:
request: 3m
---
apiVersion: aigateway.envoyproxy.io/v1alpha1
kind: AIServiceBackend
metadata:
name: qwen3-0--6b
spec:
timeouts:
request: 3m
schema:
name: OpenAI
backendRef:
name: qwen3-0--6b-lb
kind: Service
port: 8080
name: qwen3-0--6b
kind: Backend
group: gateway.envoyproxy.io
---
apiVersion: gateway.envoyproxy.io/v1alpha1
kind: Backend
metadata:
name: qwen3-0--6b
namespace: default
spec:
endpoints:
- fqdn:
hostname: qwen3-0--6b-lb.default.svc.cluster.local
port: 8080
44 changes: 34 additions & 10 deletions docs/examples/envoy-ai-gateway/basic.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -85,38 +85,62 @@ spec:
value: qwen2-0--5b
backendRefs:
- name: qwen2-0--5b
modelsOwnedBy: "llmaz"
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is a newly added field

timeouts:
request: 3m
- matches:
- headers:
- type: Exact
name: x-ai-eg-model
value: qwen2--5-coder
backendRefs:
- name: qwen2--5-coder
modelsOwnedBy: "llmaz"
timeouts:
request: 3m
---
apiVersion: aigateway.envoyproxy.io/v1alpha1
kind: AIServiceBackend
metadata:
name: qwen2-0--5b
spec:
timeouts:
request: 3m
schema:
name: OpenAI
backendRef:
name: qwen2-0--5b-lb
kind: Service
port: 8080
name: qwen2-0--5b
kind: Backend
group: gateway.envoyproxy.io
---
apiVersion: aigateway.envoyproxy.io/v1alpha1
kind: AIServiceBackend
metadata:
name: qwen2--5-coder
spec:
timeouts:
request: 3m
schema:
name: OpenAI
backendRef:
name: qwen2--5-coder-lb
kind: Service
port: 8080
name: qwen2--5-coder
kind: Backend
group: gateway.envoyproxy.io
---
apiVersion: gateway.envoyproxy.io/v1alpha1
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Support after v0.2.0

kind: Backend
metadata:
name: qwen2-0--5b
namespace: default
spec:
endpoints:
- fqdn:
hostname: qwen2-0--5b-lb.default.svc.cluster.local
port: 8080
---
apiVersion: gateway.envoyproxy.io/v1alpha1
kind: Backend
metadata:
name: qwen2--5-coder
namespace: default
spec:
endpoints:
- fqdn:
hostname: qwen2--5-coder-lb.default.svc.cluster.local
port: 8080
Loading