Skip to content

Commit 7bede46

Browse files
nirrozenbaumRyanRosario
authored andcommitted
updated kgateway section in bbr quickstart guide (kubernetes-sigs#2179)
Signed-off-by: Nir Rozenbaum <nirro@il.ibm.com>
1 parent 45644bc commit 7bede46

1 file changed

Lines changed: 27 additions & 0 deletions

File tree

site-src/guides/serving-multiple-inference-pools-latest.md

Lines changed: 27 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -132,6 +132,20 @@ Set the Helm chart version (unless already set).
132132
oci://us-central1-docker.pkg.dev/k8s-staging-images/gateway-api-inference-extension/charts/inferencepool
133133
```
134134

135+
=== "Kgateway"
136+
137+
```bash
138+
export GATEWAY_PROVIDER=none
139+
helm install vllm-deepseek-r1 \
140+
--dependency-update \
141+
--set inferencePool.modelServers.matchLabels.app=vllm-deepseek-r1 \
142+
--set provider.name=$GATEWAY_PROVIDER \
143+
--set experimentalHttpRoute.enabled=true \
144+
--set experimentalHttpRoute.baseModel=deepseek/vllm-deepseek-r1 \
145+
--version $IGW_CHART_VERSION \
146+
oci://us-central1-docker.pkg.dev/k8s-staging-images/gateway-api-inference-extension/charts/inferencepool
147+
```
148+
135149
=== "Other"
136150

137151
```bash
@@ -190,6 +204,19 @@ Run `helm upgrade` in order to update in place the HttpRoute mapping of the firs
190204
--version $IGW_CHART_VERSION
191205
```
192206

207+
=== "Kgateway"
208+
209+
```bash
210+
export GATEWAY_PROVIDER=none
211+
helm upgrade vllm-llama3-8b-instruct oci://us-central1-docker.pkg.dev/k8s-staging-images/gateway-api-inference-extension/charts/inferencepool \
212+
--dependency-update \
213+
--set inferencePool.modelServers.matchLabels.app=vllm-llama3-8b-instruct \
214+
--set provider.name=$GATEWAY_PROVIDER \
215+
--set experimentalHttpRoute.enabled=true \
216+
--set experimentalHttpRoute.baseModel=meta-llama/Llama-3.1-8B-Instruct \
217+
--version $IGW_CHART_VERSION
218+
```
219+
193220
=== "Other"
194221

195222
```bash

0 commit comments

Comments
 (0)