Skip to content

feature: (helm) support custom volumes and volumeMounts for epp #1945

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
k8s-ci-robot merged 2 commits into from
Dec 9, 2025
Merged

feature: (helm) support custom volumes and volumeMounts for epp #1945

Show file tree
Hide file tree
Changes from 1 commit
Commits
Show all changes
2 commits
Select commit Hold shift + click to select a range
f782177
(helm) support custom volumes and volumeMounts for epp
delavet Dec 4, 2025
5ecba15
fix for comments
delavet Dec 8, 2025
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
15 changes: 15 additions & 0 deletions config/charts/inferencepool/README.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -247,6 +247,21 @@ The following table list the configurable parameters of the chart.
| `inferenceExtension.tracing.otelExporterEndpoint` | OpenTelemetry collector endpoint. |
| `inferenceExtension.tracing.sampling.sampler` | The trace sampler to use. Currently, only `parentbased_traceidratio` is supported. This sampler respects the parent span’s sampling decision when present, and applies the configured ratio for root spans. |
| `inferenceExtension.tracing.sampling.samplerArg` | Sampler-specific argument. For `parentbased_traceidratio`, this defines the base sampling rate for new traces (root spans), as a float string in the range [0.0, 1.0]. For example, "0.1" enables 10% sampling. |
| `inferenceExtension.volumes` | List of volumes to mount in the EPP deployment as free-form YAML. Optional. |
| `inferenceExtension.volumeMounts` | List of volume mounts for the EPP container as free-form YAML. Optional. |
| `inferenceExtension.sidecar.enabled` | Enables or disables the sidecar container in the EPP deployment. Defaults to `false`. |
| `inferenceExtension.sidecar.name` | Name of the sidecar container. Required when the sidecar is enabled. |
| `inferenceExtension.sidecar.image` | Image for the sidecar container. Required when the sidecar is enabled. |
| `inferenceExtension.sidecar.imagePullPolicy` | Image pull policy for the sidecar container. Possible values: `Always`, `IfNotPresent`, or `Never`. Defaults to `IfNotPresent`. |
| `inferenceExtension.sidecar.command` | Command to run in the sidecar container as a single string. Optional. |
| `inferenceExtension.sidecar.args` | Arguments to pass to the command in the sidecar container as a list of strings. Optional. |
| `inferenceExtension.sidecar.env` | Environment variables to set in the sidecar container as free-form YAML. Optional. |
| `inferenceExtension.sidecar.ports` | List of ports to expose for the sidecar container. Optional. |
| `inferenceExtension.sidecar.livenessProbe` | Liveness probe configuration for the sidecar container. Optional. |
| `inferenceExtension.sidecar.readinessProbe` | Readiness probe configuration for the sidecar container. Optional. |
| `inferenceExtension.sidecar.resources` | Resource limits and requests for the sidecar container. Optional. |
| `inferenceExtension.sidecar.volumeMounts` | List of volume mounts for the sidecar container. Optional. |
| `inferenceExtension.sidecar.volumes` | List of volumes for the sidecar container. Optional. |
| `provider.name` | Name of the Inference Gateway implementation being used. Possible values: [`none`, `gke`, or `istio`]. Defaults to `none`. |
| `provider.gke.autopilot` | Set to `true` if the cluster is a GKE Autopilot cluster. This is only used if `provider.name` is `gke`. Defaults to `false`. |

Expand Down
2 changes: 1 addition & 1 deletion config/charts/inferencepool/templates/epp-config.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -64,7 +64,7 @@ data:
{{- end }}

---
{{- if .Values.inferenceExtension.sidecar.enabled }}
{{- if and .Values.inferenceExtension.sidecar.enabled .Values.inferenceExtension.sidecar.configMap }}
Copy link
Copy Markdown
Contributor

@nirrozenbaum nirrozenbaum Dec 8, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

what is .Values.inferenceExtension.sidecar.configMap option?
I don't see it documented in the README.
additionally - maybe we can standardize the name (instead of configMap.name) and then then only relevant field would be some free form of configMapData?

Copy link
Copy Markdown
Contributor Author

@delavet delavet Dec 8, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

As far as I know, the ability to deploy a sidecar for EPP was initially introduced in #1821. The primary purpose was to deploy an Envoy as a sidecar for EPP to achieve standalone deployment. The main purpose of ConfigMap is to provide startup configuration for Envoy.

The reason I made changes here is to support the generalization of this capability, since some EPP Sidecars may not require the deployment of a corresponding ConfigMap.

However, a series of fields related to the sidecar that were added at the time were not well documented in the README. I have now updated and supplemented them.

additionally - maybe we can standardize the name (instead of configMap.name) and then then only relevant field would be some free form of configMapData?

I think this comment is very reasonable, so I have made adjustments according to your suggestion. PTAL.

apiVersion: v1
kind: ConfigMap
metadata:
Expand Down
6 changes: 6 additions & 0 deletions config/charts/inferencepool/templates/epp-deployment.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -182,8 +182,14 @@ spec:
volumeMounts:
- name: plugins-config-volume
mountPath: "/config"
{{- if .Values.inferenceExtension.volumeMounts }}
{{- tpl (toYaml .Values.inferenceExtension.volumeMounts) $ | nindent 8 }}
{{- end }}
{{- include "gateway-api-inference-extension.latencyPredictor.containers" . | nindent 6 }}
volumes:
{{- if .Values.inferenceExtension.volumes }}
{{- tpl (toYaml .Values.inferenceExtension.volumes) $ | nindent 6 }}
{{- end }}
{{- if .Values.inferenceExtension.sidecar.volumes }}
{{- tpl (toYaml .Values.inferenceExtension.sidecar.volumes) $ | nindent 6 }}
{{- end }}
Expand Down