Skip to content

Upgrade tgi and tei version #931

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 2 commits into from
Apr 3, 2025
Merged

Upgrade tgi and tei version #931

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
2 commits
Select commit Hold shift + click to select a range
40c81d0
Upgrade tgi version to be consistent with docker compose
yongfengdu Apr 2, 2025
e5e0d7d
Upgrade text-embeddings-inference to 1.6
yongfengdu Apr 2, 2025
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 2 additions & 2 deletions helm-charts/chatqna/nv-values.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
# Copyright (C) 2024 Intel Corporation
# Copyright (C) 2025 Intel Corporation
# SPDX-License-Identifier: Apache-2.0

# To override values in subchart tgi
Expand All @@ -9,7 +9,7 @@ tgi:
accelDevice: "nvidia"
image:
repository: ghcr.io/huggingface/text-generation-inference
tag: "2.2.0"
tag: "2.4.1"
resources:
limits:
nvidia.com/gpu: 1
Expand Down
4 changes: 2 additions & 2 deletions helm-charts/common/tei/Chart.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
# Copyright (C) 2024 Intel Corporation
# Copyright (C) 2025 Intel Corporation
# SPDX-License-Identifier: Apache-2.0

apiVersion: v2
Expand All @@ -11,4 +11,4 @@ dependencies:
repository: "file://../commonlib"
version: 0-latest
# The HF TEI version
appVersion: "cpu-1.5"
appVersion: "cpu-1.6"
2 changes: 1 addition & 1 deletion helm-charts/common/tei/README.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -40,6 +40,6 @@ curl http://localhost:2081/embed -X POST -d '{"inputs":"What is Deep Learning?"}
| EMBEDDING_MODEL_ID | string | `"BAAI/bge-base-en-v1.5"` | Models id from https://huggingface.co/, or predownloaded model directory |
| global.modelUseHostPath | string | `"/mnt/opea-models"` | Cached models directory, tei will not download if the model is cached here. The host path "modelUseHostPath" will be mounted to container as /data directory. Set this to null/empty will force it to download model. |
| image.repository | string | `"ghcr.io/huggingface/text-embeddings-inference"` | |
| image.tag | string | `"cpu-1.5"` | |
| image.tag | string | `"cpu-1.6"` | |
| autoscaling.enabled | bool | `false` | Enable HPA autoscaling for the service deployment based on metrics it provides. See [HPA instructions](../../HPA.md) before enabling! |
| global.monitoring | bool | `false` | Enable usage metrics for the service. Required for HPA. See [monitoring instructions](../../monitoring.md) before enabling! |
4 changes: 2 additions & 2 deletions helm-charts/common/tei/values.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
# Copyright (C) 2024 Intel Corporation
# Copyright (C) 2025 Intel Corporation
# SPDX-License-Identifier: Apache-2.0

# Default values for tei.
Expand Down Expand Up @@ -27,7 +27,7 @@ image:
# Uncomment the following line to set desired image pull policy if needed, as one of Always, IfNotPresent, Never.
# pullPolicy: ""
# Overrides the image tag whose default is the chart appVersion.
tag: "cpu-1.5"
tag: "cpu-1.6"

# empty for CPU
accelDevice: ""
Expand Down
4 changes: 2 additions & 2 deletions helm-charts/common/teirerank/Chart.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
# Copyright (C) 2024 Intel Corporation
# Copyright (C) 2025 Intel Corporation
# SPDX-License-Identifier: Apache-2.0

apiVersion: v2
Expand All @@ -11,4 +11,4 @@ dependencies:
repository: "file://../commonlib"
version: 0-latest
# The HF TEI version
appVersion: "cpu-1.5"
appVersion: "cpu-1.6"
2 changes: 1 addition & 1 deletion helm-charts/common/teirerank/README.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -43,6 +43,6 @@ curl http://localhost:2082/rerank \
| RERANK_MODEL_ID | string | `"BAAI/bge-reranker-base"` | Models id from https://huggingface.co/, or predownloaded model directory |
| global.modelUseHostPath | string | `"/mnt/opea-models"` | Cached models directory, teirerank will not download if the model is cached here. The host path "modelUseHostPath" will be mounted to container as /data directory. Set this to null/empty will force it to download model. |
| image.repository | string | `"ghcr.io/huggingface/text-embeddings-inference"` | |
| image.tag | string | `"cpu-1.5"` | |
| image.tag | string | `"cpu-1.6"` | |
| autoscaling.enabled | bool | `false` | Enable HPA autoscaling for the service deployment based on metrics it provides. See [HPA instructions](../../HPA.md) before enabling! |
| global.monitoring | bool | `false` | Enable usage metrics for the service. Required for HPA. See [monitoring instructions](../../monitoring.md) before enabling! |
4 changes: 2 additions & 2 deletions helm-charts/common/teirerank/values.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
# Copyright (C) 2024 Intel Corporation
# Copyright (C) 2025 Intel Corporation
# SPDX-License-Identifier: Apache-2.0

# Default values for teirerank.
Expand Down Expand Up @@ -27,7 +27,7 @@ image:
# Uncomment the following line to set desired image pull policy if needed, as one of Always, IfNotPresent, Never.
# pullPolicy: ""
# Overrides the image tag whose default is the chart appVersion.
tag: "cpu-1.5"
tag: "cpu-1.6"

# empty for CPU
accelDevice: ""
Expand Down
4 changes: 2 additions & 2 deletions helm-charts/common/tgi/nv-values.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
# Copyright (C) 2024 Intel Corporation
# Copyright (C) 2025 Intel Corporation
# SPDX-License-Identifier: Apache-2.0

# Default values for tgi.
Expand All @@ -9,7 +9,7 @@ accelDevice: "nvidia"

image:
repository: ghcr.io/huggingface/text-generation-inference
tag: "2.2.0"
tag: "2.4.1"

resources:
limits:
Expand Down