Skip to content

Commit 559ebb2

Browse files
Refine the README for llms/doc-summarization (opea-project#1437)
Signed-off-by: unknown <[email protected]>
1 parent c58fd39 commit 559ebb2

File tree

1 file changed

+33
-9
lines changed

1 file changed

+33
-9
lines changed

comps/llms/src/doc-summarization/README.md

Lines changed: 33 additions & 9 deletions
Original file line numberDiff line numberDiff line change
@@ -1,6 +1,10 @@
11
# Document Summary LLM Microservice
22

3-
This microservice leverages LangChain to implement summarization strategies and facilitate LLM inference using Text Generation Inference on Intel Xeon and Gaudi2 processors. You can set backend service either [TGI](../../../third_parties/tgi) or [vLLM](../../../third_parties/vllm).
3+
This microservice leverages LangChain to implement advanced text summarization strategies and facilitate Large Language Model (LLM) inference using Text Generation Inference (TGI) on Intel Xeon and Gaudi2 processors. Users can configure the backend service to utilize either [TGI](../../../third_parties/tgi) or [vLLM](../../../third_parties/vllm).
4+
5+
# Quick Start Guide
6+
7+
## Deployment options
48

59
## 🚀1. Start Microservice with Docker 🐳
610

@@ -25,18 +29,18 @@ Please make sure MAX_TOTAL_TOKENS should be larger than (MAX_INPUT_TOKENS + max_
2529

2630
Step 1: Prepare backend LLM docker image.
2731

28-
If you want to use vLLM backend, refer to [vLLM](../../../third_parties/vllm/) to build vLLM docker images first.
32+
If you want to use vLLM backend, refer to [vLLM](../../../third_parties/vllm/) for building the necessary Docker image.
2933

30-
No need for TGI.
34+
TGI does not require additional setup.
3135

32-
Step 2: Build DocSum docker image.
36+
Step 2: Build DocSum docker image:
3337

3438
```bash
3539
cd ../../../../
3640
docker build -t opea/llm-docsum:latest --build-arg https_proxy=$https_proxy --build-arg http_proxy=$http_proxy -f comps/llms/src/doc-summarization/Dockerfile .
3741
```
3842

39-
### 1.3 Run Docker
43+
### 1.3 Run Docker Service
4044

4145
To start a docker container, you have two options:
4246

@@ -75,17 +79,37 @@ Set `service_name` to match backend service.
7579

7680
```bash
7781
export service_name="docsum-tgi"
78-
# export service_name="docsum-tgi-gaudi"
79-
# export service_name="docsum-vllm"
80-
# export service_name="docsum-vllm-gaudi"
82+
# Alternative you can use service_name as: "docsum-tgi-gaudi", "docsum-vllm", "docsum-vllm-gaudi"
8183

8284
cd ../../deployment/docker_compose/
8385
docker compose -f compose_doc-summarization.yaml up ${service_name} -d
8486
```
8587

88+
## 🚀2. Start Microservice with Kubernetes
89+
90+
The **DocSum microservice** can be deployed on a **Kubernetes cluster** using the provided manifests.
91+
92+
### 2.1 Deployment Overview
93+
94+
- Requires **a running Kubernetes cluster** and `kubectl` configured.
95+
- The service can be exposed using **ClusterIP, NodePort, or Ingress**.
96+
- Backend LLM service (**TGI or vLLM**) must be running.
97+
98+
### 2.2 Quick Deployment Steps
99+
100+
Run the following commands to deploy:
101+
102+
```bash
103+
kubectl apply -f deployment/k8s/docsum-deployment.yaml
104+
kubectl apply -f deployment/k8s/docsum-service.yaml
105+
kubectl apply -f deployment/k8s/docsum-ingress.yaml # If using Ingress
106+
```
107+
108+
For detailed deployment steps and configuration options, refer to the [Kubernetes Deployment Guide](../../../llms/deployment).
109+
86110
## 🚀3. Consume LLM Service
87111

88-
### 3.1 Check Service Status
112+
### 3.1 Checking Service Status
89113

90114
```bash
91115
curl http://${your_ip}:9000/v1/health_check\

0 commit comments

Comments
 (0)