Skip to content

Commit f8f8854

Browse files
dbkinderdbkinder
authored
doc: fix heading levels (#690)
Only one H1 for the title is allowed Signed-off-by: David B. Kinder <[email protected]>
1 parent 3c5fc80 commit f8f8854

File tree

2 files changed

+114
-114
lines changed

2 files changed

+114
-114
lines changed

‎comps/dataprep/vdms/README.md

Lines changed: 72 additions & 72 deletions
Original file line numberDiff line numberDiff line change
@@ -6,9 +6,9 @@ For dataprep microservice, we currently provide one framework: `Langchain`.
66

77
We organized the folders in the same way, so you can use either framework for dataprep microservice with the following constructions.
88

9-
# 🚀1. Start Microservice with Python (Option 1)
9+
## 🚀1. Start Microservice with Python (Option 1)
1010

11-
## 1.1 Install Requirements
11+
### 1.1 Install Requirements
1212

1313
Install Single-process version (for 1-10 files processing)
1414

@@ -25,11 +25,11 @@ pip install -r requirements.txt
2525
cd langchain_ray; pip install -r requirements_ray.txt
2626
``` -->
2727

28-
## 1.2 Start VDMS Server
28+
### 1.2 Start VDMS Server
2929

30-
Please refer to this [readme](../../vectorstores/vdms/README.md).
30+
Refer to this [readme](../../vectorstores/vdms/README.md).
3131

32-
## 1.3 Setup Environment Variables
32+
### 1.3 Setup Environment Variables
3333

3434
```bash
3535
export http_proxy=${your_http_proxy}
@@ -40,7 +40,7 @@ export COLLECTION_NAME=${your_collection_name}
4040
export PYTHONPATH=${path_to_comps}
4141
```
4242

43-
## 1.4 Start Document Preparation Microservice for VDMS with Python Script
43+
### 1.4 Start Document Preparation Microservice for VDMS with Python Script
4444

4545
Start document preparation microservice for VDMS with below command.
4646

@@ -56,13 +56,13 @@ python prepare_doc_vdms.py
5656
python prepare_doc_redis_on_ray.py
5757
``` -->
5858

59-
# 🚀2. Start Microservice with Docker (Option 2)
59+
## 🚀2. Start Microservice with Docker (Option 2)
6060

61-
## 2.1 Start VDMS Server
61+
### 2.1 Start VDMS Server
6262

63-
Please refer to this [readme](../../vectorstores/vdms/README.md).
63+
Refer to this [readme](../../vectorstores/vdms/README.md).
6464

65-
## 2.2 Setup Environment Variables
65+
### 2.2 Setup Environment Variables
6666

6767
```bash
6868
export http_proxy=${your_http_proxy}
@@ -76,24 +76,24 @@ export DISTANCE_STRATEGY="L2"
7676
export PYTHONPATH=${path_to_comps}
7777
```
7878

79-
## 2.3 Build Docker Image
79+
### 2.3 Build Docker Image
8080

8181
- Build docker image with langchain
8282

83-
Start single-process version (for 1-10 files processing)
83+
Start single-process version (for 1-10 files processing)
8484

85-
```bash
86-
cd ../../../
87-
docker build -t opea/dataprep-vdms:latest --build-arg https_proxy=$https_proxy --build-arg http_proxy=$http_proxy -f comps/dataprep/vdms/langchain/Dockerfile .
88-
```
85+
```bash
86+
cd ../../../
87+
docker build -t opea/dataprep-vdms:latest --build-arg https_proxy=$https_proxy --build-arg http_proxy=$http_proxy -f comps/dataprep/vdms/langchain/Dockerfile .
88+
```
8989

9090
<!-- - option 2: Start multi-process version (for >10 files processing)
9191
9292
```bash
9393
cd ../../../../
9494
docker build -t opea/dataprep-on-ray-vdms:latest --build-arg https_proxy=$https_proxy --build-arg http_proxy=$http_proxy -f comps/dataprep/vdms/langchain_ray/Dockerfile . -->
9595

96-
## 2.4 Run Docker with CLI
96+
### 2.4 Run Docker with CLI
9797

9898
Start single-process version (for 1-10 files processing)
9999

@@ -113,75 +113,75 @@ docker run -d --name="dataprep-vdms-server" -p 6007:6007 --runtime=runc --ipc=ho
113113
-e TIMEOUT_SECONDS=600 opea/dataprep-on-ray-vdms:latest
114114
``` -->
115115

116-
# 🚀3. Status Microservice
116+
## 🚀3. Status Microservice
117117

118118
```bash
119119
docker container logs -f dataprep-vdms-server
120120
```
121121

122-
# 🚀4. Consume Microservice
122+
## 🚀4. Consume Microservice
123123

124124
Once document preparation microservice for VDMS is started, user can use below command to invoke the microservice to convert the document to embedding and save to the database.
125125

126126
Make sure the file path after `files=@` is correct.
127127

128128
- Single file upload
129129

130-
```bash
131-
curl -X POST \
132-
-H "Content-Type: multipart/form-data" \
133-
-F "files=@./file1.txt" \
134-
http://localhost:6007/v1/dataprep
135-
```
130+
```bash
131+
curl -X POST \
132+
-H "Content-Type: multipart/form-data" \
133+
-F "files=@./file1.txt" \
134+
http://localhost:6007/v1/dataprep
135+
```
136136

137-
You can specify chunk_size and chunk_size by the following commands.
137+
You can specify `chunk_size` and `chunk_overlap` by the following commands.
138138

139-
```bash
140-
curl -X POST \
141-
-H "Content-Type: multipart/form-data" \
142-
-F "files=@./LLAMA2_page6.pdf" \
143-
-F "chunk_size=1500" \
144-
-F "chunk_overlap=100" \
145-
http://localhost:6007/v1/dataprep
146-
```
139+
```bash
140+
curl -X POST \
141+
-H "Content-Type: multipart/form-data" \
142+
-F "files=@./LLAMA2_page6.pdf" \
143+
-F "chunk_size=1500" \
144+
-F "chunk_overlap=100" \
145+
http://localhost:6007/v1/dataprep
146+
```
147147

148148
- Multiple file upload
149149

150-
```bash
151-
curl -X POST \
152-
-H "Content-Type: multipart/form-data" \
153-
-F "files=@./file1.txt" \
154-
-F "files=@./file2.txt" \
155-
-F "files=@./file3.txt" \
156-
http://localhost:6007/v1/dataprep
157-
```
158-
159-
- Links upload (not supported for llama_index now)
160-
161-
```bash
162-
curl -X POST \
163-
-F 'link_list=["https://www.ces.tech/"]' \
164-
http://localhost:6007/v1/dataprep
165-
```
166-
167-
or
168-
169-
```python
170-
import requests
171-
import json
172-
173-
proxies = {"http": ""}
174-
url = "http://localhost:6007/v1/dataprep"
175-
urls = [
176-
"https://towardsdatascience.com/no-gpu-no-party-fine-tune-bert-for-sentiment-analysis-with-vertex-ai-custom-jobs-d8fc410e908b?source=rss----7f60cf5620c9---4"
177-
]
178-
payload = {"link_list": json.dumps(urls)}
179-
180-
try:
181-
resp = requests.post(url=url, data=payload, proxies=proxies)
182-
print(resp.text)
183-
resp.raise_for_status() # Raise an exception for unsuccessful HTTP status codes
184-
print("Request successful!")
185-
except requests.exceptions.RequestException as e:
186-
print("An error occurred:", e)
187-
```
150+
```bash
151+
curl -X POST \
152+
-H "Content-Type: multipart/form-data" \
153+
-F "files=@./file1.txt" \
154+
-F "files=@./file2.txt" \
155+
-F "files=@./file3.txt" \
156+
http://localhost:6007/v1/dataprep
157+
```
158+
159+
- Links upload (not supported for `llama_index` now)
160+
161+
```bash
162+
curl -X POST \
163+
-F 'link_list=["https://www.ces.tech/"]' \
164+
http://localhost:6007/v1/dataprep
165+
```
166+
167+
or
168+
169+
```python
170+
import requests
171+
import json
172+
173+
proxies = {"http": ""}
174+
url = "http://localhost:6007/v1/dataprep"
175+
urls = [
176+
"https://towardsdatascience.com/no-gpu-no-party-fine-tune-bert-for-sentiment-analysis-with-vertex-ai-custom-jobs-d8fc410e908b?source=rss----7f60cf5620c9---4"
177+
]
178+
payload = {"link_list": json.dumps(urls)}
179+
180+
try:
181+
resp = requests.post(url=url, data=payload, proxies=proxies)
182+
print(resp.text)
183+
resp.raise_for_status() # Raise an exception for unsuccessful HTTP status codes
184+
print("Request successful!")
185+
except requests.exceptions.RequestException as e:
186+
print("An error occurred:", e)
187+
```

‎comps/dataprep/vdms/multimodal_langchain/README.md

Lines changed: 42 additions & 42 deletions
Original file line numberDiff line numberDiff line change
@@ -2,25 +2,25 @@
22

33
For dataprep microservice, we currently provide one framework: `Langchain`.
44

5-
# 🚀1. Start Microservice with Python (Option 1)
5+
## 🚀1. Start Microservice with Python (Option 1)
66

7-
## 1.1 Install Requirements
7+
### 1.1 Install Requirements
88

99
- option 1: Install Single-process version (for 1-10 files processing)
1010

11-
```bash
12-
apt-get update
13-
apt-get install -y default-jre tesseract-ocr libtesseract-dev poppler-utils
14-
pip install -r requirements.txt
15-
```
11+
```bash
12+
apt-get update
13+
apt-get install -y default-jre tesseract-ocr libtesseract-dev poppler-utils
14+
pip install -r requirements.txt
15+
```
1616

17-
## 1.2 Start VDMS Server
17+
### 1.2 Start VDMS Server
1818

1919
```bash
2020
docker run -d --name="vdms-vector-db" -p 55555:55555 intellabs/vdms:latest
2121
```
2222

23-
## 1.3 Setup Environment Variables
23+
### 1.3 Setup Environment Variables
2424

2525
```bash
2626
export http_proxy=${your_http_proxy}
@@ -33,23 +33,23 @@ export your_hf_api_token="{your_hf_token}"
3333
export PYTHONPATH=${path_to_comps}
3434
```
3535

36-
## 1.4 Start Data Preparation Microservice for VDMS with Python Script
36+
### 1.4 Start Data Preparation Microservice for VDMS with Python Script
3737

3838
Start document preparation microservice for VDMS with below command.
3939

4040
```bash
4141
python ingest_videos.py
4242
```
4343

44-
# 🚀2. Start Microservice with Docker (Option 2)
44+
## 🚀2. Start Microservice with Docker (Option 2)
4545

46-
## 2.1 Start VDMS Server
46+
### 2.1 Start VDMS Server
4747

4848
```bash
4949
docker run -d --name="vdms-vector-db" -p 55555:55555 intellabs/vdms:latest
5050
```
5151

52-
## 2.1 Setup Environment Variables
52+
### 2.1 Setup Environment Variables
5353

5454
```bash
5555
export http_proxy=${your_http_proxy}
@@ -61,64 +61,64 @@ export INDEX_NAME="rag-vdms"
6161
export your_hf_api_token="{your_hf_token}"
6262
```
6363

64-
## 2.3 Build Docker Image
64+
### 2.3 Build Docker Image
6565

6666
- Build docker image
6767

68-
```bash
69-
cd ../../../
70-
docker build -t opea/dataprep-vdms:latest --network host --build-arg https_proxy=$https_proxy --build-arg http_proxy=$http_proxy -f comps/dataprep/vdms/multimodal_langchain/Dockerfile .
68+
```bash
69+
cd ../../../
70+
docker build -t opea/dataprep-vdms:latest --network host --build-arg https_proxy=$https_proxy --build-arg http_proxy=$http_proxy -f comps/dataprep/vdms/multimodal_langchain/Dockerfile .
7171

72-
```
72+
```
7373

74-
## 2.4 Run Docker Compose
74+
### 2.4 Run Docker Compose
7575

7676
```bash
7777
docker compose -f comps/dataprep/vdms/multimodal_langchain/docker-compose-dataprep-vdms.yaml up -d
7878
```
7979

80-
# 🚀3. Status Microservice
80+
## 🚀3. Status Microservice
8181

8282
```bash
8383
docker container logs -f dataprep-vdms-server
8484
```
8585

86-
# 🚀4. Consume Microservice
86+
## 🚀4. Consume Microservice
8787

8888
Once data preparation microservice for VDMS is started, user can use below command to invoke the microservice to convert the videos to embedding and save to the database.
8989

9090
Make sure the file path after `files=@` is correct.
9191

9292
- Single file upload
9393

94-
```bash
95-
curl -X POST \
96-
-H "Content-Type: multipart/form-data" \
97-
-F "files=@./file1.mp4" \
98-
http://localhost:6007/v1/dataprep
99-
```
94+
```bash
95+
curl -X POST \
96+
-H "Content-Type: multipart/form-data" \
97+
-F "files=@./file1.mp4" \
98+
http://localhost:6007/v1/dataprep
99+
```
100100

101101
- Multiple file upload
102102

103-
```bash
104-
curl -X POST \
105-
-H "Content-Type: multipart/form-data" \
106-
-F "files=@./file1.mp4" \
107-
-F "files=@./file2.mp4" \
108-
-F "files=@./file3.mp4" \
109-
http://localhost:6007/v1/dataprep
110-
```
103+
```bash
104+
curl -X POST \
105+
-H "Content-Type: multipart/form-data" \
106+
-F "files=@./file1.mp4" \
107+
-F "files=@./file2.mp4" \
108+
-F "files=@./file3.mp4" \
109+
http://localhost:6007/v1/dataprep
110+
```
111111

112112
- List of uploaded files
113113

114-
```bash
115-
curl -X GET http://localhost:6007/v1/dataprep/get_videos
116-
```
114+
```bash
115+
curl -X GET http://localhost:6007/v1/dataprep/get_videos
116+
```
117117

118118
- Download uploaded files
119119

120-
Please use the file name from the list
120+
Use the file name from the list
121121

122-
```bash
123-
curl -X GET http://localhost:6007/v1/dataprep/get_file/${filename}
124-
```
122+
```bash
123+
curl -X GET http://localhost:6007/v1/dataprep/get_file/${filename}
124+
```

0 commit comments

Comments
 (0)