From 30c49704b12cc8c94aa4153c497d1174ac739326 Mon Sep 17 00:00:00 2001 From: Louie Tsai Date: Thu, 16 Jan 2025 17:40:06 -0800 Subject: [PATCH 1/3] Update README.md for OPEA OTLP tracing Signed-off-by: louie-tsai Signed-off-by: Tsai, Louie --- ChatQnA/README.md | 8 ++++++++ 1 file changed, 8 insertions(+) diff --git a/ChatQnA/README.md b/ChatQnA/README.md index c605e883cb..458773b7a9 100644 --- a/ChatQnA/README.md +++ b/ChatQnA/README.md @@ -362,3 +362,11 @@ For TGI serving on Gaudi, users could see different services like opea, TEI and Here is a screenshot for one tracing of TGI serving request. ![Screenshot from 2024-12-27 11-26-25](https://github.com/user-attachments/assets/3a7c51c6-f422-41eb-8e82-c3df52cd48b8) + +There are also OPEA related tracings. Users could understand the time breakdown of each service request by looking into each opea:schedule operation. +![image](https://github.com/user-attachments/assets/6137068b-b374-4ff8-b345-993343c0c25f) + +There might be asyn function such as llm/MicroService_asyn_generate and users need to check the tracing of the asyn function in another operation like +opea:llm_generate_stream. +![image](https://github.com/user-attachments/assets/a973d283-198f-4ce2-a7eb-58515b77503e) + From 9bab6dbb926b159973d4ce356ad3cd4dd0491d77 Mon Sep 17 00:00:00 2001 From: louie-tsai Date: Thu, 16 Jan 2025 17:52:16 -0800 Subject: [PATCH 2/3] fix pre-commit issue Signed-off-by: louie-tsai Signed-off-by: Tsai, Louie --- ChatQnA/README.md | 1 - 1 file changed, 1 deletion(-) diff --git a/ChatQnA/README.md b/ChatQnA/README.md index 458773b7a9..394797cecd 100644 --- a/ChatQnA/README.md +++ b/ChatQnA/README.md @@ -369,4 +369,3 @@ There are also OPEA related tracings. Users could understand the time breakdown There might be asyn function such as llm/MicroService_asyn_generate and users need to check the tracing of the asyn function in another operation like opea:llm_generate_stream. ![image](https://github.com/user-attachments/assets/a973d283-198f-4ce2-a7eb-58515b77503e) - From bb62f0be91aee5b4b33fe555accf5dc57f4351ec Mon Sep 17 00:00:00 2001 From: Louie Tsai Date: Tue, 21 Jan 2025 08:49:37 -0800 Subject: [PATCH 3/3] Update ChatQnA/README.md Co-authored-by: Eero Tamminen Signed-off-by: louie-tsai Signed-off-by: Tsai, Louie --- ChatQnA/README.md | 4 +++- 1 file changed, 3 insertions(+), 1 deletion(-) diff --git a/ChatQnA/README.md b/ChatQnA/README.md index 394797cecd..ffad72d858 100644 --- a/ChatQnA/README.md +++ b/ChatQnA/README.md @@ -357,6 +357,8 @@ Users could also get the external IP via below command. ip route get 8.8.8.8 | grep -oP 'src \K[^ ]+' ``` +Access the Jaeger dashboard UI at http://{EXTERNAL_IP}:16686 + For TGI serving on Gaudi, users could see different services like opea, TEI and TGI. ![Screenshot from 2024-12-27 11-58-18](https://github.com/user-attachments/assets/6126fa70-e830-4780-bd3f-83cb6eff064e) @@ -366,6 +368,6 @@ Here is a screenshot for one tracing of TGI serving request. There are also OPEA related tracings. Users could understand the time breakdown of each service request by looking into each opea:schedule operation. ![image](https://github.com/user-attachments/assets/6137068b-b374-4ff8-b345-993343c0c25f) -There might be asyn function such as llm/MicroService_asyn_generate and users need to check the tracing of the asyn function in another operation like +There could be async function such as `llm/MicroService_asyn_generate` and user needs to check the trace of the async function in another operation like opea:llm_generate_stream. ![image](https://github.com/user-attachments/assets/a973d283-198f-4ce2-a7eb-58515b77503e)