You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
We provide a OPEA microservice benchmarking tool which is designed for microservice performance testing and benchmarking. It allows you to define test cases for various services based on YAML configurations, run load tests using `stresscli`, built on top of [locust](https://github.com/locustio/locust), and analyze the results for performance insights.
133
+
134
+
### Features
135
+
136
+
-**Services load testing**: Simulates high concurrency levels to test services like LLM, reranking, ASR, E2E and more.
137
+
-**YAML-based configuration**: Easily define test cases, service endpoints, and parameters.
138
+
-**Service metrics collection**: Optionally collect service metrics to analyze performance bottlenecks.
139
+
-**Flexible testing**: Supports a variety of tests like chatqna, codegen, codetrans, faqgen, audioqna, and visualqna.
140
+
-**Data analysis and visualization**: Visualize test results to uncover performance trends and bottlenecks.
141
+
142
+
### How to use
143
+
144
+
**Define Test Cases**: Configure your tests in the [benchmark.yaml](./evals/benchmark/benchmark.py) file.
This ensures the system can handle high concurrency by allowing more open files and connections.
153
+
154
+
**Run the benchmark script**:
155
+
156
+
```bash
157
+
python evals/benchmark/benchmark.py
158
+
```
159
+
160
+
Results will be saved in the directory specified by `test_output_dir` in the configuration.
161
+
162
+
163
+
For more details on configuring test cases, refer to the [README](./evals/benchmark/README.md).
164
+
165
+
166
+
### Grafana Dashboards
167
+
Prometheus metrics collected during the tests can be used to create Grafana dashboards for visualizing performance trends and monitoring bottlenecks. For more information, refer to the [Grafana README](./evals/benchmark/grafana/README.md)
Copy file name to clipboardExpand all lines: evals/benchmark/README.md
+1-1Lines changed: 1 addition & 1 deletion
Original file line number
Diff line number
Diff line change
@@ -1,6 +1,6 @@
1
1
# OPEA Benchmark Tool
2
2
3
-
This Tool provides a microservices benchmarking framework that uses YAML configurations to define test cases for different services. It executes these tests using `stresscli`, built on top of [locust](https://github.com/locustio/locust), a performance/load testing tool for HTTP and other protocols and logs the results for performance analysis and data visualization.
3
+
This Tool provides a microservice benchmarking framework that uses YAML configurations to define test cases for different services. It executes these tests using `stresscli`, built on top of [locust](https://github.com/locustio/locust), a performance/load testing tool for HTTP and other protocols and logs the results for performance analysis and data visualization.
0 commit comments