[Umbrella] inference engine metrics installation #375
Description
What would you like to be added:
After #316, we have prometheus metrics support for controller, however, we need to expose inference engines' metrics as well for further development, like traffic routing.
- vllm
- sglang
- llama.cpp
- TGI
Why is this needed:
Provide observability and support for further development like traffic routing.
Completion requirements:
This enhancement requires the following artifacts:
- Design doc
- API change
- Docs update
The artifacts should be linked in subsequent comments.