Parent: #1773 · Phase 3
Add an HTTP /metrics Prometheus sidecar to SMG's gRPC servicers so they're scrape-able in gRPC mode like upstream SGLang — "enable that for other engines" on the producer side. Today no servicer exposes /metrics (each serve_grpc starts only a grpc.aio.server); the SGLang /metrics referenced in motivation comes from upstream SGLang's HTTP sidecar (the on_request_manager_ready hook, grpc_servicer/.../sglang/server.py:37), not SMG's servicer.
Implementation
- Shared sidecar in
grpc_servicer: a small aiohttp/uvicorn /metrics endpoint on a configurable --metrics-port, started alongside each serve_grpc (sglang/, vllm/, tokenspeed/ server.py); bind/serve failure logged, must not abort the gRPC server. (MLX excluded — no PD; add later only if general engine metrics are wanted.)
- Source: reuse each engine's native Prometheus registry where it exists (vLLM; SGLang via its sidecar hook), plus a shared collector re-exposing the scheduler load snapshot already computed for
GetLoads — guarantees coverage for tokenspeed without bespoke work.
- Close the W1 loop: advertise
metrics_url/prometheus_port in the server_args struct each servicer reports (grpc_servicer/.../sglang/servicer.py:535-568) so W1 auto-discovers it.
Acceptance criteria
- Each servicer serves valid Prometheus exposition on
--metrics-port.
GetServerInfo.server_args advertises the metrics address.
- The gRPC server is unaffected if the sidecar fails to bind.
Tests
- pytest:
/metrics returns valid exposition; server_args advertises the address.
Parent: #1773 · Phase 3
Add an HTTP
/metricsPrometheus sidecar to SMG's gRPC servicers so they're scrape-able in gRPC mode like upstream SGLang — "enable that for other engines" on the producer side. Today no servicer exposes/metrics(eachserve_grpcstarts only agrpc.aio.server); the SGLang/metricsreferenced in motivation comes from upstream SGLang's HTTP sidecar (theon_request_manager_readyhook,grpc_servicer/.../sglang/server.py:37), not SMG's servicer.Implementation
grpc_servicer: a small aiohttp/uvicorn/metricsendpoint on a configurable--metrics-port, started alongside eachserve_grpc(sglang/,vllm/,tokenspeed/server.py); bind/serve failure logged, must not abort the gRPC server. (MLX excluded — no PD; add later only if general engine metrics are wanted.)GetLoads— guarantees coverage for tokenspeed without bespoke work.metrics_url/prometheus_portin theserver_argsstruct each servicer reports (grpc_servicer/.../sglang/servicer.py:535-568) so W1 auto-discovers it.Acceptance criteria
--metrics-port.GetServerInfo.server_argsadvertises the metrics address.Tests
/metricsreturns valid exposition;server_argsadvertises the address.