Skip to content

[Metrics] W1: gRPC engine-metrics scraping (/engine_metrics across transports) #1774

Description

@slin1237

Parent: #1773 · Phase 1

Make GET /engine_metrics include gRPC workers so engine metrics consolidate across transports. Today the fetch is HTTP-only (worker/manager.rs:978-1012, fan_out :53-85); a gRPC worker yields grpc://host:port/metrics, fails, and is silently dropped (an all-gRPC fleet returns "All backend requests failed.").

Implementation

  • Discover an HTTP metrics endpoint for gRPC workers during metadata discovery (workflow/steps/local/discover_metadata.rscreate_worker.rs), from GetServerInfo.server_args (a full Struct, proto/sglang_scheduler.proto:461):
  • Store as a label, not a WorkerSpec field — WorkerSpec is in crates/protocols (sacred); only model_gateway uses this, so the iron law keeps it out of protocols. Add the keys to SGLANG_GRPC_KEYS (routers/grpc/client.rs:783); expose Worker::metrics_url() beside base_url() (worker/worker.rs:664).
  • Branch the fetch on connection_mode() in get_engine_metrics/fan_out (worker/manager.rs:53-85,978-1012), mirroring monitor.rs:627-632. HTTP keeps GET {url}/metrics; gRPC uses metrics_url(). Unknown-endpoint workers are skipped and counted.
  • Aggregation unchanged (worker/metrics_aggregator.rs).

New metric

  • smg_engine_metrics_scrape_total{connection_mode,result} (success/failure/skipped). Replace the blanket "All backend requests failed" when the real cause is "no gRPC endpoint known".

Acceptance criteria

  • An all-gRPC fleet returns merged engine metrics from /engine_metrics.
  • Mixed HTTP+gRPC fleets merge both, labeled per worker.
  • A gRPC worker with metrics disabled is skipped + counted, not an error.

Tests

  • Unit: server_argsmetrics_url extraction (explicit / derived / disabled); fan_out URL per connection mode.
  • Integration: gRPC stub advertising metrics_url + stub /metrics → merged into /engine_metrics.

Note: W1 is always-on (a bug fix, no flag). Works for upstream engines that already expose /metrics without W4.

Metadata

Metadata

Assignees

Labels

enhancementNew feature or requestgrpcgRPC client and router changesmetrics-consolidationEpic: Engine Metrics Consolidation & PD Observability

Type

No type

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions