feat: metrics support for llmaz #316

googs1025 · 2025-03-15T02:08:14Z

What this PR does / why we need it

change helm chart
change kustomize config
add prometheus operator install docs

Which issue(s) this PR fixes

Fixes part of #307

Special notes for your reviewer

Does this PR introduce a user-facing change?

None

pacoxu · 2025-03-17T02:15:05Z

/kind feature
/lgtm

googs1025 · 2025-03-17T02:17:46Z

/hold
I want to test this for myself to check this if working as expect

kerthcet · 2025-03-17T03:23:59Z

chart/values.yaml

    port: 8443
    protocol: TCP
-    targetPort: https
+    targetPort: 8080


I think we can not change the values.yaml directly, it's auto generated which means it will be overwritten once we run make helm-package.

chart/values.global.yaml

googs1025 · 2025-03-17T11:39:03Z

Makefile

 $(ENVTEST): $(LOCALBIN)
 	test -s $(LOCALBIN)/setup-envtest || GOBIN=$(LOCALBIN) go install sigs.k8s.io/controller-runtime/tools/setup-envtest@latest

+.PHONY: prometheus


I wonder if it would be possible to support make prometheus here

googs1025 · 2025-03-17T12:41:25Z

chart/templates/metrics-monitor.yaml

@@ -1,4 +1,7 @@
-{{- if .Values.prometheus.enable -}}
+{{- if .Values.prometheus.enable }}
+{{- if not (.Capabilities.APIVersions.Has "monitoring.coreos.com/v1/ServiceMonitor") }}


some docs: https://helm.sh/docs/chart_template_guide/builtin_objects/
to check ServiceMonitor resourecs

So if we haven't install the prometheus-operator, it will report error here, right? This is useful.

yes, test in local(without install the prometheus-operator)

root@VM-0-5-ubuntu:/home/ubuntu/llmaz# vi chart/values.global.yaml root@VM-0-5-ubuntu:/home/ubuntu/llmaz# helm upgrade --install llmaz ./chart -f ./chart/values.global.yaml --namespace=llmaz-system Release "llmaz" does not exist. Installing it now. Error: execution error at (llmaz/templates/prometheus/serviceaccount.yaml:3:4): The cluster does not support the required API resource `monitoring.coreos.com/v1/ServiceMonitor`. root@VM-0-5-ubuntu:/home/ubuntu/llmaz#

kerthcet · 2025-03-18T10:38:14Z

Hi @googs1025 I updated some configs, ptal. Documentation is still missing.

googs1025 · 2025-03-18T12:17:18Z

Hi @googs1025 I updated some configs, ptal. Documentation is still missing.

Thanks for your help! I'll test it locally to double check. Also, what kind of docs do we need?

kerthcet · 2025-03-18T14:46:09Z

Like how to install the prometheus operator, step to step to view the metrics.

googs1025 · 2025-03-19T05:10:33Z

chart/templates/deployment.yaml

          periodSeconds: 20
        name: manager
        ports:
+        - containerPort: 8080


use make helm-package
Generate the missing port part 🤔

it's ok then. we just avoid to update the charts manually. This is because we added the metric patch in kustomize.

so do I need to revert ? or just add this is ok?

chart/templates/prometheus/serviceaccount.yaml

googs1025 · 2025-03-19T12:42:49Z

Like how to install the prometheus operator, step to step to view the metrics.

We can directly see the docs which is more direct

https://github.com/googs1025/llmaz/tree/fix/metrics_bind/docs/prometheus-operator

kerthcet · 2025-03-24T11:33:01Z

Sorry for the late reply, I will test tonight.

kerthcet · 2025-03-26T04:11:59Z

I tried this yesterday, but my network is broken, I'll try this later. Once I have a result, I'll comment here.

kerthcet · 2025-04-14T14:59:54Z

please rebase then I'll merge this pr.

kerthcet · 2025-04-14T14:59:58Z

@googs1025

kerthcet · 2025-04-14T16:49:33Z

See the last commit for fixes, generally the label selector not match.

kerthcet · 2025-04-15T09:26:34Z

encounter bug with make helm-install, see arttor/helmify#169

kerthcet · 2025-04-15T09:39:36Z

Install manually, it works great.

Could you rebase then let's merge this PR, the helm chart problem I'll solve it in #343

googs1025 · 2025-04-15T10:55:17Z

use make install make deploy make install-prometheus
this works as expected

root@VM-0-12-ubuntu:/home/ubuntu# curl 'http://localhost:9090/api/v1/query?query=controller_runtime_reconcile_errors_total'
{"status":"success","data":{"resultType":"vector","result":[{"metric":{"__name__":"controller_runtime_reconcile_errors_total","container":"manager","controller":"backendruntime","endpoint":"https","instance":"10.6.2.8:8443","job":"llmaz-controller-manager-metrics-service","namespace":"llmaz-system","pod":"llmaz-controller-manager-7d894849f6-2cx9s","service":"llmaz-controller-manager-metrics-service"},"value":[1744711370.609,"0"]},{"metric":{"__name__":"controller_runtime_reconcile_errors_total","container":"manager","controller":"cert-rotator","endpoint":"https","instance":"10.6.2.8:8443","job":"llmaz-controller-manager-metrics-service","namespace":"llmaz-system","pod":"llmaz-controller-manager-7d894849f6-2cx9s","service":"llmaz-controller-manager-metrics-service"},"value":[1744711370.609,"0"]},{"metric":{"__name__":"controller_runtime_reconcile_errors_total","container":"manager","controller":"openmodel","endpoint":"https","instance":"10.6.2.8:8443","job":"llmaz-controller-manager-metrics-service","namespace":"llmaz-system","pod":"llmaz-controller-manager-7d894849f6-2cx9s","service":"llmaz-controller-manager-metrics-service"},"value":[1744711370.609,"0"]},{"metric":{"__name__":"controller_runtime_reconcile_errors_total","container":"manager","controller":"playground","endpoint":"https","instance":"10.6.2.8:8443","job":"llmaz-controller-manager-metrics-service","namespace":"llmaz-system","pod":"llmaz-controller-manager-7d894849f6-2cx9s","service":"llmaz-controller-manager-metrics-service"},"value":[1744711370.609,"14"]},{"metric":{"__name__":"controller_runtime_reconcile_errors_total","container":"manager","controller":"service","endpoint":"https","instance":"10.6.2.8:8443","job":"llmaz-controller-manager-metrics-service","namespace":"llmaz-system","pod":"llmaz-controller-manager-7d894849f6-2cx9s","service":"llmaz-controller-manager-metrics-service"},"value":[1744711370.609,"0"]}]}}root@VM-0-12-ubuntu:/home/ubuntu#

googs1025 · 2025-04-15T10:55:28Z

/unhold

Signed-off-by: googs1025 <[email protected]>

Signed-off-by: kerthcet <[email protected]>

googs1025 · 2025-04-15T11:12:17Z

/unhold

kerthcet · 2025-04-15T15:38:46Z

/lgtm
/approve
/hold cancel

InftyAI-Agent added needs-triage Indicates an issue or PR lacks a label and requires one. needs-priority Indicates a PR lacks a label and requires one. do-not-merge/needs-kind Indicates a PR lacks a label and requires one. labels Mar 15, 2025

InftyAI-Agent requested a review from kerthcet March 15, 2025 02:08

InftyAI-Agent added feature Categorizes issue or PR as related to a new feature. lgtm Looks good to me, indicates that a PR is ready to be merged. and removed do-not-merge/needs-kind Indicates a PR lacks a label and requires one. labels Mar 17, 2025

InftyAI-Agent assigned pacoxu Mar 17, 2025

InftyAI-Agent added the do-not-merge/hold Indicates that a PR should not merge because someone has issued a /hold command. label Mar 17, 2025

kerthcet reviewed Mar 17, 2025

View reviewed changes

googs1025 force-pushed the fix/metrics_bind branch from ab25576 to 8d45fd8 Compare March 17, 2025 05:16

InftyAI-Agent removed the lgtm Looks good to me, indicates that a PR is ready to be merged. label Mar 17, 2025

googs1025 force-pushed the fix/metrics_bind branch 2 times, most recently from 1ebb10b to 3a01d1c Compare March 17, 2025 08:14

googs1025 commented Mar 17, 2025

View reviewed changes

chart/values.global.yaml Show resolved Hide resolved

googs1025 commented Mar 17, 2025

View reviewed changes

kerthcet force-pushed the fix/metrics_bind branch from b4ed03e to 8cd8d55 Compare March 18, 2025 10:41

googs1025 force-pushed the fix/metrics_bind branch from e4fb804 to cf74d08 Compare March 19, 2025 05:08

googs1025 commented Mar 19, 2025

View reviewed changes

chart/templates/prometheus/serviceaccount.yaml Show resolved Hide resolved

googs1025 force-pushed the fix/metrics_bind branch from 7b6c78e to aa7d5ac Compare March 19, 2025 12:46

googs1025 changed the title ~~wip: fix miss metrics endpoint bind~~ fix miss metrics endpoint bind Mar 19, 2025

googs1025 changed the title ~~fix miss metrics endpoint bind~~ feat: metrics support for llmaz Mar 19, 2025

googs1025 mentioned this pull request Mar 30, 2025

[Misc] feature(rayclusterreplicaset): check rayclusters crd is installed before controller start vllm-project/aibrix#922

Merged

googs1025 force-pushed the fix/metrics_bind branch from a4646a3 to be54420 Compare April 15, 2025 09:58

googs1025 and others added 8 commits April 15, 2025 19:05

fix miss metrics endpoint bind

8902311

Signed-off-by: googs1025 <[email protected]>

fix metrics-monitor.yaml

c594f5c

Update the prometheus config

7eba587

Signed-off-by: kerthcet <[email protected]>

add miss port in chart/templates/deployment.yaml

9b5dc6c

fix ClusterRoleBinding roleRef name, llmaz -> prometheus

b6cd36a

add prometheus operator install docs

5b20676

Fix prometheus could not fetch metrics

f7436d3

Signed-off-by: kerthcet <[email protected]>

Update

1358a66

Signed-off-by: kerthcet <[email protected]>

googs1025 force-pushed the fix/metrics_bind branch from be54420 to 1358a66 Compare April 15, 2025 11:06

InftyAI-Agent assigned kerthcet and unassigned pacoxu Apr 15, 2025

InftyAI-Agent merged commit 617ee75 into InftyAI:main Apr 15, 2025
24 checks passed

kerthcet mentioned this pull request Apr 24, 2025

[Umbrella] inference engine metrics installation #375

Closed

7 tasks

googs1025 mentioned this pull request Sep 1, 2025

[API] stormservice support podgroup vllm-project/aibrix#1506

Merged

Uh oh!

feat: metrics support for llmaz #316

feat: metrics support for llmaz #316

Uh oh!

Conversation

googs1025 commented Mar 15, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

What this PR does / why we need it

Which issue(s) this PR fixes

Special notes for your reviewer

Does this PR introduce a user-facing change?

Uh oh!

pacoxu commented Mar 17, 2025

Uh oh!

googs1025 commented Mar 17, 2025

Uh oh!

kerthcet Mar 17, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

googs1025 Mar 17, 2025

Choose a reason for hiding this comment

Uh oh!

Uh oh!

googs1025 Mar 17, 2025

Choose a reason for hiding this comment

Uh oh!

googs1025 Mar 17, 2025

Choose a reason for hiding this comment

Uh oh!

kerthcet Mar 18, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

googs1025 Mar 19, 2025

Choose a reason for hiding this comment

Uh oh!

kerthcet commented Mar 18, 2025

Uh oh!

googs1025 commented Mar 18, 2025

Uh oh!

kerthcet commented Mar 18, 2025

Uh oh!

googs1025 Mar 19, 2025

Choose a reason for hiding this comment

Uh oh!

kerthcet Mar 19, 2025

Choose a reason for hiding this comment

Uh oh!

googs1025 Mar 19, 2025

Choose a reason for hiding this comment

Uh oh!

Uh oh!

googs1025 commented Mar 19, 2025

Uh oh!

kerthcet commented Mar 24, 2025

Uh oh!

kerthcet commented Mar 26, 2025

Uh oh!

kerthcet commented Apr 14, 2025

Uh oh!

kerthcet commented Apr 14, 2025

Uh oh!

kerthcet commented Apr 14, 2025

Uh oh!

kerthcet commented Apr 15, 2025

Uh oh!

kerthcet commented Apr 15, 2025

Uh oh!

googs1025 commented Apr 15, 2025

Uh oh!

googs1025 commented Apr 15, 2025

Uh oh!

googs1025 commented Apr 15, 2025

Uh oh!

kerthcet commented Apr 15, 2025

Uh oh!

Uh oh!

Reviewers

Assignees

googs1025 commented Mar 15, 2025 •

edited

Loading

kerthcet Mar 17, 2025 •

edited

Loading

kerthcet Mar 18, 2025 •

edited

Loading