-
Notifications
You must be signed in to change notification settings - Fork 185
Add default requests and limits for ocs-operator pods #3339
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
base: main
Are you sure you want to change the base?
Conversation
Skipping CI for Draft Pull Request. |
[APPROVALNOTIFIER] This PR is NOT APPROVED This pull-request has been approved by: mrudraia1 The full list of commands accepted by this bot can be found here.
Needs approval from an approver in each of these files:
Approvers can indicate their approval by writing |
b49ab89
to
1a61568
Compare
Signed-off-by: Mrudraia1 <[email protected]>
}, | ||
Limits: corev1.ResourceList{ | ||
"memory": resource.MustParse("1.5Gi"), | ||
"cpu": resource.MustParse("1m"), |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
1m?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
It should be "1"
"cpu": resource.MustParse("250m"), | ||
}, | ||
Limits: corev1.ResourceList{ | ||
"memory": resource.MustParse("1.5Gi"), |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Why does the exporter go till 1.5Gi, seems a lot.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
We have seen memory consumption reaching till 1.22Gi
for performance, scale tests ran during testing, therefore setting a safe limit of 1.5Gi
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Isn't the exporter stateless? In your tests what made the mem increase and maybe we have mem leaks?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I think it is stateless 🤔
The tests comprised of IO to fill up the cluster, PVC creation, deletion, PVC expansion, small file IOs and such.
Though, this was the only set of tests where we observed such a hike. In the other cases, the maximum consumption was ~ 700Mi
.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Need a bit more investigation, I understand even w/o this PR the exporter has no bounds but we better take this moment to re-look at the exporter and come to a conclusion for bumping the values.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Considering we don't have any ideas as to why we are seeing a spike, I would suggest to take this PR in and work on parallel to identify any memory leaks or such issues that could lead to a memory spike in exporter.
This PR adds the default requests and limits for the ocs-operator pods.
The default metrics are set for the following pods