Skip to content

🏃 Verification of leak goroutine on controller and manager #738

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Merged
merged 1 commit into from
Feb 6, 2020
Merged

🏃 Verification of leak goroutine on controller and manager #738

merged 1 commit into from
Feb 6, 2020

Conversation

knabben
Copy link
Member

@knabben knabben commented Dec 20, 2019

This PR adds tests both for manager and controller checking the leak of goroutines on start and stop, I put the threshold of 5 in comparison because it has 3 leaks after stop:

goroutine 29 [chan receive]:
k8s.io/apimachinery/pkg/watch.(*Broadcaster).loop(0xc00021fa80)

goroutine 166 [chan receive]:
k8s.io/client-go/tools/record.(*eventBroadcasterImpl).StartEventWatcher.func1(0x291e580, 0xc000212ab0, 0xc000212a80)

goroutine 167 [chan receive]:
k8s.io/client-go/tools/record.(*eventBroadcasterImpl).StartEventWatcher.func1(0x291e580, 0xc000212ae0, 0xc0003fb350)

Related with #724

@k8s-ci-robot k8s-ci-robot added the cncf-cla: yes Indicates the PR's author has signed the CNCF CLA. label Dec 20, 2019
@k8s-ci-robot
Copy link
Contributor

Hi @knabben. Thanks for your PR.

I'm waiting for a kubernetes-sigs member to verify that this patch is reasonable to test. If it is, they should reply with /ok-to-test on its own line. Until that is done, I will not automatically test new commits in this PR, but the usual testing commands by org members will still work. Regular contributors should join the org to skip this step.

Once the patch is verified, the new status will be reflected by the ok-to-test label.

I understand the commands that are listed here.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

@k8s-ci-robot k8s-ci-robot added the needs-ok-to-test Indicates a PR that requires an org member to verify it is safe to test. label Dec 20, 2019
@k8s-ci-robot k8s-ci-robot added the size/M Denotes a PR that changes 30-99 lines, ignoring generated files. label Dec 20, 2019
@alenkacz
Copy link
Contributor

alenkacz commented Jan 1, 2020

Hmm are the leaked goroutines expected? Or should the test rather not have a treshold of 5 but rather 0 and we should fix those leaks before landing this?

@DirectXMan12 DirectXMan12 changed the title Verification of leak goroutine on controller and manager 🏃 Verification of leak goroutine on controller and manager Jan 6, 2020
@DirectXMan12
Copy link
Contributor

Those are semi-expected. We don't have a good way to work around them yet. I think we should finally have a shutdown method, but that doesn't alleviate the problem entirely. Ideas welcome.

For now, the threshold should be whatever exists currently (so the three), with a comment to ratchet down as we fix the issues.

alenkacz
alenkacz reviewed Jan 16, 2020
View reviewed changes
pkg/controller/controller_test.go Outdated
@@ -90,6 +92,26 @@ var _ = Describe("controller.Controller", func() {

close(done)
})

It("should not leak goroutines when stop", func(done Done) {
// TODO(community) - After closing the proper leaks on watch this must be reduced to 0
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

should we maybe also document why are those leaks expected and what goroutines are leaking for future reference?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

yeah. We can at least link to the relevant issue somewhere around here.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

(also, for future reference, community is not actually a valid TODO target -- this isn't supposed to be who does the TODO, it's supposed to be who has context)

@vincepri
Copy link
Member

/ok-to-test

@k8s-ci-robot k8s-ci-robot added ok-to-test Indicates a non-member PR verified by an org member that is safe to test. and removed needs-ok-to-test Indicates a PR that requires an org member to verify it is safe to test. labels Jan 22, 2020
@DirectXMan12 DirectXMan12 self-assigned this Feb 5, 2020
@DirectXMan12
Copy link
Contributor

gonna push the note, otherwise we should be good

@DirectXMan12
Copy link
Contributor

/lgtm
/approve

@k8s-ci-robot k8s-ci-robot added the lgtm "Looks good to me", indicates that a PR is ready to be merged. label Feb 5, 2020
@DirectXMan12
Copy link
Contributor

/ok-to-test

@k8s-ci-robot
Copy link
Contributor

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: DirectXMan12, knabben

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files:

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

1 similar comment
@k8s-ci-robot
Copy link
Contributor

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: DirectXMan12, knabben

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files:

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

@k8s-ci-robot k8s-ci-robot added the approved Indicates a PR has been approved by an approver from all required OWNERS files. label Feb 5, 2020
@knabben
Copy link
Member Author

knabben commented Feb 6, 2020

/retest

@k8s-ci-robot k8s-ci-robot merged commit 82a78f9 into kubernetes-sigs:master Feb 6, 2020
@knabben knabben deleted the leak-stop branch February 6, 2020 01:34
@vincepri vincepri mentioned this pull request Feb 21, 2020
Closed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@DirectXMan12 DirectXMan12 DirectXMan12 left review comments

@alenkacz alenkacz alenkacz left review comments

@joelanford joelanford Awaiting requested review from joelanford

@pwittrock pwittrock Awaiting requested review from pwittrock

Assignees

@DirectXMan12 DirectXMan12

Labels
approved Indicates a PR has been approved by an approver from all required OWNERS files. cncf-cla: yes Indicates the PR's author has signed the CNCF CLA. lgtm "Looks good to me", indicates that a PR is ready to be merged. ok-to-test Indicates a non-member PR verified by an org member that is safe to test. size/M Denotes a PR that changes 30-99 lines, ignoring generated files.
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
5 participants
@knabben @k8s-ci-robot @alenkacz @DirectXMan12 @vincepri