Skip to content

Reduce flakiness on chunk flush tests #4386

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 1 commit into from
Aug 17, 2021
Merged

Conversation

bboreham
Copy link
Contributor

While running ingester tests locally I would get occasional failures in these tests.

The thinking was: we are flushing every 20ms, so wait 40ms to be sure that one cycle has happened.
However if you log the times at which things happen, it is fairly common to see 100ms delay on the flush ticker.
So wait a lot longer to be sure.

Also close down the ingester that we created for the test, and improve the error message on failure.

Checklist

  • Tests updated
  • NA Documentation added
  • NA CHANGELOG.md updated

The thinking was: we are flushing every 20ms, so wait 40ms to
be sure that one cycle has happened. However if you log the times at
which things happen, it is fairly common to see 100ms delay on the flush
ticker. So wait a lot longer to be sure.

Also close down the ingester that we created for the test, and improve
the error message on failure.

Signed-off-by: Bryan Boreham <[email protected]>
@bboreham bboreham merged commit 35919e4 into master Aug 17, 2021
@bboreham bboreham deleted the spread-flush-test-fix branch August 17, 2021 10:24
alvinlin123 pushed a commit to ac1214/cortex that referenced this pull request Jan 14, 2022
The thinking was: we are flushing every 20ms, so wait 40ms to
be sure that one cycle has happened. However if you log the times at
which things happen, it is fairly common to see 100ms delay on the flush
ticker. So wait a lot longer to be sure.

Also close down the ingester that we created for the test, and improve
the error message on failure.

Signed-off-by: Bryan Boreham <[email protected]>
Signed-off-by: Alvin Lin <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants