Skip to content

Disable test reporting for flaky tests on Helix #10352

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 1 commit into from
May 21, 2019

Conversation

JunTaoLuo
Copy link
Contributor

@JunTaoLuo JunTaoLuo commented May 17, 2019

While I look into https://github.com/aspnet/AspNetCore-Internal/issues/2482. We might want to disable test reporting of flaky Helix tests so the PR checks are more meaningful.

WIP don't review yet.

@Eilon Eilon added the area-infrastructure Includes: MSBuild projects/targets, build scripts, CI, Installers and shared framework label May 17, 2019
@JunTaoLuo JunTaoLuo marked this pull request as ready for review May 21, 2019 01:01
@JunTaoLuo JunTaoLuo requested a review from a team as a code owner May 21, 2019 01:01
@JunTaoLuo
Copy link
Contributor Author

aspnet/AspNetCore-Internal#2482 will likely take some feature work to resolve so I think we probably want to merge this and make the helix test check marks more meaningful. @natemcmaster @dougbu . Note that even this test run isn't green but that's because there are flaky tests that haven't been marked as such.

@natemcmaster
Copy link
Contributor

Note that even this test run isn't green but that's because there are flaky tests that haven't been marked as such.

Let's apply [Flaky].

Copy link
Contributor

@natemcmaster natemcmaster left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fine a workaround for now. Let's make sure to work with the dnceng team to see if we can make Helix reporting understand the concept of flaky tests.

@JunTaoLuo
Copy link
Contributor Author

Yup @ @alexperovich is looking into devops features and workarounds.

@JunTaoLuo
Copy link
Contributor Author

Seems like I mis-spoke the helix test failures are not flakiness, they are consistently failing. Will file followup issues.

@JunTaoLuo JunTaoLuo merged commit e925df6 into master May 21, 2019
@ghost ghost deleted the johluo/disable-helix-flaky-test-reporting branch May 21, 2019 22:03
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
area-infrastructure Includes: MSBuild projects/targets, build scripts, CI, Installers and shared framework
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants