Clear and concise description of the problem
Sometimes tests are flaky and people use the retry option to avoid failing CI when that happens.
But you still want to know which tests are flaky to try and improve them.
Suggested solution
Collect flaky tests (reruns > 0) in the reporter and print a warning summary at the end of the run.
The reason for suggesting the warning summary at the end is that this is obviously less important than failures and shouldn't be intermittent with other errors or warnings in the output.
Ideally link to the test by line and if possible also link back to the execution in the log (no idea if the latter is possible)
Alternative
A custom reporter could be built to provide this
Additional context
discord discussion https://discord.com/channels/917386801235247114/1462789015895019724
Validations
Clear and concise description of the problem
Sometimes tests are flaky and people use the
retryoption to avoid failing CI when that happens.But you still want to know which tests are flaky to try and improve them.
Suggested solution
Collect flaky tests (reruns > 0) in the reporter and print a warning summary at the end of the run.
The reason for suggesting the warning summary at the end is that this is obviously less important than failures and shouldn't be intermittent with other errors or warnings in the output.
Ideally link to the test by line and if possible also link back to the execution in the log (no idea if the latter is possible)
Alternative
A custom reporter could be built to provide this
Additional context
discord discussion https://discord.com/channels/917386801235247114/1462789015895019724
Validations