-
-
Notifications
You must be signed in to change notification settings - Fork 2.8k
Consider nonparametrized tests in reordering #11236
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Consider nonparametrized tests in reordering #11236
Conversation
sadra-barikbin
commented
Jul 21, 2023
- To extract fixture dependencies of nonparametrized tests as well, in order to finally consider all fixture dependencies, whether a fixture is parametrized or not, the test is parametrized or not, into reordering.
62c8ca5
to
01b5db5
Compare
…ed-tests-in-reordering' into Feature-consider-nonparametrized-tests-in-reordering
Regarding
When I fixed these tests, the job passed. |
Thanks for the PR @sadra-barikbin. I'm interested to know what is the motivation for this change. I think (no first hand knowledge, maybe @RonnyPfannschmidt knows) that the restriction of reordering to only parametrized tests is intentional. Consider for example (using def test_1(tmp_path_factory): pass
def test_2(): pass
def test_3(tmp_path_factory): pass Without this PR, the ordering is 1, 2, 3. With this PR, the ordering is 1, 3, 2. I think that most users would prefer the existing ordering, and would be confused by the new ordering, which would even put 3 before 2 when it's in an entirely different file. The new ordering goes too far if you catch my meaning. IMO the guiding principles should be:
So the way to achieve 2 without hurting 1 too much is to restrict the reordering to parametrized args. |
This PR could be seen as a prelude to the early-teardown feature to finally achieve these:
If the above gains happen to be significant, to resolve the issue you mentioned rising from very |
the key reason reordering got introduced is to reduce setup/teardown for high scope with parameters for within the scope from my pov non-parameterized tests ought not to be considered for reordering |
Hi @bluetech @RonnyPfannschmidt , long time no see!🌷 I agree with your arguments. Feel free to close the PR. |
OK thanks @sadra-barikbin for the PR and everyone for the comments. 🙇 Closing this then. 👍 |