Re-introduce pytest-xdist in supported envs

[webknjaz]

What do these changes do?

This change resurrects parallel test execution in most of the test envs. On my laptop, the full test run is almost 3x faster: 36.04s vs 105.53s (0:01:45).

Refs:

• pytest-xdist breaks asyncio-based code expecting to be run in the main thread pytest-dev/pytest-xdist#620
• https://stackoverflow.com/a/58614689/595220
• https://bugs.python.org/issue35621
• bpo-35621: Support running subprocesses in asyncio when loop is executed in non-main thread  python/cpython#14344

Are there changes in behavior for the user?

Nope.

Related issue number

#3419 + #3450

Checklist

• I think the code is well written
• Unit tests for the changes exist
• Documentation reflects the changes
• If you provide code modification, please add yourself to CONTRIBUTORS.txt
  • The format is <Name> <Surname>.
  • Please keep alphabetical order, the file is sorted by names.
• Add a new news fragment into the CHANGES folder
  • name it <issue_id>.<type> for example (588.bugfix)
  • if you don't have an issue_id change it to the pr id after creating the pr
  • ensure type is one of the following:
    • .feature: Signifying a new feature.
    • .bugfix: Signifying a bug fix.
    • .doc: Signifying a documentation improvement.
    • .removal: Signifying a deprecation or removal of public API.
    • .misc: A ticket has been closed, but it is not of interest to users.
  • Make sure to use full sentences with correct case and punctuation, for example: "Fix issue with non-ascii contents in doctest text files."

[codecov]

Codecov Report

All modified and coverable lines are covered by tests ✅

Project coverage is 98.67%. Comparing base (acc2912) to head (1e79149).
Report is 689 commits behind head on master.

✅ All tests successful. No failed tests found.

Additional details and impacted files

@@           Coverage Diff           @@
##           master    #5431   +/-   ##
=======================================
  Coverage   98.67%   98.67%           
=======================================
  Files         117      117           
  Lines       35824    35891   +67     
  Branches     4252     4258    +6     
=======================================
+ Hits        35348    35415   +67     
  Misses        320      320           
  Partials      156      156           

Flag	Coverage Δ
CI-GHA	98.56% <100.00%> (+<0.01%)	⬆️
OS-Linux	98.24% <100.00%> (+<0.01%)	⬆️
OS-Windows	96.01% <100.00%> (+<0.01%)	⬆️
OS-macOS	97.31% <100.00%> (+<0.01%)	⬆️
Py-3.10.11	97.16% <98.83%> (+<0.01%)	⬆️
Py-3.10.15	97.77% <98.83%> (+0.05%)	⬆️
Py-3.11.10	97.77% <97.67%> (+<0.01%)	⬆️
Py-3.11.9	97.21% <97.67%> (-0.02%)	⬇️
Py-3.12.7	98.31% <98.83%> (+0.05%)	⬆️
Py-3.13.0	98.25% <98.83%> (+<0.01%)	⬆️
Py-3.9.13	97.08% <98.83%> (+<0.01%)	⬆️
Py-3.9.20	97.64% <98.83%> (+<0.01%)	⬆️
Py-pypy7.3.16	97.25% <98.83%> (+<0.01%)	⬆️
VM-macos	97.31% <100.00%> (+<0.01%)	⬆️
VM-ubuntu	98.24% <100.00%> (+<0.01%)	⬆️
VM-windows	96.01% <100.00%> (+<0.01%)	⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

[webknjaz]

@Dreamsorcerer @bdraco any ideas why the http parser tests might be failing in with pytest-xdist?

[Dreamsorcerer]

In Home Assistant we have a call_soon at the end of each test to give aiohttp one more event loop iteration to shutdown https://github.com/home-assistant/core/blob/4eb096cb0a593bc35f9cd9a2b3c797d2700a4aff/tests/common.py#L369 for similar reasons

I was under the impression this should not be needed on master (v4) anymore, so shouldn't be a factor in these tests. Unless it is something different from: https://docs.aiohttp.org/en/stable/client_advanced.html#graceful-shutdown

[webknjaz]

Perhaps, somebody will be able to pair with me on the sprint day. Or just take care of rebasing+researching the solution.

[codspeed-hq]

CodSpeed Performance Report

Merging #5431 will not alter performance

_{Comparing testing/pytest-xdist (1e79149) with master (acc2912)}

Summary

✅ 14 untouched benchmarks

[patchback]

Backport to 3.10: 💔 cherry-picking failed — conflicts found

❌ Failed to cleanly apply 751c3c4 on top of patchback/backports/3.10/751c3c4e17d621ffc91b79dccb4e2a5ef8b42888/pr-5431

Backporting merged PR #5431 into master

1. Ensure you have a local repo clone of your fork. Unless you cloned it
   from the upstream, this would be your origin remote.
2. Make sure you have an upstream repo added as a remote too. In these
   instructions you'll refer to it by the name upstream. If you don't
   have it, here's how you can add it:

   $ git remote add upstream https://github.com/aio-libs/aiohttp.git

3. Ensure you have the latest copy of upstream and prepare a branch
   that will hold the backported code:

   $ git fetch upstream
   $ git checkout -b patchback/backports/3.10/751c3c4e17d621ffc91b79dccb4e2a5ef8b42888/pr-5431 upstream/3.10

4. Now, cherry-pick PR Re-introduce pytest-xdist in supported envs #5431 contents into that branch:

   $ git cherry-pick -x 751c3c4e17d621ffc91b79dccb4e2a5ef8b42888

   If it'll yell at you with something like fatal: Commit 751c3c4e17d621ffc91b79dccb4e2a5ef8b42888 is a merge but no -m option was given., add -m 1 as follows instead:

   $ git cherry-pick -m1 -x 751c3c4e17d621ffc91b79dccb4e2a5ef8b42888

5. At this point, you'll probably encounter some merge conflicts. You must
   resolve them in to preserve the patch from PR Re-introduce pytest-xdist in supported envs #5431 as close to the
   original as possible.
6. Push this branch to your fork on GitHub:

   $ git push origin patchback/backports/3.10/751c3c4e17d621ffc91b79dccb4e2a5ef8b42888/pr-5431

7. Create a PR, ensure that the CI is green. If it's not — update it so that
   the tests and any other checks pass. This is it!
   Now relax and wait for the maintainers to process your pull request
   when they have some cycles to do reviews. Don't worry — they'll tell you if
   any improvements are necessary when the time comes!

🤖 @patchback
I'm built with octomachinery and
my source is open — https://github.com/sanitizers/patchback-github-app.

[patchback]

Backport to 3.11: 💔 cherry-picking failed — conflicts found

❌ Failed to cleanly apply 751c3c4 on top of patchback/backports/3.11/751c3c4e17d621ffc91b79dccb4e2a5ef8b42888/pr-5431

Backporting merged PR #5431 into master

1. Ensure you have a local repo clone of your fork. Unless you cloned it
   from the upstream, this would be your origin remote.
2. Make sure you have an upstream repo added as a remote too. In these
   instructions you'll refer to it by the name upstream. If you don't
   have it, here's how you can add it:

   $ git remote add upstream https://github.com/aio-libs/aiohttp.git

3. Ensure you have the latest copy of upstream and prepare a branch
   that will hold the backported code:

   $ git fetch upstream
   $ git checkout -b patchback/backports/3.11/751c3c4e17d621ffc91b79dccb4e2a5ef8b42888/pr-5431 upstream/3.11

4. Now, cherry-pick PR Re-introduce pytest-xdist in supported envs #5431 contents into that branch:

   $ git cherry-pick -x 751c3c4e17d621ffc91b79dccb4e2a5ef8b42888

   If it'll yell at you with something like fatal: Commit 751c3c4e17d621ffc91b79dccb4e2a5ef8b42888 is a merge but no -m option was given., add -m 1 as follows instead:

   $ git cherry-pick -m1 -x 751c3c4e17d621ffc91b79dccb4e2a5ef8b42888

5. At this point, you'll probably encounter some merge conflicts. You must
   resolve them in to preserve the patch from PR Re-introduce pytest-xdist in supported envs #5431 as close to the
   original as possible.
6. Push this branch to your fork on GitHub:

   $ git push origin patchback/backports/3.11/751c3c4e17d621ffc91b79dccb4e2a5ef8b42888/pr-5431

7. Create a PR, ensure that the CI is green. If it's not — update it so that
   the tests and any other checks pass. This is it!
   Now relax and wait for the maintainers to process your pull request
   when they have some cycles to do reviews. Don't worry — they'll tell you if
   any improvements are necessary when the time comes!

🤖 @patchback
I'm built with octomachinery and
my source is open — https://github.com/sanitizers/patchback-github-app.