Add support for Android and iOS platforms

[timrid]

What do these changes do?

This PR adds support for Android and iOS platforms.

Since Python 3.13, Android (PEP738) and iOS (PEP730) are officially supported. In addition, Android and iOS have been supported since cibuildwheel v3.1. This makes it very easy to build wheels for Android and iOS.

PyPI now also supports uploading Android and iOS wheels (see pypi/warehouse#17559).

At https://beeware.org/mobile-wheels/, you can find an overview of all binary packages available for Android and iOS. Currently, there aren't that many, but I hope aiohttp will appear there soon :)

Are there changes in behavior for the user?

No.

Is it a substantial burden for the maintainers to support this?

Most of the compiling work is taken care of by cibuildwheel. However, the pytests are currently not run on Android and iOS, so it is theoretically possible that there may be problems during runtime on Android/iOS.

In principle, cibuildwheel also offers the option of running the pytests on Android and iOS using test-command, but this requires the Android emulator/iOS simulator to be started in the CI, which significantly slows down the building of the wheels. In addition, test-command is not currently used, which would require more adjustments to the CI. Therefore, I have decided not to use it for now and have only manually performed a very simple “smoke test” in a briefcase application to check whether the wheel is importable and usable.

Related issue number

Fixes #11750

Checklist

• I think the code is well written
• Unit tests for the changes exist
• Documentation reflects the changes
• If you provide code modification, please add yourself to CONTRIBUTORS.txt
• Add a new news fragment into the CHANGES/ folder

[codspeed-hq]

Merging this PR will not alter performance

✅ 72 untouched benchmarks
⏩ 72 skipped benchmarks¹

---

_{Comparing timrid:android-and-ios-wheels (1cc1d4b) with master (2a4247b)}

Footnotes

1. 72 benchmarks were skipped, so the baseline results were used instead. If they were deleted from the codebase, click here and archive them to remove them from the performance reports. ↩

[Dreamsorcerer]

Therefore, I have decided not to use it for now and have only manually performed a very simple “smoke test” in a briefcase application to check whether the wheel is importable and usable.

Have you tested a simple server app too?

[timrid]

Have you tested a simple server app too?

Not yet, but I can also do a simple test for this. But it will take a few days.

[codecov]

Codecov Report

❌ Patch coverage is 81.69014% with 13 lines in your changes missing coverage. Please review.
✅ Project coverage is 98.79%. Comparing base (5a9777d) to head (1cc1d4b).
⚠️ Report is 1 commits behind head on master.
✅ All tests successful. No failed tests found.

Files with missing lines	Patch %	Lines
tests/test_web_sendfile_functional.py	25.00%	5 Missing and 4 partials ⚠️
tests/test_web_functional.py	60.00%	2 Missing ⚠️
tests/test_client_functional.py	66.66%	1 Missing ⚠️
tests/test_http_parser.py	50.00%	1 Missing ⚠️

Additional details and impacted files

@@             Coverage Diff             @@
##           master   #12220       +/-   ##
===========================================
+ Coverage   47.33%   98.79%   +51.46%     
===========================================
  Files         131      129        -2     
  Lines       46732    46746       +14     
  Branches     2420     2415        -5     
===========================================
+ Hits        22120    46183    +24063     
+ Misses      24006      433    -23573     
+ Partials      606      130      -476     

Flag	Coverage Δ
Autobahn	?
CI-GHA	98.86% <81.69%> (?)
OS-Linux	98.61% <81.69%> (?)
OS-Windows	96.78% <78.87%> (?)
OS-macOS	97.89% <81.69%> (?)
Py-3.10	97.25% <81.69%> (?)
Py-3.11	98.36% <81.69%> (?)
Py-3.12	97.83% <81.69%> (?)
Py-3.13	97.50% <81.69%> (?)
Py-3.14	97.81% <81.69%> (?)
VM-macos	97.89% <81.69%> (?)
VM-ubuntu	98.61% <81.69%> (?)
VM-windows	96.78% <78.87%> (?)
cython-coverage	37.92% <12.67%> (-0.04%)	⬇️

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

[timrid]

I now added the unittests for Android and iOS to the CI, to verify that everything is working. To run the unittests I use cibuildwheel, that automatically starts an Android Emulator / iOS Simulator and runs the whole pytests inside this Emulator/Simulator.

The only issue is that the unit tests have some binary dependencies that aren't yet available for Android and iOS. Hopefully this will improve over time, but for now I had to add environment markers to quite a few dependencies and skip the tests that use these dependencies.

Another problem was that pip-compile doesn’t use the environment markers for transitive dependencies as well. Therefore, I had to include some of the transitive dependencies in the *.in files so that I could explicitly set the environment markers there.

[Dreamsorcerer]

@timrid Did you want to finish off this PR? It looks like there's only some minor comments and now several conflicts to resolve in order to get this merged.

[patchback]

Backport to 3.14: 💔 cherry-picking failed — conflicts found

❌ Failed to cleanly apply bc504da on top of patchback/backports/3.14/bc504dafcecad929eeb20cfcab71d2d66df9e28e/pr-12220

Backporting merged PR #12220 into master

1. Ensure you have a local repo clone of your fork. Unless you cloned it
   from the upstream, this would be your origin remote.
2. Make sure you have an upstream repo added as a remote too. In these
   instructions you'll refer to it by the name upstream. If you don't
   have it, here's how you can add it:

   $ git remote add upstream https://github.com/aio-libs/aiohttp.git

3. Ensure you have the latest copy of upstream and prepare a branch
   that will hold the backported code:

   $ git fetch upstream
   $ git checkout -b patchback/backports/3.14/bc504dafcecad929eeb20cfcab71d2d66df9e28e/pr-12220 upstream/3.14

4. Now, cherry-pick PR Add support for Android and iOS platforms #12220 contents into that branch:

   $ git cherry-pick -x bc504dafcecad929eeb20cfcab71d2d66df9e28e

   If it'll yell at you with something like fatal: Commit bc504dafcecad929eeb20cfcab71d2d66df9e28e is a merge but no -m option was given., add -m 1 as follows instead:

   $ git cherry-pick -m1 -x bc504dafcecad929eeb20cfcab71d2d66df9e28e

5. At this point, you'll probably encounter some merge conflicts. You must
   resolve them in to preserve the patch from PR Add support for Android and iOS platforms #12220 as close to the
   original as possible.
6. Push this branch to your fork on GitHub:

   $ git push origin patchback/backports/3.14/bc504dafcecad929eeb20cfcab71d2d66df9e28e/pr-12220

7. Create a PR, ensure that the CI is green. If it's not — update it so that
   the tests and any other checks pass. This is it!
   Now relax and wait for the maintainers to process your pull request
   when they have some cycles to do reviews. Don't worry — they'll tell you if
   any improvements are necessary when the time comes!

🤖 @patchback
I'm built with octomachinery and
my source is open — https://github.com/sanitizers/patchback-github-app.

[Dreamsorcerer]

@timrid Could you follow the backport instructions above in order to get it included in the 3.14 release?

[Dreamsorcerer]

The test failure output is awful on these CI jobs. Do you think we can have a separate test step like normal, rather than running it as a side-effect of cibuildwheel?

[timrid]

The test failure output is awful on these CI jobs. Do you think we can have a separate test step like normal, rather than running it as a side-effect of cibuildwheel?

Building wheels and running the tests on Android and iOS are pretty complicated tasks, because they have to be cross compiled an the tests run in the iOS simulator/Android emulator. These simulator/emulator has to be started first, then an dummy iOS/Android app has to be created containing the test files and then this app has to be copied and run in in simulator/emulator and then the python stdout has to be captured from the simulator/emulator and shown in the Github output. So all in all a hard task that is luckily integrated in cibuildwheel. I am not aware of an cibuildwheel option to split building and testing in two separate steps to run it in two different Github Action steps.

I am also not sure what exactly you think is awful in the output? Is it the missing colored output? Or the fact, that you have to expand the Running test suite... section to see the test output? Maybe there are options to optimize the output of cibuildwheel?

[Dreamsorcerer]

I am also not sure what exactly you think is awful in the output? Is it the missing colored output? Or the fact, that you have to expand the Running test suite... section to see the test output? Maybe there are options to optimize the output of cibuildwheel?

• It's not immediately obvious what the failure is because it needs to be expanded.
• Missing colour makes it difficult to parse.
• When expanding, it then loads a dozen lines that need to be scrolled to load the next dozen lines, which takes several minutes to actually reach the test results (I literally just give up and open the raw logs).
• That's made worse by the output being further broken, so each test result is split over 3 lines instead of one, so it takes 3x as long to get to the bottom.

[timrid]

• When expanding, it then loads a dozen lines that need to be scrolled to load the next dozen lines, which takes several minutes to actually reach the test results (I literally just give us and open the raw logs).

I think that is the most annoying issue. It seems that the grouping in Github Actions has an long standing performance issue. Maybe the performance is better when the grouping is removed. cibuildwheel is detecting Github Actions automatically and then adds grouping to the logs, but maybe we could trick it into thinking that it is not running in Github Actions, so that it does not add grouping.

• Missing colour makes it difficult to parse.

It is easily possible to add color to Android logs. iOS is more difficult, because that probably needs an upstream CPython change in the way the test runner captures and prints the output, because there the color infos get lost. I added a issue for that problem: python/cpython#150932

• That's made worse by the output being further broken, so each test result is split over 3 lines instead of one, so it takes 3x as long to get to the bottom.

This is also only an issue on iOS and not on Android. It has also something to do with the way the test runner captures and prints the output. I addressed that in the same CPython issue.