Add support for Testing module

[marcprux]

Macros work with the latest swift 6 Android toolchain on a macOS host (🎉), but Testing is not included in the built SDK, resulting in errors when building a newly created skip package init project:

[2/12] Emitting module swift6_test_demoTests
/Users/marc/Desktop/swift6-test-demo/Tests/swift6-test-demoTests/swift6_test_demoTests.swift:1:8: error: no such module 'Testing'
1 | import Testing
  |        `- error: no such module 'Testing'
2 | @testable import swift6_test_demo
3 | 
[3/12] Compiling swift6_test_demoTests swift6_test_demoTests.swift
/Users/marc/Desktop/swift6-test-demo/Tests/swift6-test-demoTests/swift6_test_demoTests.swift:1:8: error: no such module 'Testing'
1 | import Testing
  |        `- error: no such module 'Testing'
2 | @testable import swift6_test_demo
3 | 

I had added --swift-testing --install-swift-testing to the build-script in #167 and added swift-testing to get-packages-and-swift-source.swift, hoping that would be enough to build and include the module, but that doesn't seem sufficient.

Is there some additional step that is needed to get the Testing.swiftmodule built and included?

[finagolfin]

No idea, I have never tried that new testing framework with Android and only tried it once or twice on linux. Given that cross-compiling packages with macros was broken, I figured there was no point since it wouldn't work and they only finally added it to the toolchain last month, swiftlang/swift#75515.

My guess would be that their build script requires some work on installation, if it built fine on the CI. Feel free to look into it if you want to use it, I've got other more important stuff to deal with.

[finagolfin]

Now that 6.0.3 is out the door, I finally got around to looking at this. I was able to easily build this library natively on Android, given the work that others have already put into porting it to Android. All tests passed, except for two experimental ones that are labeled deprecated but still executed for some reason.

The only remaining build problem is the one we discussed in swiftlang/swift#76840 last month: this library uses a CMake toolchain file for cross-compilation in the official build-script, but we've never added Android support for that.

I'll get my swift-testing changes into a pull- as I got its posix_spawn feature working on Android, though exit tests are still disabled as many of those tests fail- and look into that CMake toolchain file support next. Also, given all the macros in this library, it's possible it just won't work when cross-compiling because of swiftlang/swift-package-manager#8094, so I will check that and let you know.

[marcprux]

Fantastic! Thanks for looking into this. Lack of Testing is the single biggest shortcoming in the Android story right now IMO, since swift package init creates test cases that won't build or run and more packages are starting to adopt the framework.

I'll be happy to help in any way with this, although I admit I quickly get lost when delving into the CMake labyrinth.

[marcprux]

You might also be interested in my index of package build statuses for Android at https://swift-everywhere.org.

The workflow is not currently building the package's tests since lack of the Testing module was causing a lot of them to unnecessarily fail, so I'm eager to be able to turn that on in the future.

[finagolfin]

Got this into the upcoming 6.1 and 6.2 bundles on the CI, try them out and report any issues. Marc said it works on linux but not macOS for him. Keep this issue open until I get it into a release bundle.

[marcprux]

Since testing is working for me both on Linux and macOS (provided I include the -Xswiftc -plugin-path -Xswiftc ${SWIFT_INSTALLATION}/lib/swift/host/plugins/testing flags workaround needed for the macros), I think we can close this issue once it makes it into a release bundle.

One question: I notice we need libandroid-execinfo to provide support for backtrace. Do you see any reason why we couldn't include libandroid-execinfo/execinfo.c in the swift-testing package itself to remove the need to this dependency? If not, I might file a PR to add it to _TestingInternals.

[finagolfin]

Up to you, Android added the API starting with API 33, so this backport library isn't needed if you target those recent APIs.

[marcprux]

Thanks for the tip. API 33 is currently on only 57% of devices (Skip targets API 29, FTR). OTOH, it is tough to justify extra work for a problem that will fix itself given enough time…

It also brings up the issue of which external library dependencies will we be requiring for an official Android SDK, but that is a broader issue that we may want to discuss in the Android WG.

[finagolfin]

I just released a 6.1 bundle, which includes Testing for the first time. Try it out and open new issues for any problems you see.