[Android] Add CoreCLR smoke tests for Android

[kotlarmilos]

Description

This PR adds CoreCLR smoke tests for Android. The libraries are selected to cover components that differ on Android compared to linux bionic and other platforms:

• Networking
• Diagnostics
• InteropServices
• Functional tests (static and dynamic linking)

This will be expanded with cryptography and additional networking tests once those tests are fixed.

Contributes to #114951

[dotnet-policy-service]

Tagging subscribers to this area: @hoyosjs
See info in area-owners.md if you want to be subscribed.

[Copilot]

Pull Request Overview

This PR adds CoreCLR runtime test jobs for Android by updating the CI pipeline configurations.

• Introduces a new CoreCLR runtime tests job for Android emulator in runtime-extra-platforms-androidemulator.yml.
• Updates the Android devices pipeline in runtime-extra-platforms-android.yml to re-enable Mono tests and add a new CoreCLR tests job.

Reviewed Changes

Copilot reviewed 2 out of 2 changed files in this pull request and generated 1 comment.

File	Description
eng/pipelines/extra-platforms/runtime-extra-platforms-androidemulator.yml	Adds a new job block for CoreCLR runtime tests using the Android emulator.
eng/pipelines/extra-platforms/runtime-extra-platforms-android.yml	Re-enables the Mono job and adds a new CoreCLR job for Android devices.

[Copilot]

[nitpick] The CoreCLR and Mono job definitions share many similar parameters. Consider abstracting the common parts into a reusable template to reduce duplication and simplify maintenance.

[kotlarmilos]

/azp run runtime-android,runtime-androidemulator

[kotlarmilos]

/azp run runtime-android,runtime-androidemulator

[azure-pipelines]

Azure Pipelines successfully started running 2 pipeline(s).

[kotlarmilos]

/azp run runtime-extra-platforms

[azure-pipelines]

Azure Pipelines successfully started running 1 pipeline(s).

[jkotas]

I see "runtime (Build android-x64 Release AllSubsets_CoreCLR" timing out on number of other PRs. I suspect that the tests run by this PR killed a bunch of Android devices.

A few examples:

#116569 - runtime (Build android-x64 Release AllSubsets_CoreCLR) Started 5 hours ago
#116557 - runtime (Build android-x64 Release AllSubsets_CoreCLR) Started 7 hours ago

[kotlarmilos]

It likely happened because tests for both Mono and CoreCLR were enabled. I’ll narrow the scope to src/tests/JIT or an even smaller subset. Do you have any suggestions for a subset that cover the basic functionality?

[jkotas]

The JIT is exactly same between regular Linux and Android. The JIT has a ton of tests (vast majority of tests under src\tests are JIT tests) so they are expensive to run. I do not think it is useful to run the JIT tests on Android in the default PR set.

Do you have any suggestions for a subset that cover the basic functionality?

For broad cheap coverage of basic functionality, I would run a small subset of libraries tests (these subsets are triggered using RunSmokeTestsOnly):

For example, here is a subset we run for native aot: https://github.com/dotnet/runtime/blob/main/src/libraries/tests.proj#L581-L592

For Android, you want to create similar subset that's tailored towards what's different on Android. For example, the subset does not need to include System.Linq.Expressions or System.Collections since they are exactly same on Android, but it may want to include System.Security.Cryptography or some networking tests since those components are different on Android.

[steveisok]

I see "runtime (Build android-x64 Release AllSubsets_CoreCLR" timing out on number of other PRs. I suspect that the tests run by this PR killed a bunch of Android devices.

A few examples:

#116569 - runtime (Build android-x64 Release AllSubsets_CoreCLR) Started 5 hours ago #116557 - runtime (Build android-x64 Release AllSubsets_CoreCLR) Started 7 hours ago

The x64 leg is for emulators and we "should" have plenty of capacity to handle the load. @kotlarmilos can you nag FR and see why we're seemingly so low on capacity?

[steveisok]

The JIT is exactly same between regular Linux and Android. The JIT has a ton of tests (vast majority of tests under src\tests are JIT tests) so they are expensive to run. I do not think it is useful to run the JIT tests on Android in the default PR set.

I think these are only being added to the pipelines that run on the rolling build or need to be manually triggered. It is a fair question to wonder if a subset of key tests makes sense generally since we have adequate linux coverage.

[kotlarmilos]

The x64 leg is for emulators and we "should" have plenty of capacity to handle the load. @kotlarmilos can you nag PR and see why we're seemingly so low on capacity?

I'll follow up on the capacity with the engineering team.

For Android, you want to create similar subset that's tailored towards what's different on Android. For example, the subset does not need to include System.Linq.Expressions or System.Collections since they are exactly same on Android, but it may want to include System.Security.Cryptography or some networking tests since those components are different on Android.

I agree. For android, we should already have coverage through the linux jobs, so I'll update the smoke tests to cover the platform-specific differences.