Skip to content

Add our Android build to Flutter's customer tests #774

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
gnprice opened this issue Jun 28, 2024 · 2 comments
Open

Add our Android build to Flutter's customer tests #774

gnprice opened this issue Jun 28, 2024 · 2 comments
Labels
a-Android Issues specific to Android, or requiring Android-specific work a-tests a-tools Our own development tooling, scripts, and infrastructure

Comments

@gnprice
Copy link
Member

gnprice commented Jun 28, 2024

This will be a followup to:

After we have the Android build running in our own CI, we should register it in Flutter's "customer tests" (like we did with our flutter analyze and flutter test as #239) so that it runs as part of Flutter's CI too. In particular this would have caught #771 at an earlier stage — the PR that made that change in the Flutter repo would have already seen a test failure.

I think there aren't any existing tests of this kind in the flutter/tests registry, and it's possible it'll call for some augmenting of that system's API. In particular it runs each test suite for 10 iterations by default, to catch flakes; but an Android build is a lot slower than a flutter test run on a comparable size of app, enough to make that expensive. Plus when it's purely a build, it's hard to imagine it flaking. So we may want to run that test for only 1 iteration, while still running our flutter test for 10 iterations. Anyway, that'll be a subject to raise in #hackers-tests in Discord if, when working on this issue, we decide something like that is indeed wanted.

@gnprice gnprice added a-Android Issues specific to Android, or requiring Android-specific work a-tools Our own development tooling, scripts, and infrastructure a-tests labels Jun 28, 2024
@gnprice gnprice added this to the Beta 3: Summer 2024 milestone Jun 28, 2024
@gnprice
Copy link
Member Author

gnprice commented Sep 27, 2024

When implementing this, one thing to take care of is that we should have it use an adjusted configuration that doesn't error out on deprecation warnings, something akin to flutter analyze --no-fatal-infos.

We saw on #968 an example of a deprecation warning in the Android code (affecting one of our plugin dependencies). As discussed at #968 (comment) , tests in the Flutter "customer tests" are expected to ignore those when deciding pass/fail.


I believe we already error out (in the Android build) only on warnings, not info:

    lint {
        // For docs on options available here:
        //   https://developer.android.com/reference/tools/gradle-api/8.5/com/android/build/api/dsl/Lint
        checkAllWarnings = true
        warningsAsErrors = true
    }

so I guess our normal config already complies with the policy as actually written. But I believe the motivation for the policy is to avoid erroring on deprecations. So if we can, it'd be good to find a way to get deprecations treated as "info" level, so that they don't cause errors.

From the doc linked in that comment:
https://developer.android.com/reference/tools/gradle-api/8.5/com/android/build/api/dsl/Lint
there's the informational option. That should do it, if there's an appropriate name we can pass to that option.

@gnprice gnprice modified the milestones: Beta 4: Fall 2024, Launch Nov 21, 2024
@gnprice
Copy link
Member Author

gnprice commented Feb 6, 2025

We've had a second occasion when this would have been useful for us:

In this instance, unlike #771, there's nothing I expect the upstream change would have ultimately done differently. (More discussion here: flutter/flutter#162675 (comment) ) Having our Android build in the upstream test suite would just have meant that we'd have heard about the change before it broke us — potentially even in the form of a PR — rather than through our own debugging after it broke us.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
a-Android Issues specific to Android, or requiring Android-specific work a-tests a-tools Our own development tooling, scripts, and infrastructure
Projects
Status: No status
Development

No branches or pull requests

1 participant