[infra] provision flutter from a gstore archive#8951
Conversation
There was a problem hiding this comment.
Code Review
This pull request updates the CI environment setup in tool/github.sh and tool/kokoro/setup.sh to download a pinned version of the Flutter SDK (3.22.0) instead of cloning the latest stable branch. The feedback highlights significant code duplication between the two scripts and recommends refactoring the provisioning logic into a shared script to adhere to the DRY principle. Additionally, it is suggested to use the -fLO flags with curl to ensure the scripts fail appropriately on server errors and correctly handle redirects.
I am having trouble creating individual review comments. Click here to see my feedback.
tool/github.sh (24-40)
[CONCERN] This logic for provisioning the Flutter SDK is duplicated across tool/github.sh and tool/kokoro/setup.sh. To adhere to the DRY principle and simplify future updates (e.g., changing the FLUTTER_VERSION), consider moving this provisioning logic into a separate shared shell script that both CI environments can source.
References
- DRY: Identify blocks of code that are 90%+ identical to existing utility methods in this repo and flag them for duplication.
tool/github.sh (30)
[CONCERN] Use the -f (or --fail) flag with curl to ensure the command returns a non-zero exit code if the server returns an error (e.g., 404). Without this, the script might continue with an empty or corrupted file despite set -e. Adding -L is also recommended to follow redirects.
curl -fLO "https://storage.googleapis.com/flutter_infra_release/releases/stable/macos/flutter_macos_${FLUTTER_VERSION}-stable.zip"
tool/github.sh (34)
[CONCERN] Use the -f (or --fail) flag with curl to ensure the command returns a non-zero exit code if the server returns an error. Adding -L is also recommended.
curl -fLO "https://storage.googleapis.com/flutter_infra_release/releases/stable/linux/flutter_linux_${FLUTTER_VERSION}-stable.tar.xz"
tool/kokoro/setup.sh (42)
[CONCERN] Use the -f (or --fail) flag with curl to ensure the command returns a non-zero exit code if the server returns an error. Adding -L is also recommended.
curl -fLO "https://storage.googleapis.com/flutter_infra_release/releases/stable/macos/flutter_macos_${FLUTTER_VERSION}-stable.zip"
tool/kokoro/setup.sh (46)
[CONCERN] Use the -f (or --fail) flag with curl to ensure the command returns a non-zero exit code if the server returns an error. Adding -L is also recommended.
curl -fLO "https://storage.googleapis.com/flutter_infra_release/releases/stable/linux/flutter_linux_${FLUTTER_VERSION}-stable.tar.xz"
|
(Gemini's code review comments were excellent. I took and applied all its advice.) |
## CI/CD Automation: Automated Flutter SDK Version Bumping & Integrity Verification ### Why This Matters To ensure stable and reproducible builds, our CI/CD scripts are pinned to a specific stable version of the Flutter SDK in `tool/provision_flutter.sh` (see: #8951). Manually monitoring and upgrading this pin is tedious, while a rolling pin risks introducing silent test breakages. This PR automates the maintenance of our pinned Flutter SDK version with safe, presubmit-verified upgrades, backed by compile-time integrity checks to prevent syntax or synchronization bugs. Fixes: #8953 --- ### Proposed Changes #### 1. Automated Version-Pin Bumping * **[NEW] [.github/workflows/update_flutter.yaml](file:///+github/workflows/update_flutter.yaml)**: Added a scheduled weekly GitHub Actions workflow (running every Monday at midnight) that: * Queries the official Flutter GCS manifest API for new stable releases. * Extracts and compares the latest version with our current pin inside `tool/provision_flutter.sh`. * If a newer version is available, it automatically edits the version constant in the script and opens a structured Pull Request so that standard presubmit checks run against the new version prior to merging. #### 2. Compile-Time Integrity Verification * **[NEW] [testSrc/unit/io/flutter/CIIntegrityTest.java](file:///testSrc/unit/io/flutter/CIIntegrityTest.java)**: Added a custom JUnit meta-test to enforce synchronization between the provisioning script and the GitHub Action: * `testFlutterProvisioningScriptIntegrity()`: Asserts `tool/provision_flutter.sh` exists and contains a valid semver-formatted `FLUTTER_VERSION="..."` constant. * `testGitHubWorkflowRegexSync()`: Asserts `.github/workflows/update_flutter.yaml` matches the exact constant pattern used in its search-and-replace step. * *Why this is here*: Prevents silent setup failures if a developer renames or reformats the version constant in the future. --- ### Verification Results 1. **JUnit Integrity Tests**: Ran the integrity suite locally: ```bash ./gradlew test --tests io.flutter.CIIntegrityTest Result: BUILD SUCCESSFUL (Both assertions compiled and passed successfully in 4 seconds). 2. **Local Run**: Until this is landed the action can't be tested on GH but this verifies that it should work: ``` ☁ flutter-intellij [infra_flutterSdkWorkflow] ⚡ # 1. Read the current pinned version from the script (should print 3.41.0) CURRENT_VERSION=$(grep -E 'FLUTTER_VERSION="[0-9.]+"' tool/provision_flutter.sh | head -n 1 | cut -d'"' -f2) echo "Current pinned version: $CURRENT_VERSION" # 2. Query the live Flutter API for the latest stable release version (should print 3.41.9+) LATEST_VERSION=$(curl -s https://storage.googleapis.com/flutter_infra_release/releases/releases_linux.json | jq -r '.releases | map(select(.channel == "stable"))[0].version') echo "Latest stable version: $LATEST_VERSION" # 3. Run the comparison and update logic if [ "$CURRENT_VERSION" != "$LATEST_VERSION" ]; then echo "New stable version detected: $LATEST_VERSION. Updating script..." # Perform the replacement (cross-compatible sed for macOS) sed -i "" "s/FLUTTER_VERSION=\"$CURRENT_VERSION\"/FLUTTER_VERSION=\"$LATEST_VERSION\"/g" tool/provision_flutter.sh echo "Successfully updated! Checking file content:" grep "FLUTTER_VERSION=" tool/provision_flutter.sh else echo "Flutter SDK is already up to date." fi Current pinned version: 3.41.0 Latest stable version: 3.41.9 New stable version detected: 3.41.9. Updating script... Successfully updated! Checking file content: FLUTTER_VERSION="3.41.9" ``` --- Review the contribution guidelines below: - [x] I’ve reviewed the contributor guide and applied the relevant portions to this PR. - [x] I've included the required information in the description above. - [x] My up-to-date information is in the `AUTHORS` file. - [x] I've updated `CHANGELOG.md` if appropriate. <details> <summary>Contribution guidelines:</summary><br> - See our [contributor guide](../CONTRIBUTING.md) and the [Flutter organization contributor guide]([https://github.com/flutter/flutter/blob/main/docs/contributing/Tree-hygiene.md) for general expectations for PRs. - Larger or significant changes should be discussed in an issue before creating a PR. - Dart contributions to our repos should follow the [Dart style guide](https://dart.dev/guides/language/effective-dart) and use `dart format`. - Java and Kotlin contributions should strive to follow Java and Kotlin best practices ([discussion](#8098)). </details> --------- Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>
Moves Flutter provisioning from a
git cloneto pulling down an archive which is much faster.First run is coming in at just under 17 minutes; down from an hour:
If this sticks, we'll be waiting 71.7% less time for builds to complete, and our deployment pipeline will be running 253% faster than it used to be. 🚀
Fixes: #8950
Review the contribution guidelines below:
AUTHORSfile.CHANGELOG.mdif appropriate.Contribution guidelines:
our contributor guide and
the Flutter organization contributor guide
for general expectations for PRs.
dart format.practices (discussion).