Skip to content

flutter-analyze bot is flaky in step "analyze flutter/flutter" #47881

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Closed
srawlins opened this issue Dec 9, 2021 · 5 comments
Closed

flutter-analyze bot is flaky in step "analyze flutter/flutter" #47881

srawlins opened this issue Dec 9, 2021 · 5 comments
Assignees
@stereotype441
Labels
area-infrastructure Use area-infrastructure for SDK infrastructure issues, like continuous integration bot changes. legacy-area-analyzer Use area-devexp instead. P2 A bug or feature request we're likely to work on type-enhancement A request for a change that isn't a bug

Comments

@srawlins
Copy link
Member

srawlins commented Dec 9, 2021

From @athomas, regarding pub scripts possibly clobbering each other or corrupting a pub cache:

One thing we could try is to set the PUB_CACHE environment variable in the script that frequently fails. We could either set it to a temp directory or use the checkout directory (where the SDK root is). The latter is cached between builds (if these caches get corrupted it's a bad thing but otherwise it might speed up the builds considerably).

CC @stereotype441 @pq
@srawlins srawlins added legacy-area-analyzer Use area-devexp instead. area-infrastructure Use area-infrastructure for SDK infrastructure issues, like continuous integration bot changes. P2 A bug or feature request we're likely to work on type-enhancement A request for a change that isn't a bug labels Dec 9, 2021
@stereotype441 stereotype441 self-assigned this Dec 9, 2021
@stereotype441
Copy link
Member

Trying this now

copybara-service bot pushed a commit that referenced this issue Dec 9, 2021
@stereotype441
Place the pub cache under the checkout directory when running snippet…
3e6d9a5 
… tests.

This is an attempt to address some recent flakiness in the Flutter analyze trybots.

Bug: #47881
Change-Id: I46eb1fca52fef4bbd3854c5da226512a579e11fb
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/222780
Reviewed-by: Samuel Rawlins <[email protected]>
Commit-Queue: Paul Berry <[email protected]>
@stereotype441
Copy link
Member

3e6d9a5 helped but did not entirely fix the problem. I'm re-titling the bug to reflect the more general flakiness problem with the flutter bot.

@stereotype441 stereotype441 changed the title Set PUB_CACHE in flutter-analyze bot flutter-analyze bot is flaky in step "analyze flutter/flutter" Dec 10, 2021
@stereotype441 stereotype441 mentioned this issue Dec 13, 2021
Merged
8 tasks
@whesse
Copy link
Contributor

whesse commented Jan 7, 2022

The duplicate step can now be removed, since the flutter script dev/bots/analyze.dart now analyzes the snippets correctly.

The temporary directory is automatically deleted when the builder finishes, and the cleanup step in this script was in a signal trap, which seems to have been causing the instability. So I am removing the cleanup step as well.

https://dart-review.googlesource.com/c/sdk/+/226685

copybara-service bot pushed a commit that referenced this issue Jan 7, 2022
@whesse
[infra] Remove duplicate flutter analyze snippets step from script
47f537c 
The snippets analyze step in [flutter]/dev/bots/analyze.dart now uses our built SDK when analyzing the code snippets. The explicit duplicate command to analyze the snippets with our SDK can now be removed.

The temp directory created in the script is not in the builder cache
on swarming, so it is automatically deleted when the builder finishes.
So we remove the cleanup step, which was trapping signals. That trap
seems to have been responsible for these failures and exit code confusion.

Bug: #47881
Change-Id: I49406954af54a0fba1ceec230c3551f8ad14e8e9
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/226685
Reviewed-by: Paul Berry <[email protected]>
@jason-simmons
Copy link
Contributor

The Flutter analyze_sample_code.dart tool is starting multiple concurrent pub global run snippets processes.

This may be unsafe (see dart-lang/pub#3165).

Flutter's analyzer CI script is running pub global run snippets once before launching analyze_sample_code.dart in order to configure the pub cache
(see https://github.com/flutter/flutter/blob/master/dev/bots/docs.sh#L34)

Dart's flutter-analyze bot may also need to do that to reduce flakes.

copybara-service bot pushed a commit that referenced this issue Jan 13, 2022
@athomas
[infra] Remove flaky flutter-analyze-try from the default CQ
418795c 
It's currently flaking on 50% of builds, providing almost no signal.

Bug: #47881
Change-Id: I4376f281ac0b78f9b14cfeeefa9b555756b5008d
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/228041
Auto-Submit: Alexander Thomas <[email protected]>
Reviewed-by: Paul Berry <[email protected]>
Commit-Queue: Paul Berry <[email protected]>
@sigmundch
Copy link
Member

I have not seen this failure come up in the last 200 runs of the flutter-analyze bot. dart-lang/pub#3285 from @sigurdm landed in the main Dart branch in bc75541 on Jan 27 and the last failure of this kind was two days prior to that on Jan 25! I'm cautiously optimistic that this is now fixed 🎉

I'll close this issue at this time, but we can reopen if we encounter it again.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@stereotype441 stereotype441

Labels
area-infrastructure Use area-infrastructure for SDK infrastructure issues, like continuous integration bot changes. legacy-area-analyzer Use area-devexp instead. P2 A bug or feature request we're likely to work on type-enhancement A request for a change that isn't a bug
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
5 participants
@srawlins @stereotype441 @sigmundch @whesse @jason-simmons