-
Notifications
You must be signed in to change notification settings - Fork 28.6k
Always use user-level pub cache #121802
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Always use user-level pub cache #121802
Conversation
@godofredoc you might want to look at the changes to the release packaging script |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
LGTM
dev/bots/prepare_package.dart
Outdated
/// These archives will be installed in the user-level cache on first | ||
/// following flutter command that accesses the cache. | ||
/// | ||
/// This assumes that all packages currently in the cache are installed from |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Is this a safe assumption? What about users overriding PUB_HOSTED_URL
? https://docs.flutter.dev/community/china#configuring-flutter-to-use-a-mirror-site
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Yes, this assumption is only as a precondition to the function, inside the prepare_package script.
Edited the comment to clarify.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
oh, I missed what library we were in :)
Future<void> _downloadPubPackageArchives() async { | ||
final Pool pool = Pool(10); // Number of simultaneous downloads. | ||
final http.Client client = http.Client(); | ||
final Directory preloadCache = Directory(path.join(flutterRoot.path, '.pub-preload-cache')); |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Do we need to add this to the top-level repo .gitignore?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Yes! good point.
Done
merged in upstream to fix linux build tests |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
@Jasguerrero can you give final approval?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
LGTM
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
LGTM
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
DBC
Did we manually inspect the results of prepare_package.dart
to ensure that it has the same size and rough contents?
There are also many style issues in this PR. In particular, the style guide asks us to avoid breaking a line on an assignment operator. See: https://github.com/flutter/flutter/wiki/Style-guide-for-Flutter-repo#prefer-a-maximum-line-length-of-80-characters
final String name = package.key; | ||
final Map<String, dynamic> versions = package.value as Map<String, dynamic>; | ||
for (final String version in versions.keys) { | ||
downloads.add(fetchPackageArchive(name, version)); |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
If the pub cache on CI has multiple versions of the same package, will they all be included in the flutter.zip?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
They should, as we iterate all versions for each package, and we store them as $name-$version.tar.gz
.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Is that the same as the old behavior, or will the release .zip now contain many versions of the same package that aren't needed. Remember that the flutter/flutter pubspec.yaml files pin all packages to a single version.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
We should have exactly the same set of package-versions as before.
The difference is that they are distributed as tar.gz files instead of inside a prepared cache.
We do the same steps to "warm up" a cache (by running pub get for some set of packages) and then, instead of folding that cache inside the zip, we iterate it and download the archive for each package.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
DBC
Did we manually inspect the results of
prepare_package.dart
to ensure that it has the same size and rough contents?
I'm working on this currently, will update.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I ran prepare_package.dart before and after this change on the same beta commit:
dart prepare_package.dart --branch=beta --no-publish --revision=1a0a03a41d6613c10bab23508dee456f184b181b --temp_dir=$HOME/git/tmp/prepare_package
And verified that the set of packages are the same, as well as the versions. The only difference is that after this change the individual packages are stored as tarballs.
If this sticks I can do a follow-up to clean up. |
Follow-up to #121802 resolving some style issues.
Follow-up to flutter#121802 resolving some style issues.
Current flutter has a .pub_cache inside the flutter_root and merges that with ~/.pub_cache if that exists,
With this PR we will always use pubs own resolution for finding the pub cache.
To add packages to the flutter.zip download they are packaged as tar.gz and added to the pub-cache on first run by using the new
pub cache preload
functionality.Fixes #37481
Pre-launch Checklist
///
).If you need help, consider asking for advice on the #hackers-new channel on Discord.