-
Notifications
You must be signed in to change notification settings - Fork 232
Use the frontend server to compile pub executables #2968
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Conversation
Possible resolution to #2962 as well - in the case of no changes the precompile step is now much faster. |
I will look more into the windows failures and do additional cleanup if we want to push forward with this change. |
cc @a-siva would you be able to speak to the implications of running from kernel versus snapshot? |
Could you clarify what kind of 'snapshot' you are referring to here?
(aot-snapshot, jit-snapshot or just plain kernel which is also sometimes referred to as a snapshot). |
It is just Line 55 in 018c965
|
if you are just using the --snapshot option and not passing in the --snapshot-kind option then the default is kernel
which means you are already only producing a kernel file in this step. |
Ok thanks @a-siva so there should be no difference in terms of the result then. I do see some slight difference in output, the md5 hash is different and the new result is a bit smaller - (18016104 bytes versus 18633360 bytes). |
Is it possible that the version of CFE used in the two cases is different. |
@jakemac53 looks great to me.. I'm not familiar with Only concern is how this will interact when users switch SDK versions forwards and backwards... But that might not be so bad.. also it might affect global packages too in the same way. |
Yes, this is the same tool that flutter uses for incremental recompilation. edit: just to clarify flutter does not use
The way I have implemented this it should all just work. I haven't changed the invalidation logic around the actual runnable executables, so they still get invalidated (deleted) in the same way as before. When running from a cached dill file, the frontend server automatically throws it away if its kernel version doesn't match afaik. At least in some way it does handle this case and does a full recompile. |
@jonasfj the remaining test failure is here https://github.com/dart-lang/pub/pull/2968/checks?check_run_id=2442925564#step:7:415 (same failing test on the 3 bots).
Do you have any more details/info on this test? |
@jonasfj This is ready to land from my perspective, but I will let you hit the button unless you want me to :D. |
Opening for discussion, I wanted to try this out to see if we can speed up the rebuilds of pub executables - it does appear to be much faster (basically immediate if there is no change). Surprisingly, it is almost twice as fast on the initial precompile as well for me. I think that probably means that the vm snapshot functionality is doing something additional - I don't know what the implications of using kernel instead might be.
Specifically this:
.dart_tool/pub/incremental
where it stores the previous dill outputs for each executable.Previous snapshot behavior:

New fe server behavior, first build:

New behavior, pub get and run again (no changes):

Note that in the rebuild case with no changes, it didn't actually log the time at all which I believe means it was <1 second.
cc @mit-mit