Incremental compiler fails to compile flutter hello_world example app

[aam]

IKG compiler for flutter hello_world fails to resolve dart:ui class references, produces warnings and errors to in that regard. It's behavior is different from batch compiler kernelFromProgram, which compiles flutter examples correctly.

Snippet of the errors:

file:///flutter/flutter/packages/flutter/lib/src/painting/box_painter.dart:67:5: Warning: 'Radius' isn't a type.
    Radius right: Radius.zero
    ^
file:///flutter/flutter/packages/flutter/lib/src/painting/box_painter.dart:85:59: Error: Getter not found: 'Radius'.
  static const BorderRadius zero = const BorderRadius.all(Radius.zero);
                                                          ^
file:///flutter/flutter/packages/flutter/lib/src/painting/box_painter.dart:85:59: Error: Compilation aborted due to fatal error.
  static const BorderRadius zero = const BorderRadius.all(Radius.zero);
                                                          ^
[+22086 ms] "flutter run" took 75,045ms.

[aam]

This is needed for #28051

[sigmundch]

I looked a bit into this with run.dart and found a few problems:

• If I run run.dart and give it a path to the sdk root with a Uri that is missing a scheme, we fail to find the sdk libraries. This is because of the extra check we do in source_loader.dart. I believe we don't need to do that anymore and have a fix for this issue.
• Then IKG fails when compiling the library cycle that contains dart:core because target.extraLibraryDependencies lists dart:ui. Fasta loads all those extra dependencies when compiling dart:core and IKG is not aware of these implicit dependencies.

I believe the right fix for the latter is to update how we deal with extraLibraryDependencies, I'll follow up on that. Most likely I'll remove dart:ui from the extra deps, and add it explicitly in the patch_sdk script.

Finally the issue reported above I believe is due to the re-exports, so they should also be covered once we fix #30448

[aam]

@a-siva

[sigmundch]

sent https://codereview.chromium.org/3006483002 to address the issue in source loader, and https://codereview.chromium.org/3007503002/ to address the issue with dart:ui.

For the latter, turns out IKG already had some of the logic for the implicit imports, but it was hardcoded to consider the VmTarget only, rather than the target that was being used during compilation.

I started an email discussion to change how we use "extraRequiredLibraries" long term, but those 2 CLs should address your issues here. One issue remains about exports, which is tracked in #30448

[aam]

Awesome, thanks @sigmundch !

[sigmundch]

I landed the fix for dart:ui, but partially addressed the source loader issue. There is still one more change to make to accept a URI without a scheme in the public API. That however should not block you.

[aam]

Thanks @sigmundch , but somehow, on eed3b5e I'm still getting many warnings when compiling with IKG:

[+4181 ms] compile debug message: Type 'VoidCallback' not found.
[        ] compile debug message: 'PointerDeviceKind' isn't a type.
[        ] compile debug message: 'Offset' isn't a type.

Not sure if submitted CLs intended to get rid of those in IKG.

[sigmundch]

Unfortunately no - I believe all those errors are a consequence of a reexport, so they will only be addressed after a fix for #30448 lands.

The changes above fix the other minor issues we found in the process of reproducing your error:

• f539785 addresses the problem that dart:ui was not found in IKG if it was also listed in the extraRequiredLibraries in FlutterTarget when compiling from sources
• d9ae60e is a first step to address the errors you saw when passing an sdk-root without a scheme. For now I'm improving the error message, later I plan to make it unnecessary to specify the scheme.

[sigmundch]

@aam - the final fix is not in, but @peter-ahe-google just landed a workaround for #30448 in aff5d76. Can you give it a try?

[aam]

Thanks folks. Just sync'ed to aff5d76 and tried flutter gallery build - there are no warning messages, but now there is an error produced:

[+3618 ms] compile debug message: 'Animation&AnimationEagerListenerMixin^#U0^' is exported from both 'package:flutter/src/animation/animation_controller.dart' and 'package:flutter/src/animation/animations.dart'.
[        ] compile debug message: 'Animation&AnimationEagerListenerMixin&AnimationLocalListenersMixin^#U0^^' is exported from both 'package:flutter/src/animation/animation_controller.dart' and 'package:flutter/src/animation/animations.dart'.
[        ] compile debug message: 'Animation&AnimationEagerListenerMixin&AnimationLocalListenersMixin&AnimationLocalStatusListenersMixin^#U0^^^' is exported from both 'package:flutter/src/animation/animation_controller.dart' and 'package:flutter/src/animation/animations.dart'.
[  +25 ms] stderr:>Unhandled exception:
[        ] stderr:>NoSuchMethodError: The getter 'length' was called on null.
[        ] stderr:>Receiver: null
[        ] stderr:>Tried calling: length
[        ] stderr:>#0      Object._noSuchMethod (dart:core-patch/object_patch.dart:42)
[        ] stderr:>#1      Object.noSuchMethod (dart:core-patch/object_patch.dart:46)
[        ] stderr:>#2      Uri.parse (dart:core/uri.dart:788)
[        ] stderr:>#3      DillLibraryBuilder.finalizeExports (package:front_end/src/fasta/dill/dill_library_builder.dart:150)
[        ] stderr:>#4      DillLoader.finalizeExports.<anonymous closure> (package:front_end/src/fasta/dill/dill_loader.dart:64)
[        ] stderr:>#5      _HashVMBase&MapMixin&&_LinkedHashMapMixin.forEach (dart:collection-patch/compact_hash.dart:356)
[        ] stderr:>#6      DillLoader.finalizeExports (package:front_end/src/fasta/dill/dill_loader.dart:62)
[        ] stderr:>#7      DillTarget.buildOutlines (package:front_end/src/fasta/dill/dill_target.dart:54)
[        ] stderr:><asynchronous suspension>
[        ] stderr:>#8      KernelDriver._compileCycle.<anonymous closure>.appendNewDillLibraries (package:front_end/src/incremental/kernel_driver.dart:233)
[        ] stderr:><asynchronous suspension>
[        ] stderr:>#9      KernelDriver._compileCycle.<anonymous closure> (package:front_end/src/incremental/kernel_driver.dart:269)
[        ] stderr:><asynchronous suspension>
[        ] stderr:>#10     PerformanceLog.runAsync (package:front_end/src/base/performace_logger.dart:52)
[        ] stderr:><asynchronous suspension>
[        ] stderr:>#11     KernelDriver._compileCycle (package:front_end/src/incremental/kernel_driver.dart:213)
[        ] stderr:><asynchronous suspension>
[        ] stderr:>#12     KernelDriver.getKernel.<anonymous closure>.<anonymous closure> (package:front_end/src/incremental/kernel_driver.dart:166)
[        ] stderr:><asynchronous suspension>
[        ] stderr:>#13     PerformanceLog.runAsync (package:front_end/src/base/performace_logger.dart:52)
[        ] stderr:><asynchronous suspension>
[        ] stderr:>#14     KernelDriver.getKernel.<anonymous closure> (package:front_end/src/incremental/kernel_driver.dart:163)
[        ] stderr:><asynchronous suspension>
[        ] stderr:>#15     PerformanceLog.runAsync (package:front_end/src/base/performace_logger.dart:52)
[        ] stderr:><asynchronous suspension>
[        ] stderr:>#16     KernelDriver.runWithFrontEndContext.<anonymous closure> (package:front_end/src/incremental/kernel_driver.dart:195)
[        ] stderr:>#17     CompilerContext.runInContext.<anonymous closure> (package:front_end/src/fasta/compiler_context.dart:89)
[        ] stderr:>#18     _rootRun (dart:async/zone.dart:1120)
[        ] stderr:>#19     _CustomZone.run (dart:async/zone.dart:1001)
[        ] stderr:>#20     runZoned (dart:async/zone.dart:1467)
[        ] stderr:>#21     CompilerContext.runInContext (package:front_end/src/fasta/compiler_context.dart:89)
[        ] stderr:>#22     CompilerContext.runWithOptions (package:front_end/src/fasta/compiler_context.dart:96)
[        ] stderr:>#23     KernelDriver.runWithFrontEndContext (package:front_end/src/incremental/kernel_driver.dart:194)
[        ] stderr:><asynchronous suspension>
[        ] stderr:>#24     KernelDriver.getKernel (package:front_end/src/incremental/kernel_driver.dart:131)
[        ] stderr:><asynchronous suspension>
[        ] stderr:>#25     IncrementalKernelGeneratorImpl.computeDelta.<anonymous closure> (package:front_end/src/incremental_kernel_generator_impl.dart:102)
[        ] stderr:><asynchronous suspension>
[        ] stderr:>#26     PerformanceLog.runAsync (package:front_end/src/base/performace_logger.dart:52)
[        ] stderr:><asynchronous suspension>
[        ] stderr:>#27     IncrementalKernelGeneratorImpl.computeDelta (package:front_end/src/incremental_kernel_generator_impl.dart:100)
[        ] stderr:>#28     _FrontendCompiler.compile (package:frontend_server/server.dart:128)
[        ] stderr:><asynchronous suspension>
[        ] stderr:>#29     starter.<anonymous closure> (package:frontend_server/server.dart:217)
[        ] stderr:><asynchronous suspension>
[        ] stderr:>#30     _RootZone.runUnaryGuarded (dart:async/zone.dart:1307)
[        ] stderr:>#31     _BufferingStreamSubscription._sendData (dart:async/stream_impl.dart:330)
[        ] stderr:>#32     _BufferingStreamSubscription._add (dart:async/stream_impl.dart:257)
[        ] stderr:>#33     _SinkTransformerStreamSubscription._add (dart:async/stream_transformers.dart:68)
[        ] stderr:>#34     _EventSinkWrapper.add (dart:async/stream_transformers.dart:15)
[        ] stderr:>#35     _StringAdapterSink.add (dart:convert/string_conversion.dart:268)
[        ] stderr:>#36     _LineSplitterSink._addLines (dart:convert/line_splitter.dart:156)
[        ] stderr:>#37     _LineSplitterSink.addSlice (dart:convert/line_splitter.dart:131)
[        ] stderr:>#38     StringConversionSinkMixin.add (dart:convert/string_conversion.dart:189)
[        ] stderr:>#39     _SinkTransformerStreamSubscription._handleData (dart:async/stream_transformers.dart:120)
[        ] stderr:>#40     _RootZone.runUnaryGuarded (dart:async/zone.dart:1307)
[        ] stderr:>#41     _BufferingStreamSubscription._sendData (dart:async/stream_impl.dart:330)
[        ] stderr:>#42     _BufferingStreamSubscription._add (dart:async/stream_impl.dart:257)
[        ] stderr:>#43     _SinkTransformerStreamSubscription._add (dart:async/stream_transformers.dart:68)
[        ] stderr:>#44     _EventSinkWrapper.add (dart:async/stream_transformers.dart:15)
[        ] stderr:>#45     _StringAdapterSink.add (dart:convert/string_conversion.dart:268)
[        ] stderr:>#46     _StringAdapterSink.addSlice (dart:convert/string_conversion.dart:273)
[        ] stderr:>#47     _Utf8ConversionSink.addSlice (dart:convert/string_conversion.dart:348)
[        ] stderr:>#48     _Utf8ConversionSink.add (dart:convert/string_conversion.dart:341)
[        ] stderr:>#49     _ConverterStreamEventSink.add (dart:convert/chunked_conversion.dart:86)
[        ] stderr:>#50     _SinkTransformerStreamSubscription._handleData (dart:async/stream_transformers.dart:120)
[        ] stderr:>#51     _RootZone.runUnaryGuarded (dart:async/zone.dart:1307)
[        ] stderr:>#52     _BufferingStreamSubscription._sendData (dart:async/stream_impl.dart:330)
[        ] stderr:>#53     _BufferingStreamSubscription._add (dart:async/stream_impl.dart:257)
[        ] stderr:>#54     _StreamController&&_SyncStreamControllerDispatch._sendData (dart:async/stream_controller.dart:796)
[        ] stderr:>#55     _StreamController._add (dart:async/stream_controller.dart:667)
[        ] stderr:>#56     _StreamController.add (dart:async/stream_controller.dart:613)
[        ] stderr:>#57     _Socket._onData (dart:io-patch/socket_patch.dart:1643)
[        ] stderr:>#58     _RootZone.runUnaryGuarded (dart:async/zone.dart:1307)
[        ] stderr:>#59     _BufferingStreamSubscription._sendData (dart:async/stream_impl.dart:330)
[        ] stderr:>#60     _BufferingStreamSubscription._add (dart:async/stream_impl.dart:257)
[        ] stderr:>#61     _StreamController&&_SyncStreamControllerDispatch._sendData (dart:async/stream_controller.dart:796)
[        ] stderr:>#62     _StreamController._add (dart:async/stream_controller.dart:667)
[        ] stderr:>#63     _StreamController.add (dart:async/stream_controller.dart:613)
[        ] stderr:>#64     new _RawSocket.<anonymous closure> (dart:io-patch/socket_patch.dart:1228)
[        ] stderr:>#65     _NativeSocket.issueReadEvent.issue (dart:io-patch/socket_patch.dart:784)
[        ] stderr:>#66     _microtaskLoop (dart:async/schedule_microtask.dart:41)
[        ] stderr:>#67     _startMicrotaskLoop (dart:async/schedule_microtask.dart:50)
[        ] stderr:>#68     _runPendingImmediateCallback (dart:isolate-patch/isolate_patch.dart:99)
[        ] stderr:>#69     _RawReceivePortImpl._handleMessage (dart:isolate-patch/isolate_patch.dart:152)

[scheglov]

Do you use FileByteStore, or MemoryByteStore?

This commit changes our expectations about libraries, so old entries in persistent ByteStore are not valid. Try to remove the results in the file system cache, or increment KernelDriver.DATA_VERSION.

[aam]

@scheglov , if I understand correctly, I didn't explicitly set which byte store to use(https://github.com/flutter/engine/blob/master/frontend_server/lib/server.dart#L114), so it defaulted to NullByteStore.
I tried using MemoryByteStore (..byteStore = new MemoryByteStore()), that didn't change the way it failed.