Indicated size is invalid. when running parallel instances of dart pub run

[gaaclarke]

This tracker is for issues related to:

• Dart VM

description

I have a CI task that is using Dart to do code generation. After I parallelized execution of the instances of "pub run" I noticed a flake where sometimes the execution would fail to load the kernel binary. In this instances pub run is not caching the build because I'm using it to execute local code, so there appears to be a race condition in reading artifacts from the disk.

error

Can't load Kernel binary: Invalid kernel binary: Indicated size is invalid.

example

https://cirrus-ci.com/task/6651964250390528?logs=local_tests#L408

steps to reproduce

Here is the roughly the code I introduced and it is executing 6 instances at once:

Future<int> _runPigeon(...) async {
  final Process generate =
    await Process.start('dart', <String>['pub', 'run', 'pigeon', '--input', input]);
  return await generate.exitCode;
}

Future<int> _generateDart(Map<String, String> jobs) async {
  final List<Future<int>> results = <Future<int>>[];
  for (final MapEntry<String, String> job in jobs.entries) {
    results.add(
        _runPigeon(input: job.key, dartOut: job.value, streamOutput: false));
  }
  final List<int> resultsList = await Future.wait(results);
  return resultsList.firstWhere((int x) => x != 0, orElse: () => 0);
}

version

Dart version 2.17.0 (build 2.17.0-27.0.dev)

[gaaclarke]

This could also be a bug in that version of Dart and I just happened to see it after I parallelized the tasks. I can't prove it's related.

[alexmarkov]

Looks like a pub issue, probably a duplicate of dart-lang/pub#3165.
/cc @sigurdm

[sigurdm]

Yeah - hopefully this should be fixed by: dart-lang/pub#3285 that has just landed

(Please reopen if this is not true).

A workaround would be to do a single. non-parallel run first to ensure the compilation is stable, and then run the rest in parallel...