pub build / pub serve fails on 2.0.0-dev.39

[cedx]

If I build or serve my app using the --use-old-frontend flag of dart2js (via the pub transformers in the pubspec.yaml file), everything is OK.
If I remove the --use-old-frontend flag, a lot of errors raise from the charcode package (probably an indirect dependency of my app):

[...]
[Error from Dart2JS on pms_isol|web/js/main.dart]:
web/packages/charcode/html_entity.dart:621:1:
The control character U+0000 can only be used in strings and comments.
/// for all ('∀')
^
[Error from Dart2JS on pms_isol|web/js/main.dart]:
web/packages/charcode/html_entity.dart:630:1:
The control character U+0000 can only be used in strings and comments.
/// empty set (null set); see also U+8960, ⌀ ('∅')
^
[Error from Dart2JS on pms_isol|web/js/main.dart]:
web/packages/charcode/html_entity.dart:744:35:
String starting with ' must end with '.
/// left floor (APL downstile) ('⌊')
                                  ^
[Error from Dart2JS on pms_isol|web/js/main.dart]:
web/packages/charcode/html_entity.dart:621:17:
Expected a declaration, but got ')'.
/// for all ('∀')
                ^
[Error from Dart2JS on pms_isol|web/js/main.dart]:
web/packages/charcode/html_entity.dart:630:46:
Expected a declaration, but got '('.
/// empty set (null set); see also U+8960, ⌀ ('∅')
                                             ^
[Error from Dart2JS on pms_isol|web/js/main.dart]:
web/packages/charcode/html_entity.dart:630:47:
Expected a declaration, but got ''''.
/// empty set (null set); see also U+8960, ⌀ ('∅')
[...]

[sigmundch]

Thanks for filing this bug!

I was able to reproduce this locally as well.

I need to investigate in more detail, but there is something odd about how the pub/dart2js integration is working here, as file contents are not encoded/decoded correctly between the two. If I take the same project files and compile with dart2js on the command-line, then there is no issue.

I'll keep looking and report again soon.

/cc @nex3 in case this rings any bells from your understanding of the integration with barback

[nex3]

Not ringing any bells, no...

[sigmundch]

Good news - I found the cause behind this. Fix is out for review (https://dart-review.googlesource.com/c/sdk/+/46945).

The new front-end reads files in binary form, while the old one read them as Strings. Both were requesting them as Strings from pub, but we re-encoded strings incorrectly using code units instead of utf8. That codepath was basically never used before.

The fix is to do the correct translation. However, I may follow up with a change in pub to skip using strings entirely to avoid the double conversion in the first place.

[sigmundch]

I landed a fix, which should become available on the next dev release.