question: mirrors will be disabled in 2.0 - Is "reflectable" the way to go?

[MikeMitterer]

I mean for Dart4Web?
Please we need a clear strategy for this, essential, part of the language.

[eernstg]

I'm not laying out strategies, but it shouldn't be very hard to create a version of reflectable that uses 'build' rather than being a transformer. It would require main to invoke an initialization method, and it would presumably not include support for execution based on 'dart:mirrors' (it would have only one mode, based on generated code). One possible approach would be to make this 'reflectable 2.0', and then completely abandon the transformer based approach. The only thing missing here is some time to do it. ;-)

[MikeMitterer]

Thanks for your answer. We will see what the future brings. In my opinion this question should have been answered before Dart 2.0. The organic approach is nice but the whole reflection thing is to important to be so open.

Without knowing a clear strategy on reflections I can't make my packages Dart 2 ready. Rewriting the mirrors/reflection part twice is to expensive. It's really sad.

[MikeMitterer]

Just a reminder:

Remove support for dart:mirrors and dart:isolate from dart4web
dart-lang/sdk#30538

[eernstg]

@MikeMitterer, how would the branch https://github.com/dart-lang/reflectable/tree/use_build work for you? You'd need to change every root library: initializeReflectable() must now be called at the beginning of main, and the root library must import the generated code (if the root library is in the file myProgram.dart then the generated code will be in myProgram.reflectable.dart, which must be imported by myProgram.dart).

Code generation takes place when you do pub run reflectable:reflectable_builder <target> from the root of your package, where <target> is your root library (or several of them, as in test/*_test.dart).

There is currently no support for re-generating the code "when needed", but we're looking into that. So don't worry too much if you think "but I don't want to do pub run ... all the time".

[MikeMitterer]

Huuuh - Cool! Sounds great. I'll try it in the next few days.

[MikeMitterer]

@eernstg

Thanks!

I played with these samples https://github.com/MikeMitterer/dart-mdl-mustache/tree/mdlVersion/samples/browser2

Where

• └pkgref is a library-package (anotherlib.dart) that uses build_reflectable and defines it's own annotation for reflection
• └reflection has a lib (rlib.dart) that defines its own annotations and a main.dart which also defines another set of annotations.

And... what really surprised me - everything I tried just worked! 💪
My concerns were if this whole thing works cross-package - it does.

In my opinion this package should be somehow declared a the official way to handle reflections in Dart.

If there is no "official-way" to handle mirrors/reflections we end up with different packages using different techniques for this - and thats something nobody wants.

That's it - very clean:

import 'main.reflectable.dart';

main() {
    initializeReflectable();
    ...
}

Can I share this issue in Dartisans on G+?

---

The only thing that this lib crash was when I turned on dartdec in pubspec:
https://github.com/MikeMitterer/dart-mdl-mustache/blob/mdlVersion/samples/browser2/reflection/pubspec.yaml (turned it off after the crash)
This is the error-message I got:

Error compiling dartdevc module:reflection_test_browser2|web/web__main.js

[error] The argument type 'Iterable<ClassMirror>' can't be assigned to the parameter type 'List'. (/private/var/folders/m4/9rg16xhd2rlbln753n9zk1540000gn/T/pub_oztN7T/web/main.dart, line 69, col 42)
[error] The argument type 'Iterable<String>' can't be assigned to the parameter type 'List'. (/private/var/folders/m4/9rg16xhd2rlbln753n9zk1540000gn/T/pub_oztN7T/web/main.dart, line 71, col 36)
[error] The function expression type '(MyAnnotation) → void' isn't of type '(Object) → void'. This means its parameter or return type does not match what is expected. Consider changing parameter type(s) or the returned type(s). (/private/var/folders/m4/9rg16xhd2rlbln753n9zk1540000gn/T/pub_oztN7T/web/main.dart, line 109, col 18)

Please fix all errors before compiling (warnings are okay).

This is what https://github.com/MikeMitterer/dart-mdl-mustache/blob/mdlVersion/samples/browser2/reflection/web/main.dart shows:

[eernstg]

Thanks for all the kind words!

.. share this issue in Dartisans on G+?

You're welcome to do that!

I think the issues you mention with dartdevc are standard strong mode issues (that is, the code isn't as strictly typed as it needs to be with strong mode). Compilation with dartdevc can be made to work with a few adjustments (not that they useful fixes, they just illustrate what was going wrong):

diff --git a/samples/browser2/reflection/web/main.dart b/samples/browser2/reflection/web/main.dart
index bf65e45..4969e69 100644
--- a/samples/browser2/reflection/web/main.dart
+++ b/samples/browser2/reflection/web/main.dart
@@ -66,9 +66,9 @@ void main() {
     final InstanceMirror imLanguage = rtest.reflect(language);
     final ClassMirror cmLanguage = imLanguage.type;

-    log("Classes with rtest-Annotation:",rtest.annotatedClasses);
+    log("Classes with rtest-Annotation:",rtest.annotatedClasses as Iterable<Object>);

-    log("Instance members (keys):",cmLanguage.instanceMembers.keys);
+    log("Instance members (keys):",cmLanguage.instanceMembers.keys as Iterable<Object>);

     final methods = new List<String>();
     cmLanguage.instanceMembers.keys
@@ -106,7 +106,7 @@ void main() {

     final names = new List<String>();
     cmVersion.metadata.where((final Object o) => o is MyAnnotation)
-        .forEach((final MyAnnotation my) => names.add(my.name));
+        .forEach((final /*MyAnnotation*/ my) => names.add((my as MyAnnotation).name));
     log("MyAnnotation - Names:",names);

     final Animal animal = new Dog();

[MikeMitterer]

@eernstg I got it right that this branch will be part of the next reflectable release?

[eernstg]

Yes, that's the plan. I need some more feedback and maybe some adjustments, but then it should be doable.

Note that we're discussing code generation strategies: For now the good choice is to always do code generation for a complete package as one step (never just one program at a time). Example: For test_reflectable that means pub run reflectable:reflectable_builder test/*_test.dart, never ... test/just_one_test.dart.

[MikeMitterer]

OK. Generating code for the whole package sounds reasonable for me if you are talking about a lib or an app but for tests this doesn't sound very practically to me.

Code generation for the above sample takes ~10secs on my machine. This is OK for the whole app but very annoying while writing unit tests. The whole idea behind UT is having quick turnaround cycles...

[eernstg]

I'm pushing for a nicer solution to this, but right now there's still a workaround:

To generate code for a single root library (and not look at "everything else"), delete .dart_tool and all generated files, and then just go ahead and do

pub run reflectable:reflectable_builder <root-library-under-development-right-now>

as often as you need. When you wish to generate code for a different set of root libraries, delete .dart_tool and generated files again, and then start running the builder for that different set.

[jakemac53]

Fwiw, the long build time of ~10s is most likely because of doing single builds (calling build) instead of using the file watching mode (calling watch).

On the first build it has to build up an analysis context for the entire app, which can take a while, but subsequent builds will re-use the same context and update the changed files.

[eernstg]

Sounds like I need to know how/when to call watch, and do that inreflectable_builder.dart (or maybe create a new reflectable_watch.dart).

[jakemac53]

Because of how this is invoked with pub run I actually don't know if watch will work at all.... it might auto-kill itself after the first build because it thinks the build script has changed (or gives up trying to check)

[jakemac53]

Honestly, it would be a better experience long term for users to give instructions on how to write a build.dart file of their own (this is what built_value, source_gen, etc do).

Then they can actually get incremental builds and integrate other build steps into their pipeline as needed.

[eernstg]

But I have no particular need to run it with pub run I just need to know how to run it.

However, is it really the intention that every client package must write a build.dart and there can never be anything like a reusable reflectable_builder which can be used (perhaps like reflectable_builder <my-root-library>, or reflectable_builder <my-directory> or even reflectable_builder (working on "everything in this package")?

[eernstg]

.. it thinks the build script has changed

But why does it think that the build script has been changed in a situation where the number of modified files on disk is zero?!

[jakemac53]

However, is it really the intention that every client package must write a build.dart and there can never be anything like a reusable reflectable_builder which can be used (perhaps like reflectable_builder , or reflectable_builder or even reflectable_builder (working on "everything in this package")?

No, this is not the long term goal, its just the only way to run things right now (or via pub run as reflectable is doing). The long term goal is for this script to be autogenerated based on configuration, similar to how transformers work today.

But why does it think that the build script has been changed in a situation where the number of modified files on disk is zero?!

Because when running in pub run there is no way to tell what script is actually being ran, the uri you get for the root library is just a root relative http uri from the bin directory which could be in any package.

[eernstg]

Because when running in pub run there is no way to tell ..

OK, sounds like I should just remove all references to pub run in the documentation and give instructions to run bin/reflectable_builder.dart directly. That's sort of an exception, but it should not take extra time for developers in their daily work (as soon as they know, and it's in the PATH, it just works).

[jakemac53]

OK, sounds like I should just remove all references to pub run in the documentation and give instructions to run bin/reflectable_builder.dart directly. That's sort of an exception, but it should not take extra time for developers in their daily work (as soon as they know, and it's in the PATH, it just works).

If you are referring to the script in the reflectable package, this will also not work. It would require reaching into the pub cache or something along those lines (which would mean you would have to update your PATH every time you updated), and also wouldn't be supported by package:build (it enforces pub package semantics so you can only read files from the lib directory of other packages). It would therefore refuse to compute a digest of that file.

The only really supported way today of using package:build is for each user to create their own build script inside each package that they use reflectable.

[eernstg]

... reaching into the pub cache ...

Well, that's exactly the reason why I went for pub run (until I noticed that it is not supported). So that will definitely not be the recommended approach when things stabilize, but it seems to be the best option right now.

[eernstg]

How would anything stop me from running

> dart $REFLECTABLE_PATH/bin/reflectable_builder.dart web/myProgram.dart

where $REFLECTABLE_PATH reaches into the pub cache? It potentially breaks on every pub get / pub upgrade, but it is possible (I just did it and it works fine, anyway).

[jakemac53]

It won't stop you from running that file, but you will not be able to get incremental rebuilds because it will refuse to compute the input hash of that file.

We enforce the same rules on ourselves as we do on our users, because we go through the same abstractions to read/write files.

[eernstg]

.. that file

Which file is that?

[jakemac53]

Which file is that?

The build script itself, $REFLECTABLE_PATH/bin/reflectable_builder.dart

[eernstg]

Ah, $REFLECTABLE_PATH/bin/reflectable_builder.dart. But we can get access to other files in package reflectable, so it should be possible to let build.dart in each client package be almost textually identical to each other, so each client package developer would be able to just copy a standard build.dart, which would just be a few lines of code including import 'package:reflectable/reflectable_builder_now_in_lib.dart';, possibly changing the target from web/*.dart to bin/whatever.dart, and that's it.

So the point would be that client package developers need to write a build.dart because that's the only way they can get access to reflectable_builder.dart!

[jakemac53]

Yep! That is the recommended approach today.

[eernstg]

A script like this seems to work, and could easily be copied into each client package and customized a bit:

import 'package:reflectable/reflectable_builder.dart' as builder;

main(List<String> arguments) async {
  await builder.reflectableBuild(arguments);
}

And, of course, the client package could then in the typical setup give some fixed arguments, such that nobody gets confused. ;-)

Branch use_build does this now, e.g., in test_reflectable:

> rm -rf .dart_tool; rm -f test/*_test.reflectable.dart
> dart tool/build.dart test/*_test.dart
... # 11101ms with 69 outputs
> dart tool/build.dart test/*_test.dart
... # 177ms with 0 outputs
> touch test/basic_test.dart
> dart tool/build.dart test/*_test.dart
... # 3437ms with 1 output

[jakemac53]

It does exist (see changelog), but it doesn't look like it made it into an official release before the 2.0.0 switch so you would have to be on a dev release.

[eernstg]

Update on running reflectable on our continuous testing framework: Unfortunately the idea I had Wednesday did not work. I have a better idea (it looks exactly like a better idea right now, anyway ;-), but I won't get around to try it before early January.

[jakemac53]

Fwiw, package:build_runner is getting close to a release which would likely change how package:reflectable encourages its users to do things (although the existing way would still work).

If you are waiting to publish until early January anyways it may be worth waiting for that (it should be published either before the new year or very shortly after).

[eernstg]

Sounds good! In any case, it wouldn't be a problem to create a new version of reflectable with a different test_reflectable/tool/build.dart and publish that, so we don't really have to coordinate, I just need to make sure I'm reasonably up to date.

[jakemac53]

Sounds good! In any case, it wouldn't be a problem to create a new version of reflectable with a different test_reflectable/tool/build.dart and publish that, so we don't really have to coordinate, I just need to make sure I'm reasonably up to date.

The difference is that there would be no manual build.dart script at all :D. You would instead publish a build.yaml which describes which builders should be applied to anybody that depends on you.

[jakemac53]

Just sent out #107 which updates to the latest and greatest, have a look!

[MikeMitterer]

Any news on this?

[eernstg]

I still haven't managed to make it work with continuous testing (and I'm currently at DartConf). Half of Monday is allocated for trying again.

[luisvt]

Hi everybody, I created a library that does what you are trying to achieve with reflectable:
https://pub.dartlang.org/packages/built_mirrors

I would be very pleased to help in anything

[MikeMitterer]

Thanks @luisvt I think build_mirrors and reflectable are the two major players in this game!
Thanks to both of you for your great work!

My problem is that reflection (mirrors) is a very central point in my packages (MDL, Dice (I'll take this from Lars... in the next few weeks.) and and in many other packages.
Different systems for such a central topic is just nonsense! https://plus.google.com/+MikeMitterer/posts/b5TPA7G2XFk

However - it's hard to make a decision in this case...

[eernstg]

Did 3 half days on that continuous testing issue last week. Trying some more ideas tomorrow.

[MikeMitterer]

I really don't want to bother you but do you think we'll have a chance to get reflectable until end of February?

[eernstg]

I have this day job doing the Dart language specification, and we're really busy putting together this Dart 2 thing, but I'll push really hard for it.

[MikeMitterer]

I tried to use the GH repo until the new version gets released but now I get a version-constraint-error which I'm not able to resolve (Don't know where it comes from...)

[eernstg]

Funny, https://pub.dartlang.org/packages/analyzer/versions/0.31.1 should exist. This is what I get with pub upgrade, that is, when solving all version constraints. Maybe the culprit is that 'Overriding the upper bound Dart SDK constraint' doesn't happen in your setup?:

$ pub upgrade
Resolving dependencies... (9.8s)
Overriding the upper bound Dart SDK constraint to <=2.0.0-dev.35.0 for the following packages:

args, barback, boolean_selector, build_barback, build_config, built_collection, charcode, cli_util, code_transformers, convert, crypto, csslib, dart_style, fixnum, front_end, glob, graphs, html, http, http_multi_server, http_parser, io, isolate, js, kernel, logging, matcher, meta, multi_server_socket, node_preamble, package_config, package_resolver, path, plugin, pool, pub_semver, quiver, reflectable, shelf, shelf_packages_handler, shelf_static, shelf_web_socket, source_map_stack_trace, source_maps, source_span, stack_trace, stream_channel, string_scanner, term_glyph, test, typed_data, yaml

To disable this you can set the PUB_ALLOW_PRERELEASE_SDK system environment variable to `false`, or you can silence this message by setting it to `quiet`.
  analyzer 0.31.1 (0.31.2-alpha.0 available)
  args 1.4.1
  async 2.0.6
  barback 0.15.2+14
  boolean_selector 1.0.3
  build 0.12.0+2
  build_barback 0.5.0+3
  build_config 0.2.5
  build_resolvers 0.2.0
  build_runner 0.8.0
  build_test 0.10.1+1
  built_collection 3.0.5
  built_value 5.2.1
  charcode 1.1.1
  cli_util 0.1.2+1
  code_builder 3.0.3
  code_transformers 0.5.1+4
  collection 1.14.7
  convert 2.0.1
  crypto 2.0.2+1
  csslib 0.14.1
  dart_style 1.0.9+1 (1.0.10 available)
  fixnum 0.10.7
  front_end 0.1.0-alpha.9 (0.1.0-alpha.10 available)
  glob 1.1.5
  graphs 0.1.0
  html 0.13.3
  http 0.11.3+16
  http_multi_server 2.0.4
  http_parser 3.1.1
  io 0.3.2+1
  isolate 1.1.0
  js 0.6.1
  kernel 0.3.0-alpha.9 (0.3.0-alpha.10 available)
  logging 0.11.3+1
  matcher 0.12.1+4
  meta 1.1.2
  mime 0.9.6
  multi_server_socket 1.0.1
  node_preamble 1.4.0
  package_config 1.0.3
  package_resolver 1.0.2
  path 1.5.1
  plugin 0.2.0+2
  pool 1.3.4
  pub_semver 1.3.2
  quiver 0.28.0
  shelf 0.7.2
  shelf_packages_handler 1.0.3
  shelf_static 0.2.7
  shelf_web_socket 0.2.2
  source_map_stack_trace 1.1.4
  source_maps 0.10.4
  source_span 1.4.0
  stack_trace 1.9.2
  stream_channel 1.6.4
  stream_transform 0.0.10
  string_scanner 1.0.2
  term_glyph 1.0.0
> test 0.12.32+2 (was 0.12.32+1)
  typed_data 1.1.5
  utf 0.9.0+4
  watcher 0.9.7+7
  web_socket_channel 1.0.7
  yaml 2.1.13
Changed 1 dependency!
Precompiling executables... 
Precompiled test:test.

[eernstg]

Asked around about this version conflict, and the relation to the ever-present Dart 2 transition. Pub version constraints are handled in Seattle, so it'll probably be a few hours before they see it.

Edit Mar 21, 9:30am: No response yet.

[MikeMitterer]

The analyzer related error message means nothing. Thats a pub-solver problem. If it can't resolve a dependency it spits out this kind of message - no relation to reality. However, I have reduce my dependency to only reflectable:

name: 'reflection_test_browser2'

version: 0.0.1
description: An absolute bare-bones web app.
#author: Your Name <email@example.com>
#homepage: https://www.example.com

environment:
  sdk: '>=1.9.0 <2.0.0'

dependencies:
        
dev_dependencies:
  test: any
  
  reflectable:
    #'^1.0.0'
    git: git@github.com:dart-lang/reflectable.git
    #path: ../_repo/reflectable

Now I get

Package build_runner has no versions that match >=0.7.0 <0.8.0 derived from:
- reflectable 2.0.0-dev.2.0 depends on version ^0.7.0

Again - the message itself is crap. It means just that it cant resolve SOME! dependencies.

I'm running Dart 1.24.3 on this machine. I can't upgrade to 2.x because I think this would break some of my libs... I thought reflectable will also support Dart SDK < 2.x - I'm wrong?

[zoechi]

If there are no other dependencies, then it can only be a conflict in the Dart SDK version constraint
(you still have test in there though)
Is the stuff you want to depend on on the master branch? Otherwise you'd need to add ref: some-branch
https://www.dartlang.org/tools/pub/dependencies#git-packages

[MikeMitterer]

@zoechi Thanks but I already spend several hours fiddling with those version problems. It really starts to piss me off. All I want is to get rid off mirrors using the reflectable package. All I do is fighting with such stupid things.

[MikeMitterer]

I gave it another try:

dependency_overrides:
  test: "^0.12.32+2"

Should work but doesn't!

Package build_runner has no versions that match >=0.7.0 <0.8.0 derived from:
- reflectable 2.0.0-dev.2.0 depends on version ^0.7.0

So it's not "test" There are some other, Google internal, dependencies that can't be resolved on my side.

[MikeMitterer]

Finally - a cloned repo with these settings works:

name: reflectable
version: 2.0.0-dev.2.0
description: >
  This package allows programmers to reduce certain usages of dynamic
  reflection to a statically specified subset thereof, based on generated
  code with the same behavior as mirrors provided by 'dart:mirrors'. The
  generated code does not use dynamic reflection and thus improves the
  resource economy (esp. by using a smaller amount of space than dart2js
  generated code).
author: The Dart Team <[email protected]>
homepage: https://www.github.com/dart-lang/reflectable

environment:
  sdk: '>=1.12.0 <2.0.0'

dependencies:
  analyzer: any
  barback: any
  build: any
  build_barback: any
  build_config: any
  build_runner: any
  
  code_transformers: '>=0.4.1 <0.6.0'
  dart_style: '>=0.2.0 <2.0.0'
  glob: ^1.1.0
  logging: ^0.11.0
  path: '>=1.2.0 <1.6.0'
  source_span: '>=1.0.0 <1.5.0'

dependency_overrides:
  analyzer: ^0.31.0
  barback: ^0.15.0
  build: ^0.12.0
  build_barback: ^0.5.0
  build_config: ^0.2.0
  build_runner: ^0.7.0
  code_builder: "^3.0.3"
  
dev_dependencies:
  test: any
  
transformers:
- $dart2js:
    commandLineOptions: [--show-package-warnings]
    # We do not want to compile anything in this package.
    $include: []

[eernstg]

Cool, thanks! That pubspec.yaml works here, too. I'm trying to see where the important differences are, but it seems to make no difference here: With pub upgrade I get 'Changed 1 dependency!' and that's just the version number of reflectable which changes between '2.0.0-dev.2.0' (from your pubspec.yaml) to '2.0.0-dev.3.0' (in my current pubspec.yaml). So none of the "real" dependencies change at all! Checking some more...

[MikeMitterer]

It's a Dart 2 thing. I switched to Dart 2.0.0-dev.40.0 - now it works. I thought there would be a nice migration path from 1.x to 2.x... However at least it works this way. Thanks.

[eernstg]

I hope there will be a nice migration path from 1.x to 2.x for everybody, but it's not a trivial exercise, and many issues are connected with "the context" in some sense. In any case, I hope it works better for you now, and thanks for the input!

[MikeMitterer]

My package DryIce (DI) https://github.com/MikeMitterer/dryice/tree/reflectable runs now with generated mirrors - which is pretty cool but now I need "reflectable" on pub because otherwise it's not possible to pub publish my package. Any new on this?

[eernstg]

DryIce .. runs now with generated mirrors ..

Cool! This PR just landed 2.0.0-dev.3.0, and I can then create and publish reflectable 2.0.0 .

[eernstg]

Reflectable 2.0.0 is available from pub.dartlang.org now. Note that the improved support for type arguments in https://github.com/dart-lang/reflectable/tree/support_static_type_arguments_nov17 has not yet been integrated into the master branch, but it will be soon.

[MikeMitterer]

!!!!! MANY THANKS! Great news. Wish you a good day.