Moving forwards with ES6 and ES7

[bjouhier]

ES6

ES6 is around the door as V8 will get upgraded in node 0.12 (assuming that 0.12 makes it some day soon 😄 ). There is also the io.js initiative which is pushing for a more aggressive adoption of new V8 versions.

This raises a challenge for streamline because ES6 introduces a lot of new syntax in the language. These new syntaxes do not pose any serious conceptual challenge to streamline but they pose a practical challenge: the narcissus compiler / decompiler does not support them and will need to be enhanced.

One way forwards would be to lock streamline at ES5 level and ban ES6 features from streamline code. I don't like the idea at all because ES6 brings a lot of improvements: let and const, destructuring assignments, fat arrow, etc.

The easiest way to move forwards is probably to enhance the narcissus compiler / decompiler layer.

Another way would be switch to a different compiler / decompiler which would support the ES6 syntax. It does not look like uglify does it yet but it is more active than narcissus which hasn't been touched for 3 years. The problem with a different compiler is that the transforms (the tricky code) will have to be adapted for a different AST. I'm not too thrilled.

If we keep narcissus, the move can be done progressively. We don't need to support all ES6 features right away, we can introduce them progressively.

ES6 also introduces promises as part of the standard runtime library. This is not a problem for streamline because interoperability with promises is already there.

ES7

ES7 is likely to bring another very interesting feature: async/await support.

If this happens my intent is to support async/await as another generation option, like fibers and generators. The transformed code will be very close to the original code (especially in fast mode) because streamline is conceptually aligned on async/await. In some sense streamline is ES5 + async/await with a different syntax.

Some people may actually use this option to upgrade their code to ES7 and then abandon streamline because ES7 will give them all they need without a preprocessor.

But there may still be a few reasons to stay with streamline. Syntax is one (async/await is bulkier and does not play as well with chaining). But performance is a more important one in my eyes. The --fibers mode is very fast when you have deep call stacks on top of low level async APIs. So it will be worth benching the await option against other options before deciding to leave streamline. Instrumentation (streamline-flamegraph) may be another reason to postpone the move.

CoffeeScript

ES6 is largely irrelevant for CScripters as they have been programming with the ES6 goodies forever anyway.

ES7 may be relevant though. CS may natively get an async/await at that time and it should not be too difficult to create a script to upgrade to that.

Conclusion

Some of you may find it disappointing that I envision a conversion to ES7 async/await in the long term. But we have to be realistic: there is no way streamline can gather as much momentum as an async/await which would be baked into the language. I think it is much wiser to accept ES7 as a future and prepare for that than to end up cornered into a proprietary approach.

In the short term streamline remains more relevant than ever, especially as it can provide a seamless upgrade path to ES7 async/await later. The preprocessor approach is an advantage because it does tie the code to a specific API and it makes it possible to take advantage of new language features (generators, async/await) as they get delivered. In some sense streamline will have given us a 4+ years head start on native async support in JS.

Thoughts?

[spollack]

Bruno, this all makes good sense to me. onward!

[aseemk]

Bruno, I forgot to respond to this initially, but it was a fantastic writeup. I agree with you. Thank you!

[bjouhier]

Thanks @spollack and @aseemk.

I'm having some second thoughts on staying with narcissus though. Esprima could be a better way forwards because it supports source maps and it would allow me to drastically simplify formatting and sourcemap handling.

[aseemk]

@bjouhier: another alternative for ES6, instead of re-implementing Streamline on a new parser, is to simply plug in an ES6→ES5 transpiler first. 6to5 is gaining a lot of traction (e.g. Facebook, Atom) and looks very well-implemented. It includes a programmatic API:

http://6to5.org/docs/usage/api/

[bjouhier]

@aseemk Great idea. That's a bit like running streamline behind the CoffeeScript transpiler. I had looked at 6to5 and seen that it uses esprima but I hadn't thought of using it directly. You just saved me many hours of work 👍

[aseemk]

Nice, glad! Hat-tip to @gasi for pointing me to 6to5. Agreed it's a nice analogy to CoffeeScript.

[spollack]

@jshepard on my team here just suggested the same idea earlier this week of using 6to5 ... very cool!

[bjouhier]

Great minds think alike 😄

[aseemk]

Just FYI 6to5 has been renamed to Babel.js: https://babeljs.io/

[bjouhier]

@aseemk @spollack
FYI, I've worked on it today. I decided to go ahead and replace narcissus by esprima, rather than preprocess with babel. It is probably a bit more work but it has several advantages:

• streamline will directly generate ES6 code. Should be better for performance and more future proof. For example, let declarations will be compiled natively instead of being emulated with an extra closure.
• esprima has good support for sourcemaps. So I can eliminate format.js and use escodegen instead.
• the AST is cleaner.

I've converted the callbacks transform today (the big one) and I'm at the point where the diskUsage._js example runs. Next step is to run all the unit tests and fix the glitches. Overall it has been a rather easy conversion because most of the rewriting rules are expressed with code templates.

[spollack]

Bruno, that's exciting, nice!