Anonymous methods

[eernstg]

In response to #259, this issue is a proposal for adding anonymous methods to Dart.

Edit: We could use the syntax e -> {...} in order to maintain syntactic similarity with the pipe operator proposed as #43, as mentioned here. See #265 for details. Later edit: Changed generic anonymous methods to be a separate extension of this feature, such that it is easier to see how it works without that part.

Examples

The basic syntax of an anonymous method is a term of the form .{ ... } which is added after an expression, and the dynamic semantics is that the value of that expression will be "the receiver" in the block, that is, we can access it using this, and methods can be called implicitly (so foo(42) means this.foo(42), unless there is a foo in the lexical scope).

To set the scene, take a look at the examples in #259, version 1 (original), 2 (unfolded), and 3 (folded). Here is a version of that same example that we could write using anonymous methods:

// Version 4, using anonymous methods.

void beginFrame(Duration timeStamp) {
  // ...
  ui.ParagraphBuilder(
    ui.ParagraphStyle(textDirection: ui.TextDirection.ltr),
  ).{
    addText('Hello, world.');
    build().{
      layout(ui.ParagraphConstraints(width: logicalSize.width));
      canvas.drawParagraph(this, ui.Offset(...));
    };
  };
  ui.SceneBuilder().{
    pushClipRect(physicalBounds);
    addPicture(ui.Offset.zero, picture);
    pop();
    ui.window.render(build());
  };
}

In this version the emphasis is on getting the complex entities (the objects that we are creating and initializing) established at first. With the given ParagraphBuilder in mind, we can read the body of the anonymous method (where we add some text to that paragraph builder and build it). We continue to work on the paragraph returned by build, doing a layout on it, and then using it (this) in the invocation of drawParagraph.

Similarly, the second statement gets hold of the scene builder first, and then works on it (with three method invocations on an implicit this, followed by one regular function call to render).

Compared to version 3, this version essentially turns the design inside out, because we put all the entities on the table before we use them, which allows drawParagraph and render to receive some much simpler arguments than in version 3.

Compared to version 2, this version allows for a simple use of statements, without the verbosity and redundancy of using names like paragraphBuilder many times. This means that we can use non-trivial control flow if needed:

ui.SceneBuilder().{
  pushClipRect(physicalBounds);
  for (var picture in pictures) {
    addPicture(randomOffset(), picture);
  }
  pop();
  ui.window.render(build());
};

We can of course also return something from an anonymous method, and we can use that to get to a sequential form rather than the nested one shown above:

ui.ParagraphBuilder(
  ui.ParagraphStyle(textDirection: ui.TextDirection.ltr),
).{
  addText('Hello, world.');
  return build();
}.{
  layout(ui.ParagraphConstraints(width: logicalSize.width));
  canvas.drawParagraph(this, ui.Offset(...));
};

We can choose to give the target object a different name than this:

main() {
  "Hello".(that){ print(that.length); };
}

We can use a declared type for the target object in order to get access to it under a specific type:

main() {
  List<num> xs = <int>[];
  xs.(Iterable<int> this){ add(3); }; // Dynamic check.
}

Note that this implies that there is an implicit check that xs is Iterable<int> (which may be turned into a compile-time error, e.g., by using the command line option --no-implicit-casts). Also, the addition of 3 to xs is subject to a dynamic type check (because xs could be a List<Null>).

As a possible extension of this feature, we can provide access to the actual type arguments of the target object at the specified type:

main() {
  List<num> xs = <int>[];
  xs.(Iterable<var X> this){ // Statically safe.
    print(X); // 'int', not 'num'.
    num x = 3; // Note that `xs.add(x)` requires a dynamic check.
    if (x is X) add(x); // Statically safe, no dynamic checks.
  };
}

We are using type patterns, #170, in order to specify that X must be bound to the actual value of the corresponding type argument for xs. We also specify that X must be a subtype of num, such that the body of the anonymous function can be checked under that assumption, and this is a statically safe requirement because the static type of xs is List<num>.

Finally, if the syntax works out, we could extend this feature to provide a new kind of function literals. Such a function literal takes exactly one argument which is named this, and the body supports implicit member access to this, just like an instance method and an anonymous method:
use an anonymous method as an alternate syntax for a function literal that accepts a single argument (and, of course, treats that argument as this in its body):

main() {
  List<String> xs = ['one', 'two'];
  xs.forEach(.{ print(substring(1)); }); // 'ne', 'wo'
}

Proposal

This is a draft feature specification for anonymous methods in Dart.

Syntax

The grammar is modified as follows:

<cascadeSection> ::= // Modified
    '..'
    <cascadeHeadSelector>
    <cascadeTailSelector>*
    (<assignmentOperator> <expressionWithoutCascade>)?

<cascadeHeadSelector> ::= // New alternative added
    <cascadeSelector> <argumentPart>*
  | <anonymousMethod>

<cascadeTailSelector> ::= // New alternative added
    <assignableSelector> <argumentPart>*
  | <anonymousMethodSelector>

<selector> ::= // New alternative added
    <assignableSelector>
  | <anonymousMethodSelector>
  | <argumentPart>

<anonymousMethodSelector> ::= // New
    '.' <anonymousMethod>
  | '?.' <anonymousMethod>

<anonymousMethod> ::= // New
    <typeParameters>? ('(' <normalFormalParameter> ')')? <block>

<simpleFormalParameter> ::= // Modified
    <declaredIdentifierOrThis>
  | 'covariant'? <identifierOrThis>

<identifierOrThis> ::= <identifier> | 'this'

<declaredIdentifierOrThis> ::=
    'covariant'? <finalConstVarOrType> <identifierOrThis>

Static Analysis

It is a compile-time error if a formal parameter is named this, unless it is a parameter of an anonymous method or a function literal.

An anonymous method invocation of the form e.{ <statements> } or e..{ <statements> } is treated as e.(T this){ <statements> } respectively e..(T this){ <statements> }, where T is the static type of e.

An anonymous method invocation of the form e?.{ <statements> } is treated as e?.(T this){ <statements> } where T is the static type of e (with non-null types, T is the non-null type corresponding to the static type of e).

The rules specifying that an expression e starting with an identifier id is treated as this.e in the case where id is not declared in the enclosing scopes remain unchanged.

However, with anonymous methods there will be more situations where this can be in scope, and when an anonymous method is nested inside an instance method, the type of this will be the type of the receiver of the anonymous method invocation, not the enclosing class.

In an anonymous method invocation of the form e.(T id){ <statements> } or e..(T id){ <statements> }, it is a compile-time error unless the static type of e is assignable to T. (Note that id can be this.) Moreover, it is a compile-time error if T is dynamic.

It is a compile-time error if a statement of the form return e; occurs such that the immediately enclosing function is an anonymous function of the form e..(T id){ <statements> }. This is because the returned value would be ignored, so the return statement would be misleading.

The static type of an anonymous method invocation of the form e.(T id){ <statements> } is the return type of the function literal (T id){ <statements> }. The static type of e?.(T id){ <statements> } is
S?, where S is the return type of the function literal (T id){ <statements> }.

The static type of an anonymous method invocation of the form e..(T id){ <statements> } is the static type of e.

Dynamic Semantics

Evaluation of an expression of the form e.(T id){ <statements> } proceeds as follows: e is evaluated to an object o. It is a dynamic error unless the dynamic type of o is a subtype of T. Otherwise, (T id){ <statements> }(o) is evaluated to an object o2, and o2 is the result of the evaluation.

Evaluation of an expression of the form e?.(T id){ <statements> } proceeds as follows: e is evaluated to an object o. If o is the null object then the null object is the result of the evaluation, otherwise it is a dynamic error unless the dynamic type of o is a subtype of T. Otherwise, (T id){ <statements> }(o) is evaluated to an object o2, and o2 is the result of the evaluation.

Evaluation of an expression of the form e..(T id){ <statements> } proceeds as follows: e is evaluated to an object o. It is a dynamic error unless the dynamic type of o is a subtype of T. Otherwise, (T id){ <statements> }(o) is evaluated to an object o2, and o is the result of the evaluation.

Discussion

As mentioned up front, we could use -> rather than . to separate the receiver from an associated anonymous method, which would make this construct similar to an application of the pipe operator (#43).

This might be slightly confusing for the conditional variant (where we would use ? -> rather than ?.) and the cascaded variant (where we might use --> rather than .., and ? --> rather than ?.., if we add null-aware cascades).

It might be useful to have an 'else' block for a conditional anonymous method (which would simply amount to adding something like ('else' <block>)? at the end of the <anonymousMethodSelector> rule), but there is a syntactic conflict here: If we use else then we will have the combination of punctuation and keywords (e?.{ ... } else { ... } ). Alternatively, if we use : then we get a consistent use of punctuation, and we get something which is similar to the conditional operator (b ? e1 : e2), but it will surely surprise some readers to have : as a larger-scale separator (in e?.{ ... } : { ... }, the two blocks may be large).

Note that all other null-aware constructs could also have an 'else' part, specifying what to do in the case where the receiver turns out to be null, such that the expression as a whole does not have to evaluate to the null object.

Note that we could easily omit support for this parameters in function literals, or we could extend the support to even more kinds of functions.

We insist that the receiver type for an anonymous method cannot be dynamic. This is because it would be impractical to let every expression starting with an identifier denote a member access on that receiver:

main() {
  dynamic d = 42;
  d.{ print(this); }; // Oops, `42` does not have a `print` method!
}

However, another trade-off which could be considered is to allow a receiver of type dynamic, but give it the type Object in the body.

[kasperpeulen]

Brilliant, applause from me 👏 .

You could also combine this with the cascade operator and have:

object
  ..{ print(this) }
  .methodOnObject()

I wonder if one liner syntax is possible:

string
  .trimLeft()
  .((it) => it + "!")

[lrhn]

This is in some ways similar to #43 (pipe-operator).

You take the result of an expression and use it as the first argument to a following function, e1 -> function(). Here you can omit the arguments of a function and have it be an implicit (this), which is clever, but the cases where you do specify the argument, it's less powerful than #43, which can accept existing functions and extra arguments to those (foo->print() vs foo.{print(this)}).

If we extend #43 with e1 -> { block where this is bound to value of e1 }, then I think it would be the best of two worlds.

The feature makes the block work as a function body, so 42.{ return this; } evaluates to 42.
Another option is to make it an inline block in the surrounding function where this is bound to something else. Then 42.{return this;} would return 42 from the surrounding function.
In that case e1.{...} is a statement, not an expressions, but we also preserve the property that a separate function body is recognizable by its preceding parameters.

[eernstg]

@tatumizer wrote:

For Iterable - I don't understand it.

The point is that e.{ <statements> } is an abbreviation of e.(T this){ <statements> } where T is the statically known type of e. The extension to allow type arguments is needed in order to make such a method similar to an instance method (because instance methods have access to the type arguments of the class), but we would need to specify the names of those type arguments:

class A<X, Y> {}
class B<Z, W> extends A<W, Z> {} // Note the swap!
A<int, String> foo() => B();

main() {
  var x = foo();
  x.{ ... }; // How would we access Z, W, X, Y here?
}

It would surely be too hard to read (and error prone) if we were to say that X and Y are in scope in the body of the anonymous method, just because those are the declared names of the type parameters in the class A.

It would also be quite confusing if we just specify that we wish to access the type variables under the names X and Y, without saying that it's the type variables for A, not B—we would then be able to swap the binding (such that X is the second type parameter of A and Y is the first one) if we change the return type of foo to B<String, int> (noting that B<String, int> <: A<int, String>, and the body already has this type).

So if we want to allow the anonymous method to have access to the actual type arguments of the receiver it must be at a specified type, and those type parameters will have to be declared explicitly. xs.<X>(Iterable<X> this).{ ... } does just that.

For "xs.forEach" - "this" is too long and confusing IMO.

That's a trade-off, of course. My intention was to insist on the association between implicit member access and the reserved word this, such that developers can rely on foo(42) to mean foo(42) (because foo is declared in an enclosing scope), otherwise it means this.foo(42) (because foo is not in the lexical scope, but it is a member of the type of this), and otherwise it's an error.

It is true that we may now need to think a little more in order to understand what this means, but we'd always be able to find it by means of a scan of enclosing {} blocks: If we're inside an anonymous method then this is the receiver for that, otherwise this is the current instance of the enclosing class.

We could use a different name like it, and we could require that it is mentioned every time, but this would detract from the conciseness of the mechanism:

// Version 5, using an explicit `it` to denote the receiver.

void beginFrame(Duration timeStamp) {
  // ...
  ui.ParagraphBuilder(
    ui.ParagraphStyle(textDirection: ui.TextDirection.ltr),
  ).{
    it.addText('Hello, world.');
    it.build().{
      it.layout(ui.ParagraphConstraints(width: logicalSize.width));
      canvas.drawParagraph(it, ui.Offset(...));
    };
  };
  ui.SceneBuilder().{
    it.pushClipRect(physicalBounds);
    it.addPicture(ui.Offset.zero, picture);
    it.pop();
    ui.window.render(it.build());
  };
}

The use of forEach is different, by the way, because it uses the anonymous method syntax to express a function literal: xs.forEach(.{ print(substring(1)); }) is the same thing as xs.forEach((x) { print(x.substring(1)); }).

The point is that we could choose to introduce this kind of function literal, based on the similarity to anonymous methods, and they would of course be all about the implicit use of this (otherwise we'd just consider various other abbreviated forms of function literal, like _.length as an abbreviation of (x) => x.length).

[eernstg]

@kasperpeulen wrote:

You could also combine this with the cascade operator

That's the intention! (I'm going to work on the 'proposal' part later today), and also e?.{ ... }.

one liner syntax

That's rather tricky, because we'd need to disambiguate the end of the function body:

myString.(it) => it + "!"

Does that parse as (myString.(it) => it) + "!" or as myString.(it) => (it + "!")?

[eernstg]

@tatumizer wrote:

we can now write ..

Adding in the semicolon that we will presumably need (until we settle on something in the area of optional semicolons):

var x = 0.{ // receiver can be anything
   // long computation
  return result;
};

Right, this means that we could use anonymous methods to express "block expressions". I'm not sure it would be recommended in the style guide, but it's an interesting corner that I hadn't thought of. ;-)

[eernstg]

@lrhn wrote:

.. extend #43 with e1 -> { block where this is bound to value of e1 }

That's definitely an interesting idea! The pipe operator will surely call for the addition of multiple returns (such that we can do f -> g also when g takes more than one argument), and that might be handled by something like this:

(e1, e2) -> (T x, S this){ ... }

which would allow us to declare names for all the incoming objects (and, presumably, we would still be able to use this for one of them, and get the implicit access to members).

We should be able to do this later even in the case where we set out with a more restricted model (exactly one incoming object).

[yjbanov]

This looks similar to the argument blocks idea, but with a focus on mutable classes. It relies on the property of the implicit this supporting side effects. While such code exists in Flutter and some other areas (e.g. builder in package:built_value), I think it would be 10x more interesting if it could work with immutable objects. For example, can this be made applicable in build() functions?

I think we should be very careful about introducing conveniences for mutables or other code that deals with side-effects. We do not want to discourage immutability even further.

[eernstg]

@tatumizer wrote:

we have nested "this" which is very confusing (javascript is famous for that).

In JavaScript there are a bunch of reasons why the meaning of this is confusing (e.g., see this discussion on StackOverflow).

In Dart we have a well-defined notion of lexical scoping, and the meaning of this is simply the current instance of the enclosing class. If we add support for this with implicit member access in anonymous methods then we would still have a lexical criterion: Search up through enclosing {} blocks and this denotes the innermost one that is a class or an anonymous method.

So, granted, it does make this a bit more involved, but I don't think it's a huge cost.

[eernstg]

@tatumizer wrote:

Everything looks consistent here

True, it does add up quite nicely!

[lrhn]

The question here is what the goal of the operation is.

If it is to make implicit this-calls possible, as a way to reduce the syntactic overhead of introducing a variable and writing x. before every invocation, then nothing short of re-binding this will solve it.
I'm not sure that's worth it.

If the goal is to be able to introduce something that otherwise works as a method on an object, locally, then I think it's a red herring. A method added on the side can't do anything that a simple function can't, except for the this. shorthand. Looking like a method isn't itself useful.

Having a number of statements in an expression context is also not new, an immediately applied function literal will do that for you.

What would be new and useful is a way to introduce a statement in expression context without entering a new function body, allowing control flow in and out of the expression. That's probably also a tall order implementation-wise (I'm sure the VM assumes that an expression cannot break, continue, return or yield, and allowing that likely won't be worth the effort).

So, all in all, the only thing really useful I see here is a way to locally rebind this, something like with (something) { assert(this == something); } (to use JS syntax). Making it a selector might be useful, but I fear that having a block embedded in another expression which is really a function body, but which doesn't look like it (no parameters), can hurt readability.

[eernstg]

@yjbanov wrote:

This looks similar to the argument blocks idea

I proposed similar things already a couple of years ago and it's definitely possible that these ideas blend in with each other over time. I actually thought that the named argument blocks would be intended to allow for something like passing named arguments (using a sequence of assignment-ish constructs) rather than being regular blocks (containing full-fledged statements), but it's certainly not hard to see the similarities.

We do not want to discourage immutability even further

I want to encourage immutability, and support developers in knowing and enforcing it. We would presumably do this by adding support for value classes, and I hope we'll get around to adding that soon.

However, I don't think my proposal here is particularly strongly tied to mutable state.

It is true that it is concerned with the provision of a scope where statements can be executed, and statements are frequently used to specify computations with side effects. There's nothing that prevents you from using an anonymous method with a receiver which is immutable. It just means that the available methods on that object won't mutate it. For instance, you might want to use the methods in order to extract various parts of the immutable state of the receiver and passing them on to functions.

can this be made applicable in build() functions?

I would expect this to be an obvious use case for anonymous methods (and they would presumably fit in quite well with the ...Builder objects being used in there, because they will build an object in an imperative style, i.e., based on several method invocations on the builder).

---

But I guess the main point you are making is that we should have at least similarly good support for the creation of immutable objects as the support that we have for building an object graph of mutable objects (using anonymous methods, or whatever we have).

I think this would mainly be concerned with the ability to pass complex arguments in an argument list that would be used to build a new immutable object, and the ability for developers to read that kind of code.

We do have Flutter's widespread usage of named arguments and a trailing comma based formatting for that, but I do recognize that we might want improvements here as well.

[eernstg]

@lrhn wrote:

The question here is what the goal of the operation is.

If e.{ ... } is concise and readable because it combines implicit member access with full-fledged statements in an expression context then I think it's OK to consider the mechanism based on the combination of properties that it actually offers, and I'm not really convinced that we would need to designate a single property as the "goal" of this mechanism.

Of course, we could use (T this){ ... }(e) and then conclude that e.{ ... } is unnecessary, or we could use (T it){ ... }(e) and edit the body to mention it whenever needed, but you could use a similar type of argument to get rid of lots and lots of other language constructs as well.

What would be new and useful is a way to introduce a statement
in expression context without entering a new function body,

That would indeed be useful, because it would allow us to use return to return from the enclosing function/method rather than from the anonymous method, and use await relative to the enclosing function/method, etc.

But, as you mention, that might also have some more complex implications for the implementation. The anonymous methods that I proposed here should be rather easy to implement (the feature spec already hints that they could be mostly syntactic sugar).

[lrhn]

Indeed my rule for whether a specialized construct is worth it when a general construct already enables the functionality is that:

• The specialized functionality is signficantly "better" (e.g., more readable, concise, and less error prone).
• The situation where the specialized functionality applies occurs "often enough".
• It's a good language construct that fits in the language.

This comparison is also applied against potential features, not just existing ones. If we can add another good feature which is more general, and makes the improvement of this feature insignfiicant, then we should probably do that instead.

For the foo.{ this bound here } construct, my main objections are that I don't think it happens often enough (mainly because we already have cascades which handles the simple cases) and that it's not readable enough (because it's not clear that this is a separate function body with its own returns).

Going to foo.<T>(T x) { .... } makes it a non-significant improvement over <T>(T x) {...}(foo). The chaining is nice, but the pipe operator is more generally useful and would allow foo->(<T>(T x){...})() if you really need it.

Making the nested block look like a method does not give any significant advantage, it's the localized change of this which is the main feature, because it allows implicit this calls to be shorter than explicit it.something() calls. All other differences are not significant improvements, or are better handled by a pipe operator (and so we should introduce that instead, I wouldn't want to have both).

Since the "method" has access to local variables in a different function, it's not really a method anyway, it's a different syntax for a function literal (including the inferral of return type) which is immediately invoked, and with the option of nameing the parameter this

Another option is to allow any function to denote one of its positional parameters as this, not just this construct. That would be more generally useful, a static helper function could meaningfully treat one argument as the main operand.
Then, with a pipe operator, you can do foo->(this){addSomething();addSomethingElse();}(). Not as clean, but still usable, and again the improvement of the proposed feature over this is not significant enough.

[eernstg]

I've explored the syntactic implications of allowing Kotlin-ish abbreviated function literals (like { print(it); } where we get an implicitly declared parameter list of the form (it), and type inference may provide the parameter type): #265. Looks like we can easily do that.