Problem: Wrapping functions and forwarding arguments is noisy

[natebosch]

It's not possible to write a signature expressing that the return value is a function having the same argument count and types as one of the parameters. This leads to ugly blocks where we define the same function N times with varying argument counts and lots of duplication in generic types and forwarding the arguments.

For example expectAsync0 through expectAsync6: https://github.com/dart-lang/test/blob/ac21330338e0a7ad68fb19058bb92260706dd6cf/pkgs/test_api/lib/src/frontend/expect_async.dart#L295

This gets even more complex with things like optional arguments.

[kevmoo]

See also https://api.dartlang.org/stable/dart-async/Zone-class.html

We have three flavors of many methods callBack, binaryCallback, unaryCallback

[lrhn]

For what it's worth, there is no simple solution to this issue (which is probably why no-one has commented).

It's easy to make a function that returns something of the same type as the argument — if it returns the argument.
Creating a new value with a type provided as a type argument, or derived from the run-time type of an argument, feels very much like dependently typed code. Dart has no syntax for creating an object given a type that is only available at run-time.

Any solution would have to be very dynamic (going on reflective). For example, if we added a helper function like:

/// Creates a function of type `T`. 
///
/// When the created function is called, the [callback] function is called with  
/// a list of the positional arguments and a map of the named arguments that
/// were actually passed in the call. The return value of the [callback] function
/// becomes the return value of the returned function's invocation, 
/// and must have the correct type.
external T createFunctionProxy<T extends Function>(dynamic callback(
     List<dynamic> positionalArguments, 
     Map<Symbol, dynamic> namedArguments));

This would be just as dynamic as a noSuchMethod call, and the object would be created at run-time based on a run-time type.

A simpler solution, which might solve 95% of the issues, but not be as hard on the dynamics, would be a way to do pre- and post-actions on invocations.

/// Creates a new function with the same type as [function].
///
/// When the created function is called, the [before] function is called first,
/// before the parameters are passed to [function]. If [before] throws,
/// the [function] is not called.
/// When the call to [function] returns a value, the [after] function is called
/// before the result is returned to the caller.
T wrapFunction<T extends Function>(T function, {void before()?, void after()?});

This would be much more type-safe. (Whether the returned function will have type T or function.runtimeType would probably depend on what is easier to implement).

Anything that requires intercepting arguments is untypabel.
Anything that requires intercepting a return value is untypable (but if we had Foo Function as a supertype of all functions returning Foo, we might be able to do better).

[natebosch]

Creating a new value with a type provided as a type argument, or derived from the run-time type of an argument, feels very much like dependently typed code. Dart has no syntax for creating an object given a type that is only available at run-time.

Any solution would have to be very dynamic (going on reflective).

I hope there is something more static we could do here. As a strawman, imagine something like:

T Function(<*args*>) runWithOverrides<T, <*args*>>(T Function(<*args*>) callback) {
  return (<*args*>) {
    return runZoned(() => callback(<*args*>),
        zoneValue: {someZoneKey: someZoneValue});
  };
}

It shouldn't matter the runtime type of callback, so long as the returned function has the same static type, the usage should all work correctly.

Real motivating examples I've seen so far don't care about doing anything with the arguments except to forward them without modification. They may care about modifying the return value, but we already have ways to express that with generics.

One motivating example is linked above - it would be nice in package:pool if we could return a callback of any statically known arity and named arguments and when it's called, wait for a resource before invoking the wrapped callback.

dart-lang/tools#1552

This use case would likely need to change a FutureOr<T> Function(<*args*>) into a Future<T> Function(<*args*>), but doesn't care about the arguments.

Another is in package:test we have functions which wrap a callback in one that can record how many times it has been called. Today it uses Function.apply, we need a separate name for each number of arguments expectAsync0 on up, and we can't handle named arguments at all.

https://github.com/dart-lang/test/blob/f811df70f79a2398c9328e4faf6e5c1db86da81f/pkgs/test_api/lib/src/frontend/expect_async.dart#L193

I think in other issues I've seen @munificent mention how there are parallels between an argument list and some notion of a "Tuple" that has both named and positional values. I'm not sure if that could help here.

[rrousselGit]

Real motivating examples I've seen so far don't care about doing anything with the arguments except to forward them without modification.

I'd disagree.
A common use-case when dealing with higher-order functions is to want to inject a parameter that the function is known to have.

An example would be React/Redux, which injects the store's state through function currying.

I think the main difficulty is because Dart lacks both tuples and records.
With both of them, we could make Function.apply type-safe, where the positional parameters are mapped into a tuple, and the named parameters into a record.

So the prototype of Function.apply would be:

Result apply<Result, Positional, Named>(
  Result Function(...Positional, {...Named}) function,
  Positional positionalParameters,
  Named namedParameters,
);

[lrhn]

Abstracting over argument lists is an idea. Without destructuring, they'll necessarily be opaque, but if you can reapply them at a point where the static type is known to match, then it could work.

It's true that tuples can be generalized to contain named entries as well as indexed ones, and that parameter types are tuple types, formal parameters are tuple destructuring, and actual arguments are tuple values. If we allowed a general supertype over all tuples, then a function type would be ReturnType Function ArgumentType where ArgumentType extends Tuple. (But then, Object is just a 1-tuple so all values are tuples, and Tuple is the new top type).

That could work (and does work in functional languages like SML).

Without such an abstraction, we'd have to introduce a new kind of type variable to range over parameter signatures and allow captured argument lists to have that type, but otherwise be mostly opaque (because if not, we'll have introduce tuples anyway).

If we have tuple spreads, and partial tuple destructuring, then we can do all sorts of things with functions, including currying at arbitrary arities:

R Function(...P2) Function(...P1) curry<R, P1 extends Tuple, P2 extends Tuple>(
  R Function(...P1, ...P2) f) => (...P1 t1) => (...P2 t2) => f(...t1, ...t2);

(Here ...P1 is a tuple type spread, so (...P1 t1) is a single parameter capturing a tuple of arguments). Or something.

I have no idea how inference will work.

[TimWhiting]

Tuple types params and tuple type destructuring would be amazing! I can imaging that inference would be difficult though, either P1 or P2 would have to be specified in your example, unless you had a context type for the returned function.

If you could use . or something as a placeholder for inference you could do something like this.

final myFunc = curry<int,.>((int d, int c, String g) => '$d $g $c');
final newFunc = myFunc(10);
print(newFunc(20, 'hi')); //  outputs: 10 hi 20

[munificent]

If we allowed a general supertype over all tuples, then a function type would be ReturnType Function ArgumentType where ArgumentType extends Tuple. (But then, Object is just a 1-tuple so all values are tuples, and Tuple is the new top type).

That could work (and does work in functional languages like SML).

It could work, but it won't come together as naturally as it does in SML where you don't have subtyping to contend with and where the language was designed around tuples from day one.

Consider:

void oneOrTwo(Object a, [int? b]) {
  print('$a $b');
}

oneOrTwo(1, 2);

I presume that would print 1 2 and not (1, 2) null. So we don't implicitly tuple all argument lists because that would cause all hell to break loose.

I think that implies that we don't want to implicitly unpack argument lists either. So this:

var tuple = (1, 2);
oneOrTwo(tuple);
oneOrTwo((1, 2));

Would print (1, 2) null twice.

We could try to look at the type of the parameter list to decide whether to implicitly unpack:

takeTwo(int a, int b) => print('$a $b');
takeTuple((int a, int b)) => print('$a $b');

var tuple (1, 2);
takeTwo(tuple); // Prints "1 2".
takeTuple(tuple); // Prints "1 2".

But that can get weird:

takeTwoOrTuple<T>(T a, [int? b]) => '$a $b';

callThing<T>(T arg) => takeTwoOrTuple<T>(arg, 2);

callThing(1); // "1 2"?
callThing((1, 3)); // "(1, 3) 2" or "1 3"?

I don't think we want to specialize generics. The take away from all of this is that Dart probably does still need a fundamental notion of a parameter list for functions, which is distinct from tuple types. A 2-parameter function is a different type from a function that takes a single 2-tuple.

To go from one to the other, users will need to explicitly pack and unpack. The former is just a tuple expression, and the latter is some kind of argument list spread operator, which I think is tractable and really useful.

It's just not automatic or implicit. Just like how list flattening in list literals isn't automatic. You have to request it by using ....

[lrhn]

@munificent
This all depends very much on whether a tuple is an object at all.
If (Object, Object) (the two-element tuple type) is not a subtype of Object, then you'd simply not be able to do callThing((1, 3)).

The advantage of not making tuples into objects is that they won't have identity, and you can unbox and box as you see fit. You can have a parameter passing convention where a function returning a two-tuple puts both values on the stack.
And if we define that tuples are sequences of objects, there cannot be nested tuples. If you try to write (1, (2, 3)) it's completely equivalent to (1, 2, 3). You auto-spread tuples because you have to. A tuple is just a sequence of values. If a tuple contains another, it contains that sequence as a sub-sequence.

The disadvantage is that you can't abstract over all types any more. A function returning a two-tuple is not related to a function returning an Object, they're ABI incompatible. A type variable with a bound of Object? won't be able to hold a tuple type. (That's annoying). You can't have a List<(int, int)> unless you can abstract over all values, not jus tall objects.
(C# does this, where generic code is specialized for each value type it's applied to, like C++ template instantiation, and Object, the reference type, is just considered a single shape. We could too.)

So if you'd still have a super-type of all types, say Value, which includes Null (the zero-tuple), Object (the one-tuple) and Object×Object (two-tuples), etc, and we allow generic code specialization, we don't need to make tuples objects.

Speed vs. convenience. I'm fairly sure non-Object tuples can be made more speed efficient. Possibly at a space-cost if we need to specialized too much code for AoT.

(And then oneOrTwo(tuple) and onOrTwo((1, 2)) would both print "1 2", and the latter would give you an analyzer warning about unnecessary parentheses).

[munificent]

Ah, true, I suppose we could try to treat them like real primitive value types. It would feel very strange to do that when even numbers aren't primitive values in Dart, but I guess it's a thing we could do. My hunch is that the complexity that it would force onto users outweighs the performance benefits we'd get, but I could be wrong.

[TimWhiting]

I think dart could use a primitive value type, and I think tuples could be a good candidate. I don't see tuples as requiring the normal class things (inheritance, dynamic dispatch, mixins). For tuples as multiple return values, it seems like there is quite a benefit for not allocating objects especially if a large fraction of methods use tuples / multiple return values. A side effect of them not being objects is that they would not be eligible for representation as dynamic, which is probably fine since any library APIs should be strongly typed to begin with. @munificent I'm curious what sort of complexity it force onto users that you are talking about?

[lrhn]

@munificent Agree. If tuples are the only value-types we introduce, it probably won't be worth it.
If we also add general value types, ones you can declare yourself (basically the C# model) and tuples on top, and do proper optimizations/specialization for generics on value types, then it might be a value proposition by itself. And quite a different feature.

[munificent]

@munificent I'm curious what sort of complexity it force onto users that you are talking about?

It might mean things like:

• You can't instantiate a generic type with a tuple type as a type argument. Or, if you can, it might mean that our compilers have to do specialization of generic instantiations, which might increase code size.
• You can't override a method that returns Object with one that returns a tuple type. You can tighten return types in overrides for other types.
• You may not be able to pass a tuple to a function that takes Object, or, if you can, there would have to be some implicit boxing operation that might be user-visible.

Stuff like that.

[Levi-Lesches]

Tangentially, #1880 introduces a new syntax for the simple case of forwarding a function (wrapping is a little harder). It desugars an empty signature into the return type and parameter list of the specified function while avoiding tuples and other dynamic mechanisms.