the performance of async and await syntax are quite slow

[jumperchen]

Scenario

Sometimes we design an API to allow the returned value to be a Future or a Sync, which depends on its logic, so we will declare the returned value to be a Future in the function/method syntax.

For example,

foo() async {
   if (...) return 1; //sync value;
   else return new Future.value(2); // a future value
}

And then if the 'foo()' is called in a for loop for thousand� times, the async and await syntax will be the performance nightmare.

For example,

import 'dart:async';

main () async {

  // Async syntax
  var sw1 = new Stopwatch()..start();
  var count1 = 0;
  for (var i = 0; i < 10000; i++) {
    count1 += await foo();
  }
  print('Async and Await: ${sw1.elapsedMilliseconds}ms');

  // Future syntax
  var sw2 = new Stopwatch()..start();
  var count2 = 0;
  for (var i = 0; i < 10000; i++) {
    bar().then((_) {
      count2 += _;
      if (i == 9999)
        print('Future: ${sw2.elapsedMilliseconds}ms');
    });
  }

  // Sync syntax
  var sw3 = new Stopwatch()..start();
  var count3 = 0;
  for (var i = 0; i < 10000; i++) {
    count3 += baz();
  }
  print('Sync: ${sw3.elapsedMilliseconds}ms');

}

foo() async {
  return 1;
}

Future<int> bar() {
  return new Future.value(2);
}

baz() {
  return 3;
}

The result will be

Async and Await: 130ms
Sync: 0ms
Future: 79ms

An Idea

In our workaround, we implement a Sync class to extend the Future class, and by using the syntax of bar() to cheat the compiler, and it can speed up in the runtime.

For example,

Future<int>  bar() {
   if (isSync) return new Sync.value(2);
  else return new Future.value(2);
}

So it could be an idea to make this trick in Dart VM.

[jumperchen]

btw, the deferred load library can use the same trick.

For example, (in our workaround)

class FooFactory {
   static final SyncLoader syncLoader = Sync.load<bool, Foo>(wait: () => foo.loadLibrary(), then: (bool _) => new foo.Foo());
   static Future<Foo> lazyLoadFoo() {
     return syncLoader.future;
   }
 }

// usage   
FooFactory.lazyLoadFoo().then((f) => f.bar());

The result will be:
Only the first time is Future, and then all will be Sync later. (no more delay in runtime)

[lrhn]

If you just return a 1 from an async function, it will be wrapped up in a Future, so you do get the extra allocation and extra indirections in accessing the value. That's unavoidable. Asynchrony isn't free, and making a function that is called thousands of times asynchronous is going to cost. So, comparing it to a synchronous function isn't useful.

Now, comparing different asynchronous implementations is much more interesting :)

It's not surprising that latency suffers when you go to an async function. The difference between

foo() => new Future.value(42);

and

bar() async => 42;

is that the latter delays execution of the body until a later microtask. That latency is unavoidable as long as the language semantics requires that delay.

If you look at this dartpad, then dart2js doesn't really get delayed by await. It does get delayed by async - which is expected, but it's delayed more than what can be explained by a single scheduleMicrotask.

It can't explain all of the difference for the VM, though, it's await is significantly slower than the corresponding Future/then based approach.

[jumperchen]

Yes, I know that comparing Sync and Async is not fair, but in the real life it happens that we have to design an API to serve two different implementations in Sync and Async ways and only expose one public API.
It would be better if your team can come out with an official solution for us, rather than tricking with the language syntax issue to speed up. :)

[lrhn]

I agree that everything should be faster, and we are definitely not fast enough at async code yet.

That said, hiding a synchronous operation behind an async API will always have a cost. We should just try to ensure that the cost is as minimal as possible, but it won't go away.

[matanlurey]

FYI, AngularDart has requested a lint to remove all uses of async/await from our code-base:

#57714

[eernstg]

@jumperchen, here is an example showing how you can use FutureOr to abstract over the choice between synchronous and asynchronous computations:

import 'dart:async';

abstract class SyncOrAsync<X> {
  FutureOr<X> m(X x);
}

class Sync<X> implements SyncOrAsync<X> {
  X m(X x) => x;
}

class Async<X> implements SyncOrAsync<X> {
  Future<X> m(X x) async => x;
}

bool b = true; // Whatever.

main() async {
  int result;
  SyncOrAsync<int> o = b ? Sync<int>() : Async<int>();
  FutureOr<int> x = o.m(42);
  if (x is Future) {
    result = await x;
  } else {
    result = x;
  }
  print(result);
}

With FutureOr you can make the decision to return a future or a sync value dynamically, or you could make the choice as in the example (where it's dynamic when the static type of the receiver is SyncOrAsync, but statically known for receivers of type Sync and Async).

PS: It is not uncommon to recommend that FutureOr should only be used for callbacks (so myMethod(FutureOr<C> f()) is OK, but FutureOr<C> myMethod() is bad style), but the situation you're discussing here actually fits quite well. So if anyone makes that recommendation, please ignore it at least now and then. ;-)

[eernstg]

FYI, AngularDart has requested a lint to remove all uses of async/await from our code-base

@matanlurey, I'd prefer if AngularDart would put pressure on everyone (who can do something about it) to make async/await run fast. ;-)

[jumperchen]

@eernstg Yes, we had that already. (a fake future)

/**
 * A wrapper for calling the then() synchronously.
 * Support [catchError] and [whenComplete] as well.
 */
class Sync<T> implements Future<T> {
  final T raw;
  Future _error;
  dynamic _next;

  Sync.value([T v]) : this.raw = v is Sync ? v.raw : v;

  Future/*<S>*/ then /*<S>*/(onValue(T value), { Function onError }) {
    try {
      _next = Sync.resolve(onValue(raw));
      return _next;
    } catch (error, stack) {
      if (onError != null) {
        try {
          onError(error, stack);
        } catch (e, st) {
          _error = new Future.error(e, st);
          return _error as Future/*<S>*/;
        }
      }
      // simulate a future exception.
      _error = new Future.error(error, stack);
      return _error as Future/*<S>*/;
    }
  }
 static Future<T> resolve<T>(obj) {
    if (obj is Future || obj is Sync) {
      return obj;
    } else {
      return new Sync<T>.value(obj);
    }
  }
 ... // omitted

[eernstg]

@jumperchen, a fake future is in fact exactly what you don't need: With FutureOr there is no need to wrap the sync value, there is no need to have a class Sync<T> implements Future<T>, and hence you also avoid all the potential issues that you may encounter when you are using some instances whose type is Future<...>, but which are actually not futures.

[jumperchen]

@eernstg Yes, it could be but when we introduced the Sync was to get rid of the uses of async/await from our code-base, and at that time we already had 300K lines of code that we had to refactor with this issue, and we didn't want to use if/else to check whether the returned value is future or not for each API to deal with that, so we came up with this trick to save our times and it didn't break anything till now.

[matanlurey]

@eernstg:

FYI, AngularDart has requested a lint to remove all uses of async/await from our code-base

@matanlurey, I'd prefer if AngularDart would put pressure on everyone (who can do something about it) to make async/await run fast. ;-)

It's been done for years, but not been a priority for the language team (this isn't just a backend issue).

The fact that you can write:

func() async {
  return await 5;
}

... and this isn't either (a) a compile-time error or (b) just semantic sugar for:

func() {
  return new Future.value(5);
}

... makes it really difficult to optimize. And that's not even getting into the complex cases.

[eernstg]

@jumperchen, I acknowledge that 300K lines of asynchronous code is a very powerful argument in favor of introducing a "fake future" and then making all that code run synchronously, essentially without editing any of it. It's cool that you made that work!

However, I think you will still pay something for that approach (in addition to the obvious: that you're creating a lot of wrapper objects of type Sync<T> in order to pass around a T, and you'll run a lot of code in Sync.then and via a function literal, where you would otherwise have had a plain ;).

In particular, we can't let await be a no-op. Async/await driven code is simpler than concurrency, because the scheduler is non-preemptive, so there are lots of race conditions that just never occur; however, at times you will rely on the scheduler (that is, on being suspended) in order to ensure progress. For instance, the following code will loop forever if you remove await from main (or compile it into a no-op):

import 'dart:async';

var x = true;

f() async {
  x = false;
}

main() async {
  var fut = f();
  while (x) {
    await print(".");
  }
}

The point is that a compiler cannot be allowed to compile await e; as a synchronous operation even in the case where the value of e turns out to be a non-future. The language specification has the following about the evaluation of an await expression:

First, the expression e is evaluated to an object o.
Then, if o is not an instance of Future, then let f be the result of creating a
new object using the constructor Future.value() with o as its argument; otherwise
let f be o.
Next, the stream associated with the innermost enclosing asynchronous
for loop (17.6.3), if any, is paused. The current invocation of the function body
immediately enclosing a is suspended until after f completes.
At some time after f is completed, control returns to the current invocation.
...

So, given that your software needs to work on the actual values of the futures or Syncs that are passed around, you will have a bunch of await expressions, and they will have to suspend the current execution for a while (OK, I know that dart2js is trying to make it cheaper than it sounds, but you can only go so far).

This could be taken as a hint that there would be a real performance benefit in writing the code in a synchronous style, compared to the approach where synchronous execution is (mostly) achieved based on using Syncs rather than futures.

I can understand that you may still prefer to use the fake futures because they allow you to avoid touching your code.

But when this trade-off between explicitly synchronous computations (with some amount of code duplication) and fake-future based semi-synchronous computations (with complete reusability of the same code for asynchronous execution) can be made up front, it may be worthwhile to write some synchronous code as well.

[natebosch]

at times you will rely on the scheduler (that is, on being suspended) in order to ensure progress.

We certainly do have lots of code that relies on this today and it may be infeasible to refactor it. Is there a reason to think that code (ab)using await like this couldn't be refactored if we had enough motivation? This case could be:

await new Future.microtask():
print(".");

[matanlurey]

@eernstg:

at times you will rely on the scheduler (that is, on being suspended) in order to ensure progress.

Yes, that's been made an example many times. Honestly most users consider it a bug, however.

I think @mraleph once suggested we just allow await null for this reason.

I could even see it possible to just allow a "naked" await:

var i = 0;
while (true) {
  await;
  print('${i++}');
}

... instead of relying on await 5.

I found reading the V8 team's performance overview of their problems with async/await (and optimizing it based on the specification - which is similar to Dart) very interesting and relevant to the problems outlined here.

I recommend reading V8's performance overview on async/await (similar issues):

[matanlurey]

(Of course if we compile Future --> Promise and use JS's async/await in Dart2JS, /shrug)

[eernstg]

Is there a reason to think that code (ab)using await like this couldn't be refactored ..

@natebosch, surely we could lint away await e where the static type of e is not assignable to Future (so it's at least very unlikely to be a future at run time), and we could introduce await; as a readable way to request a suspension that isn't tied to any particular future completion. So I agree with you that those expressions could be refactored into something "nicer".

The other side of the same coin is that (I think) we should preserve the notion that "await will wait", because that makes it easier for developers to reason about the correctness of their programs (when they do need suspension-for-progress). If we weren't doing that then we could just allow the compiler to make await e the same thing as e in every situation where e is not a future. So these two things go together.

most users consider it a bug [to rely on suspension for progress]

@matanlurey, that's a delicate balance. It is basically the same thing as saying that it is a bug to have side-effects in your program (at least whenever it uses asynchrony). There is no end to the beautiful and well-founded arguments that you can make for that position, but those arguments have been put forward for half a century and we still have side-effects, so maybe the convenience/comprehensibility trade-off is non-trivial. I think the main point is that it is important to be able to understand software precisely, no matter which paradigm it's written in. In this case that would be a strong argument for linting await nonFuture; and adding support for await;. ;-)

[mkustermann]

The performance of async/await in the Dart VM has been improved by a factor of 10x. The VM specific issue for that was #37668, which has been closed.

It's unclear if this bug refers to async/await performance in general or in the Dart VM. Though I'll remove the area-vm label, since the VM improvements have landed.

If this was meant as a Dart VM bug only, I believe we can close it.

[franklinyow]

@mraleph @lrhn
Do you think this is addressed or some other work needed?

[lrhn]

Checking the benchmark I linked above, it seems that async/await is on par with directly written future code. Some of that is probably helped by us no longer being asynchronous on entry.
I don't have benchmarks for more complicated code, so I also don't have any known issues (other than that async* functions should also start synchronously).

[mkustermann]

Let's close this issue then. If there's other performance issues, please file a new bug.