Add support for hybrid VM/browser tests.

CHANGELOG.md

@@ -1,3 +1,8 @@
+## 0.12.18
+
+* Add the `spawnHybridUri()` and `spawnHybridCode()` functions, which allow
+  browser tests to run code on the VM.
+
 ## 0.12.17+3
 
 * Internal changes only.

README.md

@@ -14,6 +14,7 @@
   * [Whole-Package Configuration](#whole-package-configuration)
 * [Tagging Tests](#tagging-tests)
 * [Debugging](#debugging)
+* [Browser/VM Hybrid Tests](#browser-vm-hybrid-tests)
 * [Testing with `barback`](#testing-with-barback)
 * [Further Reading](#further-reading)
 
@@ -126,7 +127,7 @@ void main() {
 
 A single test file can be run just using `pub run test path/to/test.dart`.
 
-![Single file being run via pub run"](https://raw.githubusercontent.com/dart-lang/test/master/image/test1.gif)
+![Single file being run via "pub run"](https://raw.githubusercontent.com/dart-lang/test/master/image/test1.gif)
 
 Many tests can be run at a time using `pub run test path/to/dir`.
 
@@ -605,6 +606,80 @@ can see and interact with any HTML it renders. Note that the Dart animation may
 still be visible behind the iframe; to hide it, just add a `background-color` to
 the page's HTML.
 
+## Browser/VM Hybrid Tests
+
+Code that's written for the browser often needs to talk to some kind of server.
+Maybe you're testing the HTML served by your app, or maybe you're writing a
+library that communicates over WebSockets. We call tests that run code on both
+the browser and the VM **hybrid tests**.
+
+Hybrid tests use one of two functions: [`spawnHybridCode()`][spawnHybridCode] and
+[`spawnHybridUri()`][spawnHybridUri]. Both of these spawn Dart VM
+[isolates][dart:isolate] that can import `dart:io` and other VM-only libraries.
+The only difference is where the code from the isolate comes from:
+`spawnHybridCode()` takes a chunk of actual Dart code, whereas
+`spawnHybridUri()` takes a URL. They both return a
+[`StreamChannel`][StreamChannel] that communicates with the hybrid isolate. For
+example:
+
+[spawnHybridCode]: http://www.dartdocs.org/documentation/test/latest/index.html#test/test@id_spawnHybridCode
+[spawnHybridUri]: http://www.dartdocs.org/documentation/test/latest/index.html#test/test@id_spawnHybridUri
+[dart:isolate]: https://api.dartlang.org/stable/latest/dart-isolate/dart-isolate-library.html
+[StreamChannel]: https://pub.dartlang.org/packages/stream_channel
+
+```dart
+// ## test/web_socket_server.dart
+
+// The library loaded by spawnHybridUri() can import any packages that your
+// package depends on, including those that only work on the VM.
+import "package:shelf/shelf_io.dart" as io;
+import "package:shelf_web_socket/shelf_web_socket.dart";
+import "package:stream_channel/stream_channel.dart";
+
+// Once the hybrid isolate starts, it will call the special function
+// hybridMain() with a StreamChannel that's connected to the channel
+// returned spawnHybridCode().
+hybridMain(StreamChannel channel) async {
+  // Start a WebSocket server that just sends "hello!" to its clients.
+  var server = await io.serve(webSocketHandler((webSocket) {
+    webSocket.sink.add("hello!");
+  }), 'localhost', 0);
+
+  // Send the port number of the WebSocket server to the browser test, so
+  // it knows what to connect to.
+  channel.sink.add(server.port);
+}
+
+
+// ## test/web_socket_test.dart
+
+@TestOn("browser")
+
+import "dart:html";
+
+import "package:test/test.dart";
+
+void main() {
+  test("connects to a server-side WebSocket", () async {
+    // Each spawnHybrid function returns a StreamChannel that communicates with
+    // the hybrid isolate. You can close this channel to kill the isolate.
+    var channel = spawnHybridUri("web_socket_server.dart");
+
+    // Get the port for the WebSocket server from the hybrid isolate.
+    var port = await channel.stream.first;
+
+    var socket = new WebSocket('ws://localhost:$port');
+    var message = await socket.onMessage.first;
+    expect(message.data, equals("hello!"));
+  });
+}
+```
+
+![A diagram showing a test in a browser communicating with a Dart VM isolate outside the browser.](https://raw.githubusercontent.com/dart-lang/test/master/image/hybrid.png)
+
+**Note**: If you write hybrid tests, be sure to add a dependency on the
+`stream_channel` package, since you're using its API!
+
 ## Testing With `barback`
 
 Packages using the `barback` transformer system may need to test code that's

lib/src/frontend/spawn_hybrid.dart

@@ -0,0 +1,172 @@
+// Copyright (c) 2016, the Dart project authors.  Please see the AUTHORS file
+// for details. All rights reserved. Use of this source code is governed by a
+// BSD-style license that can be found in the LICENSE file.
+
+import 'dart:async';
+import 'dart:convert';
+
+import 'package:async/async.dart';
+import 'package:path/path.dart' as p;
+import 'package:stream_channel/stream_channel.dart';
+
+import '../backend/invoker.dart';
+import '../util/remote_exception.dart';
+import '../utils.dart';
+
+/// A transformer that handles messages from the spawned isolate and ensures
+/// that messages sent to it are JSON-encodable.
+///
+/// The spawned isolate sends three kinds of messages. Data messages are emitted
+/// as data events, error messages are emitted as error events, and print
+/// messages are printed using `print()`.
+final _transformer = new StreamChannelTransformer(
+    new StreamTransformer.fromHandlers(handleData: (message, sink) {
+      switch (message["type"]) {
+        case "data":
+          sink.add(message["data"]);
+          break;
+
+        case "print":
+          print(message["line"]);
+          break;
+
+        case "error":
+          var error = RemoteException.deserialize(message["error"]);
+          sink.addError(error.error, error.stackTrace);
+          break;
+      }
+    }), new StreamSinkTransformer.fromHandlers(handleData: (message, sink) {
+      // This is called synchronously from the user's `Sink.add()` call, so if
+      // [ensureJsonEncodable] throws here they'll get a helpful stack trace.
+      ensureJsonEncodable(message);
+      sink.add(message);
+    }));
+
+/// Spawns a VM isolate for the given [uri], which may be a [Uri] or a [String].
+///
+/// This allows browser tests to spawn servers with which they can communicate
+/// to test client/server interactions. It can also be used by VM tests to
+/// easily spawn an isolate.
+///
+/// The Dart file at [uri] must define a top-level `hybridMain()` function that
+/// takes a `StreamChannel` argument and, optionally, an `Object` argument to
+/// which [message] will be passed. Note that [message] must be JSON-encodable.
+/// For example:
+///
+/// ```dart
+/// import "package:stream_channel/stream_channel.dart";
+///
+/// hybridMain(StreamChannel channel, Object message) {
+///   // ...
+/// }
+/// ```
+///
+/// If [uri] is relative, it will be interpreted relative to the `file:` URL for
+/// the test suite being executed. If it's a `package:` URL, it will be resolved
+/// using the current package's dependency constellation.
+///
+/// Returns a [StreamChannel] that's connected to the channel passed to
+/// `hybridMain()`. Only JSON-encodable objects may be sent through this
+/// channel. If the channel is closed, the hybrid isolate is killed. If the
+/// isolate is killed, the channel's stream will emit a "done" event.
+///
+/// Any unhandled errors loading or running the hybrid isolate will be emitted
+/// as errors over the channel's stream. Any calls to `print()` in the hybrid
+/// isolate will be printed as though they came from the test that created the
+/// isolate.
+///
+/// Code in the hybrid isolate is not considered to be running in a test
+/// context, so it can't access test functions like `expect()` and
+/// `expectAsync()`.
+///
+/// **Note**: If you use this API, be sure to add a dependency on the
+/// **`stream_channel` package, since you're using its API as well!
+StreamChannel spawnHybridUri(uri, {Object message}) {
+  Uri parsedUrl;
+  if (uri is Uri) {
+    parsedUrl = uri;
+  } else if (uri is String) {
+    parsedUrl = Uri.parse(uri);
+  } else {
+    throw new ArgumentError.value(uri, "uri", "must be a Uri or a String.");
+  }
+
+  String absoluteUri;
+  if (parsedUrl.scheme.isEmpty) {
+    var suitePath = Invoker.current.liveTest.suite.path;
+    absoluteUri = p.url.join(
+        p.url.dirname(p.toUri(p.absolute(suitePath)).toString()),
+        parsedUrl.toString());
+  } else {
+    absoluteUri = uri.toString();
+  }
+
+  return _spawn(absoluteUri, message);
+}
+
+/// Spawns a VM isolate that runs the given [dartCode], which is loaded as the
+/// contents of a Dart library.
+///
+/// This allows browser tests to spawn servers with which they can communicate
+/// to test client/server interactions. It can also be used by VM tests to
+/// easily spawn an isolate.
+///
+/// The [dartCode] must define a top-level `hybridMain()` function that takes a
+/// `StreamChannel` argument and, optionally, an `Object` argument to which
+/// [message] will be passed. Note that [message] must be JSON-encodable. For
+/// example:
+///
+/// ```dart
+/// import "package:stream_channel/stream_channel.dart";
+///
+/// hybridMain(StreamChannel channel, Object message) {
+///   // ...
+/// }
+/// ```
+///
+/// Returns a [StreamChannel] that's connected to the channel passed to
+/// `hybridMain()`. Only JSON-encodable objects may be sent through this
+/// channel. If the channel is closed, the hybrid isolate is killed. If the
+/// isolate is killed, the channel's stream will emit a "done" event.
+///
+/// Any unhandled errors loading or running the hybrid isolate will be emitted
+/// as errors over the channel's stream. Any calls to `print()` in the hybrid
+/// isolate will be printed as though they came from the test that created the
+/// isolate.
+///
+/// Code in the hybrid isolate is not considered to be running in a test
+/// context, so it can't access test functions like `expect()` and
+/// `expectAsync()`.
+///
+/// **Note**: If you use this API, be sure to add a dependency on the
+/// **`stream_channel` package, since you're using its API as well!
+StreamChannel spawnHybridCode(String dartCode, {Object message}) {
+  var uri = new Uri.dataFromString(dartCode,
+      encoding: UTF8, mimeType: 'application/dart');
+  return _spawn(uri.toString(), message);
+}
+
+/// Like [spawnHybridUri], but doesn't take [Uri] objects and doesn't handle
+/// relative URLs.
+StreamChannel _spawn(String uri, Object message) {
+  var channel = Zone.current[#test.runner.test_channel] as MultiChannel;
+  if (channel == null) {
+    // TODO(nweiz): Link to an issue tracking support when running the test file
+    // directly.
+    throw new UnsupportedError(
+        "Can't connect to the test runner.\n"
+        'spawnHybridUri() is currently only supported within "pub run test".');
+  }
+
+  ensureJsonEncodable(message);
+
+  var isolateChannel = channel.virtualChannel();
+  channel.sink.add({
+    "type": "spawn-hybrid-uri",
+    "url": uri,
+    "message": message,
+    "channel": isolateChannel.id
+  });
+
+  return isolateChannel.transform(_transformer);
+}

lib/src/runner/hybrid_listener.dart

@@ -0,0 +1,88 @@
+// Copyright (c) 2016, the Dart project authors.  Please see the AUTHORS file
+// for details. All rights reserved. Use of this source code is governed by a
+// BSD-style license that can be found in the LICENSE file.
+
+import "dart:async";
+import "dart:isolate";
+
+import "package:async/async.dart";
+import "package:stack_trace/stack_trace.dart";
+import "package:stream_channel/stream_channel.dart";
+
+import "../util/remote_exception.dart";
+import "../utils.dart";
+
+/// A sink transformer that wraps data and error events so that errors can be
+/// decoded after being JSON-serialized.
+final _transformer = new StreamSinkTransformer.fromHandlers(
+    handleData: (data, sink) {
+      ensureJsonEncodable(data);
+      sink.add({"type": "data", "data": data});
+    }, handleError: (error, stackTrace, sink) {
+      sink.add({
+        "type": "error",
+        "error": RemoteException.serialize(error, stackTrace)
+      });
+    });
+
+/// Runs the body of a hybrid isolate and communicates its messages, errors, and
+/// prints to the main isolate.
+///
+/// The [getMain] function returns the `hybridMain()` method. It's wrapped in a
+/// closure so that, if the method undefined, we can catch the error and notify
+/// the caller of it.
+///
+/// The [data] argument contains two values: a [SendPort] that communicates with
+/// the main isolate, and a message to pass to `hybridMain()`.
+void listen(AsyncFunction getMain(), List data) {
+  var channel = new IsolateChannel.connectSend(data.first as SendPort);
+  var message = data.last;
+
+  Chain.capture(() {
+    runZoned(() {
+      var main;
+      try {
+        main = getMain();
+      } on NoSuchMethodError catch (_) {
+        _sendError(channel, "No top-level hybridMain() function defined.");
+        return;
+      } catch (error, stackTrace) {
+        _sendError(channel, error, stackTrace);
+        return;
+      }
+
+      if (main is! Function) {
+        _sendError(channel, "Top-level hybridMain is not a function.");
+        return;
+      } else if (main is! ZoneUnaryCallback && main is! ZoneBinaryCallback) {
+        _sendError(channel,
+            "Top-level hybridMain() function must take one or two arguments.");
+        return;
+      }
+
+      // Wrap [channel] before passing it to user code so that we can wrap
+      // errors and distinguish user data events from control events sent by the
+      // listener.
+      var transformedChannel = channel.transformSink(_transformer);
+      if (main is ZoneUnaryCallback) {
+        main(transformedChannel);
+      } else {
+        main(transformedChannel, message);
+      }
+    }, zoneSpecification: new ZoneSpecification(print: (_, __, ___, line) {
+      channel.sink.add({"type": "print", "line": line});
+    }));
+  }, onError: (error, stackTrace) async {
+    _sendError(channel, error, stackTrace);
+    await channel.sink.close();
+    Zone.current.handleUncaughtError(error, stackTrace);
+  });
+}
+
+/// Sends a message over [channel] indicating an error from user code.
+void _sendError(StreamChannel channel, error, [StackTrace stackTrace]) {
+  channel.sink.add({
+    "type": "error",
+    "error": RemoteException.serialize(error, stackTrace ?? new Chain.current())
+  });
+}

lib/src/runner/plugin/platform_helpers.dart

@@ -45,6 +45,7 @@ Future<RunnerSuiteController> deserializeSuite(String path,
     'platform': platform.identifier,
     'metadata': suiteConfig.metadata.serialize(),
     'os': platform == TestPlatform.vm ? currentOS.identifier : null,
+    'path': path,
     'collectTraces': Configuration.current.reporter == 'json'
   });
 

lib/src/runner/remote_listener.dart

@@ -78,7 +78,8 @@ class RemoteListener {
           ? null
           : OperatingSystem.find(message['os']);
       var platform = TestPlatform.find(message['platform']);
-      var suite = new Suite(declarer.build(), platform: platform, os: os);
+      var suite = new Suite(declarer.build(),
+          platform: platform, os: os, path: message['path']);
       new RemoteListener._(suite, printZone)._listen(channel);
     }, onError: (error, stackTrace) {
       _sendError(channel, error, stackTrace);
@@ -194,6 +195,8 @@ class RemoteListener {
       });
     });
 
-    liveTest.run().then((_) => channel.sink.add({"type": "complete"}));
+    runZoned(() {
+      liveTest.run().then((_) => channel.sink.add({"type": "complete"}));
+    }, zoneValues: {#test.runner.test_channel: channel});
   }
 }