Futures

If you pass !_ instead of _ when calling a streamline function, the function will execute synchronously and return a future. The future is just a function that you can call later to obtain a result. Let us see how it works on an example:

function countLines(path, _) {
  return fs.readFile(path, "utf8", _).split('\n').length;
}

function compareLineCounts(path1, path2, _) {
  // parallelize the two countLines operations
  var n1 = countLines(path1, !_);
  var n2 = countLines(path2, !_);
  // join the results
  return n1(_) - n2(_);
}

In this example, countLines is called synchronously, with !_. These calls start the fs.readFile asynchronous operations and return two futures (n1 and n2). Later, n1(_) and n2(_) retrieve the results via callbacks that are automatically generated by the streamline transformation engine.

Note: syntax has changed. In 0.8 and before you could just omit the _ parameter or pass null. This syntax was cute but it led to difficult debugging when the _ was left out by mistake. The new syntax makes it possible to verify (at runtime) that the function is called with either _ or !_.

Streamline does not really provide any special API to manipulate futures. The future which is returned is a simple function that takes a single callback parameter. You can evaluate the future from streamline source by calling it with _ but you can also pass the future to a non-streamline module which will evaluate it by calling it with a standard node callback (function(err, result) { ... }).

You can use futures to parallelize operations on an array. For example, the following function will compute the number of lines for an array of file names:

var flows = require("streamline/lib/util/flows");

function lineCounts(paths, _) {
  // start all the operations in parallel
  var futures = paths.map(function(path) {
    return countLines(path, !_);
  });
  // collect the results into an array
  return flows.collect(_, futures);
}

Of course, this lineCounts function is another streamline function. So you can call it with !_; it will return a future which resolves into an array of line counts. You can call it either as:

// classical way: pseudo-synchronous
var counts = lineCounts(paths, _);

or as:

// get a future
var countsF = lineCounts(paths, !_);
// do other things while lineCounts is running
doSomethingElse(_);
// get the future's value
var counts = countsF(_);

If you are interested by the more academic aspects of this design, you can read this: http://bjouhier.wordpress.com/2011/04/04/currying-the-callback-or-the-essence-of-futures/