Pipeline support

[klumsy]

I know that this has been written up before , and the current consensus is that 1) it has a limitation that it doesn't allow functions with multiple arguments, and that it is a niche functionality only useful for DSLs.. I think that 1) it doesn't have to have that limitation, and you can look at F#'s implementation to show this, and 2 it can be very generic and useful, as seen in both F# and more so in powershell.

here is a link to how it is done in F# http://lorgonblog.wordpress.com/2008/03/30/pipelining-in-f/

Here is some coffee of mine doing a scenario. first how i have to call it now, then how i'd like to be able to call it, and third how i can do it with a helper function, then following up with how it can be done in kaffiene currently. The example is rather contrived but it should be sufficient to communicate

where = (list,func) -> item for item in list when (func.call _ :item)  
select = (list, prefix,postfix) -> "#{prefix}#{item}#{postfix}" for item in list
getnums = (x) -> [1..x]

is there any easier way i can write the following below

results = select( (where (getnums 10) ,-> @_ > 5) , "pre","post" )

alert res for res in results

what it really would like to do is be able, whether using | or say |> for a pipe do

getnum | where -> @_ > 5 | select "pre","post"

basically the pipe takes the output of the previous command and puts it as teh first argument, and then takes any other arguments after that (so 1st argument becomes 2nd, and second becomes third..

with my own pipe i can do something like
pipe getnum .to where, -> @_ > 5 .to select , "pre","post"

kaffiene though not complete lets me do this with

getnums = { [1,2,3,4,5,6,7,8,9,10] }select = (list,prefix,postfix) {
  res = []
  for x of list {
   res.push("#{prefix}#{x}#{postfix}")
    }
  return res
  }

and call it like this in kaffiene

getnums | select "pre","post"

generates the following javascript

var getnums, select; getnums = function() { return [1,2,3,4,5,6,7,8,9,10] }select = function(list,prefix,postfix) {
  var _a, x, res; res = []
  for(_a = 0; _a < list.length; _a++) { x = list[_a];
   res.push("" + (prefix) + (x) + "" + (postfix) + "")
    }
  return res
  }
__.select.call(this, getnums, "pre","post")

and

getnums | select "pre","post" | select "yo","dude"

__.select.call(this, __.select.call(this, getnums, "pre","post"), "yo","dude")

[jashkenas]

If you can edit this post to indent all code with four spaces, it'll make things easier to read.

[klumsy]

that was quick :), i figured i'd get my thoughts down, then pretty it up with whatever formatting options github had, which turned out to just be a dialect of markdown

[odf]

I think this would indeed be an immensely useful idiom. One way to think about it is as an analogue to the chaining one would get if each function were a method of the last function's result. But this is obviously much more flexible.

If I'm not mistaken, clojure does a very similar thing via a built-in macro.

[klumsy]

If i implement this well in a fork, how likely would it be to be integrated back into coffeescript?

[jashkenas]

Not likely -- see all of the previous "pipe" tickets.

[M1573RMU74710N]

I realize the general consensus is against this, but I find the idea interesting and I wonder if you guys would be so kind as to humor me for a moment and pretend this was to be implemented.

I'm curious as to your thoughts on the following possible syntax:

The pipe operator |>, passes the result of the previous operation to the next one.

By default, the previous result is passed in as the first argument:

add 5, 5 |> times 10 |> alert

possible compilation:

var __result;
__result = add(5, 5);
__result = times(__result, 10);
alert(__result);

Binding pipe operator: <|, all following operations are bound to the context of the result of the expression preceding
the binding pipe operator

The default passing of the results is turned off.

new Foo <| makeBig 10 |> toggleBar

possible compilation:

var __result, __context;
__context = new Foo();
__result = makeBig.apply( __context, [10]);
toggleBar.apply( __context, []);

If the rebinding occurs in the middle of a piping, the context is rebound, but the results of the previous piping are passed on:

new Foo <| makeBig 10 |> myBar <| addFooChild

possible compilation:

var __result, __context;
__context = new Foo();
__result = makeBig.apply( __context, [10]);
__context = myBar;
addFooChild.apply( __context, [__result]);

#@ calls the function as a method of the current __context:

new Foo <| #@makeBig 10 |> myBar <| #@addFooChild

possible compilation:

var __result, __context;
__context = new Foo();
__result = __context.makeBig(10);
__context = myBar;
__context.addFooChild(__result);

Empty context just continues default pipe behaviour:

new Foo <| #@makebig 10 |> <| addFooChild

possible compilation:

var __result, __context;
__context = new Foo();
__result = __context.makeBig(10);
addFooChild(__result);

Piping #@ simply puts __context in __result:

new Foo <| #@makebig 10 |> #@ |> myBar <| #@addFooChild

possible compilation:

var __result, __context;
__context = new Foo();
__result = __context.makeBig(10);
__result = __context;
__context = myBar;
__context.addFooChild(__result);

Additionally/Alternatively, #@ could be used as an argument.

Handling positional arguments; by default the result of the previous pipe is passed to the first argument, but this can be redirected with #

add 5, 5 |> times 10, # |> alert

possible compilation:

var __result;
__result = add(5, 5);
__result = times(10, __result);
alert(__result);

Destructuring; the return value of the previous operation can be destructured with #:

getBars "foo" |> toggleBar #1

possible compilation:

var __result;
__result = getBars("foo");
toggleBar(__result["1"])

Or:

getBars "foo" |> toggleBar #, #foo

possible compilation:

var __result;
__result = getBars("foo");
toggleBar(__result, __result["foo"])

So those are just some rough ideas I just had, not wholly fleshed out yet. Some of this syntax may be unnecessary, and some of the "defaults" might be switched around, but I hope you get the general idea. Also, I realize the examples aren't great either but I'm mainly concerned about the syntax. Not covered is assigning the results of the piping to something, but I think you can see how straight-forward that would be.

My thought is that overall, chaining should be the go-to tool for this type of semantics, but chaining has some drawbacks. With chaining all the methods have to be a property of or mixed in to some master object, and the methods have to be written to take arguments in a certain way. This type of piping allows you to use arbitrary functions and I believe allows more control over the "flow". I think in certain situations it would be a nice tool to have.

I could see some drawbacks in readability etc, but IMHO it's not too bad. It's getting away from the "it's just Javascript" philosophy a little bit, but the compiled code should be pretty straight-forward.

[odf]

My thought is that overall, chaining should be the go-to tool for this type of semantics, but chaining has some drawbacks. With chaining all the methods have to be a property of or mixed in to some master object, and the methods have to be written to take arguments in a certain way. This type of piping allows you to use arbitrary functions and I believe allows more control over the "flow". I think in certain situations it would be a nice tool to have.

Pretty much my thoughts, even though I'm not sure I'd be comfortable with all that extra syntax you're suggesting. I think the basic |> piping would be far more powerful by itself than people think.

[M1573RMU74710N]

odf: Agreed. I think it would be nice to have such diverse options, but it is a lot of syntax to add/read.

I think just simple piping + handling positional arguments + destructuring would provide enough benefit over chaining that it would be worth it to add. In addition to not being tied to "chainable" functions, a compiled pipeline would probably involve less run time overhead.

[ricardobeat]

The idea of select "pre", "post" having an invisible first argument inserted bugs me a lot. Isn't that drifting too far away from js?

I much prefer the coffeey, readable versions of this:

select getnums().filter((v) -> v._ > 5), "pre", "post"
or
select (n for n in getnums() when n > 5), "pre", "post"

[thejh]

How about this? It's less "magical" (what @ricardobeat didn't like), but you have to write a little bit more.

getnums | select input, "pre","post" | foobar input

compiles to something like

(function(input) {
  (function(input) {
    foobar(input);
  })(select(input, "pre", "post"));
})(getnums());

Maybe we could even combine both ways and just assume that input should be the first argument for the first function call or so if there's no reference to it.

Advantages:

• You can use the input multiple times without using an extra wrapper function:

  # calculate the average value
  getnums | (num/input.length for num in input) | sum
  

• You can choose which argument the input should become:

  getnums | select input, "pre","post" | foobar 10, input
  

[ricardobeat]

@thejh | is already an operator, bitwise OR.

I'd rather have your example compiled to

foobar( select( getnums(), "pre", "post" ) )

for clarity, performance and mantaining scope. But your proposal is quite reasonable considering the use of expressions. On the other hand, making the name input special, or any other keyword, is very unlikely.

A related discussion is happening in #1407, on chaining function calls.

[satyr]

getnums | select input, "pre","post" | foobar input

compiles to something like

(function(input) {
  (function(input) {
    foobar(input);
  })(select(input, "pre", "post"));
})(getnums());

I'd rather have your example compiled to

foobar( select( getnums(), "pre", "post" ) )

That could mess up evaluation order. Consider:

IN |> ctor1().method1 |> ctor2().method2 getArg(), <>

↓

ctor2().method2(getArg(), ctor1().method1(IN))

where <> is @thejh's input.

[pm100]

in the previous discussion on ticket 281 it was said that function chaining could do the same thing. This is not the case

`````` get().head().sort()requires that get 'knows' that head exists. Its return value must support the head method, similarly the return value of head must support the sort method. whereas the proposedget | head | sort```
would not have that constraint

[mikeobrien]

+1 for F# style pipelining. This is one of my favorite idioms from F#.

[avernet]

Folks, this isn't a full substitute for F# |>, but what would you think using:

pipeline = (v, fs...) -> v = f.call(v) for f in fs; v

As in:

r = pipeline 2,
    -> @*2
    -> @+1

[klumsy]

i have played around with doing that sort of thing inline before, but
really want it with the language.. here is an example.. i've moved on from
asking though. what i'm playing around with now is producing a powershell
style syntax and pipelining language that uses coffeescript for the
expressions.

*where = (list,func) -> item for item in list when (func.call _ :item)
select = (list, prefix,postfix) -> "#{prefix}#{item}#{postfix}" for item in
list
getnums = (dummy,x) -> [1..x]

pipe = (listoflists...) ->
res = null
for list in listoflists
func = list[0]
args = list[1..list.length]
res = func.apply(this,[res].concat(args))
res

results = pipe [getnums, 10] , [where, -> @_ > 5 ] ,[select , "pre","post" ]
alert result for result in results*
*
*
On Mon, Nov 21, 2011 at 3:27 PM, Alessandro Vernet <
[email protected]

wrote:

Folks, this isn't a full substitute for F# |>, but what would you think
using:

pipeline = (v, fs...) -> v = f.call(v) for f in fs; v

As in:

r = pipeline 2,
-> @*2
-> @+1

---

Reply to this email directly or view it on GitHub:
#1339 (comment)

[avernet]

@klumsy, or, using the above pipeline:

pipeline = (v, fs...) -> v = f.call(v) for f in fs; v
filter = (p) -> -> vs = @; v for v in vs when p.call v
addPrePost = (pre, post) -> -> vs = @; "#{pre}#{v}#{post}" for v in vs

r = pipeline [1..10],
    filter -> @ > 5
    addPrePost "pre", "post"

Run it

I tend to prefer this pipeline construct as it also works for pipelines that operate on single "values" instead of just arrays/lists/sequences. For the latter, using "just" Underscore's chain might even do the trick.

[klumsy]

you opened my mind to a whole nother world of possibilies with your -> ->
usage!!
taking that to another level (where it returns an object with multiple funs
for begin , process and end, i can actually implement a streaming pipeline.

Thanks.

On Mon, Nov 21, 2011 at 4:12 PM, Alessandro Vernet <
[email protected]

wrote:

@klumsy, or, using the above pipeline:

pipeline = (v, fs...) -> v = f.call(v) for f in fs; v
filter = (p) -> -> vs = @; v for v in vs when p.call v
addPrePost = (pre, post) -> -> vs = @; "#{pre}#{v}#{post}" for v in vs

r = pipeline [1..10],
filter -> @ > 5
addPrePost "pre", "post"

Run it

I tend to prefer this pipeline construct as it also works for pipelines
that operate on single "values" instead of just arrays/lists/sequences. For
the later, using "just" Underscore's chain might even do the
trick.

---

Reply to this email directly or view it on GitHub:
#1339 (comment)

[avernet]

A recent proposal to add method cascades to Dart would make it possible to write Dart code as follows:

document.query('#myTable').{
    queryAll('.firstColumn').{
        style.{
            background = 'red',
            border = '2px solid black'
        },
        text = 'first column'
    },
    queryAll('.lastColumn').{
        style.background = 'blue',
        text = 'last column'
    }
};

In CoffeeScript, we can declare a cascade function, which is very similar to the above pipeline:

cascade = (v, fs...) -> f.call(v) for f in fs; v

This allows us to write the above Dart code, in CoffeeScript, as:

cascade document,
    -> cascade (@query '#myTable'),
        -> cascade (@queryAll '.firstColumn'),
            -> cascade @style,
                -> @background = 'red'
                -> @border = '2px solid black'
            -> @text = 'first column'
        -> cascade (@queryAll '.firstColumn'),
            -> @style.background = 'blue'
            -> text = 'last column'

Try it

(Posting this as a follow-up here, as the ressemblance between cascades and pipelines is striking.)

[paulmillr]

I'd like to re-start a discussion on this.

"Method cascades" are obviously shitty, because if user'll want to define new method, he'll be forced to change prototype which is not modular etc. Not mentioning that mutable state sucks balls when used inappropriately and / or everywhere.

LiveScript has |> operator. The syntax is:

[1 2 3 4 5]
  |> map (+ 5)
  |> filter (> 6)

As you see, map and filter are simple modular functions here.

[OnesimusUnbound]

@paulmillr the map and filter functions are curried functions of prelude, a LiveScript based module. We have to be able to get the curry feature into CS to make the |> operator feasible.

Anyway, I hope to see such construct. Functional languages opened my eyes in it's ability to compose functions.

[paulmillr]

@OnesimusUnbound until then we can use (a) -> (b) -> a * b

[aurium]

I really like the @thejh's input proposal! That opens the pipe proposal for new possibilities.

I'm also concerned with the @ricardobeat's point about a new keyword, but I don't like the @satyr's <> solution. However that is not his fault, we have no enough symbols in a standard keyboard, so... What about @@ to replace the input keyword?

[carlsmith]

Just for my own sanity, would anyone be kind enough to explain what the difference is between the following two lines?

foo bar spam eggs args...

eggs args... |> spam |> bar |> foo

I fail to see what the later can do that the former can't?? The first way allows for more than one arg too.

edit set "foo.coffee", clone "98f97a41924ca81c9863"

The trick is ensuring that each function can be called with one argument which is of a kind that each of the functions will return, generally an object with one or more required properties.

[aurium]

What the difference is between Javascript and Coffeescript? Javascript can do everything that Coffeescript do. So... forget... Lets be nice. :-)

select (where (getnums 10) ,-> @_ > 5) , "pre", "post"

getnum |> where -> @_ > 5 |> select "pre", "post"

The second looks more "natural" to me. My 2¢.

[vendethiel]

#3600

[carlsmith]

Thank God for Coco. If it didn't exist, I'd be forced to use half of it.

[aurium]

@Nami-Doc #3600 was closed. Don't must this remain open?
(this issue has more ideas to discuss)

[vendethiel]

@aurium The "duplicate of" issue was closed.

[hzamani]

@vendethiel so is this open?!