add {active,N} socket option for TCP, UDP, and SCTP

[vinoski]

Add the {active,N} socket option, where N is an integer in the range
-32767..32767, to allow a caller to specify the number of data messages to
be delivered to the controlling process. Once the socket's delivered
message count either reaches 0 or is explicitly set to 0 with
inet:setopts/2 or by including {active,0} as an option when the socket is
created, the socket transitions to passive ({active, false}) mode and the
socket's controlling process receives a message to inform it of the
transition. TCP sockets receive {tcp_passive,Socket}, UDP sockets receive
{udp_passive,Socket} and SCTP sockets receive {sctp_passive,Socket}.

The socket's delivered message counter defaults to 0, but it can be set
using {active,N} via any gen_tcp, gen_udp, or gen_sctp function that takes
socket options as arguments, or via inet:setopts/2. New N values are added
to the socket's current counter value, and negative numbers can be used to
reduce the counter value. Specifying a number that would cause the socket's
counter value to go above 32767 causes an einval error. If a negative
number is specified such that the counter value would become negative, the
socket's counter value is set to 0 and the socket transitions to passive
mode. If the counter value is already 0 and inet:setopts(Socket,
[{active,0}]) is specified, the counter value remains at 0 but the
appropriate passive mode transition message is generated for the socket.

Note that the range for N is intentionally -32767..32767, not the entire
signed 16-bit integer range -32768..32767. This is because the smallest
negative value, -32767, need only have the same absolute value as the
32767, the maximum allowed N value.

This commit contains a modified preloaded prim_inet.beam due to changes in
prim_inet.erl.

Add tests for {active,N} mode for TCP, UDP, and SCTP sockets.

Add documentation for {active,N} mode for inet, gen_tcp, gen_udp, and
gen_sctp.

[RaimoNiskanen]

Just a quick reflex reaction.

Does excluding -32768 add any significant merit?

If not I think it will only surprise someone some day since not all
values in 16 bit signed two's complement are allowed.

/ Raimo

On Wed, Sep 04, 2013 at 07:41:57AM -0700, Steve Vinoski wrote:

Add the {active,N} socket option, where N is an integer in the range
-32767..32767, to allow a caller to specify the number of data messages to
be delivered to the controlling process. Once the socket's delivered
message count either reaches 0 or is explicitly set to 0 with
inet:setopts/2 or by including {active,0} as an option when the socket is
created, the socket transitions to passive ({active, false}) mode and the
socket's controlling process receives a message to inform it of the
transition. TCP sockets receive {tcp_passive,Socket}, UDP sockets receive
{udp_passive,Socket} and SCTP sockets receive {sctp_passive,Socket}.

The socket's delivered message counter defaults to 0, but it can be set
using {active,N} via any gen_tcp, gen_udp, or gen_sctp function that takes
socket options as arguments, or via inet:setopts/2. New N values are added
to the socket's current counter value, and negative numbers can be used to
reduce the counter value. Specifying a number that would cause the socket's
counter value to go above 32767 causes an einval error. If a negative
number is specified such that the counter value would become negative, the
socket's counter value is set to 0 and the socket transitions to passive
mode. If the counter value is already 0 and inet:setopts(Socket,
[{active,0}]) is specified, the counter value remains at 0 but the
appropriate passive mode transition message is generated for the socket.

Note that the range for N is intentionally -32767..32767, not the entire
signed 16-bit integer range -32768..32767. This is because the smallest
negative value, -32767, need only have the same absolute value as the
32767, the maximum allowed N value.

This commit contains a modified preloaded prim_inet.beam due to changes in
prim_inet.erl.

Add tests for {active,N} mode for TCP, UDP, and SCTP sockets.

Add documentation for {active,N} mode for inet, gen_tcp, gen_udp, and
gen_sctp.
You can merge this Pull Request by running:

git pull https://github.com/vinoski/otp sv-socket-active-n

Or you can view, comment on it, or merge it online at:

#66

-- Commit Summary --

• add {active,N} socket option for TCP, UDP, and SCTP

-- File Changes --

M erts/emulator/drivers/common/inet_drv.c (119)
M erts/preloaded/ebin/prim_inet.beam (0)
M erts/preloaded/src/prim_inet.erl (20)
M lib/kernel/doc/src/gen_sctp.xml (31)
M lib/kernel/doc/src/gen_tcp.xml (6)
M lib/kernel/doc/src/gen_udp.xml (17)
M lib/kernel/doc/src/inet.xml (51)
M lib/kernel/src/gen_sctp.erl (2)
M lib/kernel/src/gen_tcp.erl (2)
M lib/kernel/src/gen_udp.erl (2)
M lib/kernel/src/inet.erl (14)
M lib/kernel/src/inet_int.hrl (1)
M lib/kernel/test/gen_sctp_SUITE.erl (125)
M lib/kernel/test/gen_tcp_misc_SUITE.erl (114)
M lib/kernel/test/gen_udp_SUITE.erl (106)

-- Patch Links --

https://github.com/erlang/otp/pull/66.patch
https://github.com/erlang/otp/pull/66.diff

/ Raimo Niskanen, Erlang/OTP, Ericsson AB

[vinoski]

@RaimoNiskanen thanks for the feedback. Allowing that value would be fine. I excluded it only because I felt it was slightly odd that you could subtract more than you could add, but I also know that's not a very strong argument given that the socket's count can never be set to a negative number regardless. I'm happy to amend the commit to take your feedback into account, just let me know.

[psyeugenic]

I think this is a great feature! The only thing I want to note is that this should probably be in the R17-track.It's a pretty significant feature addition to be thrown in as a patch. In any regard it won't be in R16B02.

[vinoski]

@psyeugenic I assumed it would be considered only for R17 as well, but when I asked if it should be off maint or master I was told maint because it allowed the OTP team the flexibility to take it in either direction.

[psyeugenic]

Ah, so we have been over this, goody. I'm still acquiring conversations I missed during my vacation. It will be R17 then. I'll guess someone at OTP will rebase it when suitable. Topic branches in maint (opu) are currently being drained due to stabilization before release.

[rimmius]

Patch has passed first testings and has been assigned to be reviewed

[vinoski]

I've amended the commit on this branch to address the -32768 issue that @RaimoNiskanen raised.

[rimmius]

Unable to apply diff

[vinoski]

The "unable to apply diff" error was due to the netns change on the maint branch. I've rebased maint out to my branch to resolve this.

[rimmius]

Patch has passed first testings and has been assigned to be reviewed

[krestenkrab]

I would like to understand the exact difference between

• {active, once} and {active, 1}, and
• {active, false} and {active, 0} (i.e. passive mode)

Intuitively, those would be point-wise equivalent. Please correct me if I am wrong, and if so ... I think the documentation should include a description of such semantic differences.

I'm asking because it seems that it would be nice to have this defined as a proper generalisation of the existing semantics, so that we can eliminate the code that handles active/passive cases. The inet driver is complicated enough as it is :-)

[vinoski]

The differences lie in the fact that when {active,N} is set on a socket and N drops to 0 (or is specifically set to 0, see below), a final message is always sent to the controlling process to indicate the active-to-passive transition. This is done so that the controlling process can know when to reset an active mode on the socket if it wishes to do so without having to poll the current N count of the socket.

For example, consider a TCP socket named S set in passive mode with no messages yet sent to it. Setting {active,1} on S results in its controlling process being set up to receive one message from S. If S then receives a TCP message, say with payload <<"foo">>, the controlling process will then get the following two messages, after which S will automatically transition to passive mode:

{tcp,S,<<"foo">>}
{tcp_passive,S}

Setting {active,0} on S is similar but there is no data message — it results in the controlling process getting only the active-to-passive transition message, after which S transitions to passive mode.

[ferd]

Will setting the socket to {active,1} and then directly setting it to {active, once} toggle the sending of {tcp_passive,S} in the same way?

I figure it won't (because it wouldn't make much sense given the connection is still active), but I can't help but feel there is an inconsistency in how {active,1} and {active,once} do not share the same behavior even though they should roughly mean the same thing.

Isn't the ambiguity of the N value for active just something you should be ready to deal with, the same way you should be with {active,once}? We will deal with it already for things like:

1. transitioning from {active,1} to {active,once} and maybe/maybe not receiving tcp_passive and/or a data message depending on whether we got a message while in {active,1};
2. transitioning from {active,1} to {active,false} and maybe/maybe not receiving tcp_passive and/or a data message depending on whether we got a message while in {active,1}
3. transitioning from {active,1} to {active,N} where N > 1 and maybe/maybe not receiving tcp_passive and/or a data message depending on whether we got a message while in {active,1}
4. the existing conditions where you transition away from {active, true} to any other mode but {active, N} and may drop messages if we're not careful
5. the existing conditions where you transition away from {active, once} to {active,false} and may drop messages if not careful

Why should {active,0} be a special case there and sending {tcp_passive,S}? It feels to me like we're not gaining anything in there except a false sense of security in the transitions we're doing. It would remove the uncertainty of points 4 and 5, but would not remove 1,2, or 3 in any way. We'd be replacing one unexpected message (a data message) by another one (a transition message) in some scenarios, in a way that is quite confusing: is the data message I just received from the time I was in {active,1} or in {active,once}, and how long should I wait for the tcp_passive message, for example?

I guess the argument should come down to a preference on whether we want to track the N value of {active,N} implicitly or explicitly, and this patch seems to favor implicitness.

I think there's good points to be had for both, but we should be careful when making that decision?

[vinoski]

If a socket is in {active,N} mode and you then set it to {active,once} mode, no active-to-passive transition message is sent to the controlling process as a result of that action, since that action alone causes no active-to-passive transition.

The {active,once} mode and {active,1} mode are different; arguing that they're the same means you're in favor of imposing a special case for N=1, but such a special case provides no advantages. With {active,once} your code knows that as soon as the controlling process receives any data message from the socket, the socket will have transitioned into passive mode — the transition is implied by the arrival of the data message. If you want the app to receive another message after {active,once} has already sent it a message, you know you need the code to reset the socket to {active,once} mode. The same cannot be said, however, for {active,N} mode; you need to either

• include the current value of N in every message so the receiver can match N=0
• force the controlling process to poll the socket's N value
• force the controlling process to keep its own message counter
• send a special message like {tcp_passive,S}

Of these choices, the last one, which is what this patch implements, is the cleanest.

• The first approach, including N in every message, is mostly just a waste of space, since the N=0 case is the only one that indicates a socket active-to-passive transition. Adding such a count would also mean that data messages received in {active,N} mode would contain a field not present in data messages received in other active modes, resulting in a special case causing needless development confusion and code maintenance headaches.
• The second alternative of forcing code to poll the socket's N value has the issue that since all data messages sent from a socket to its controlling process have the same "shape" from a pattern-matching perspective — for example, {tcp,S,Data} — it's likely that a message handler would just have to poll on receipt of every message of that shape, which is inefficient.
• The third approach of forcing applications to keep their own counts of messages simply duplicates the count the socket already keeps and adds needless development overhead to all code using {active,N}.

And just as {active,1} should not be a special case, {active,0} should not be a special case either. Fortunately, neither is a special case in this patch. The current semantics of {active,0} are the same as for any other N value: if the setting of the value causes the socket's message counter to reach 0, then the socket transitions to passive mode and an active-to-passive transition message is sent to the controlling process.

The points 1-5 you describe are race conditions resulting from the fact that messages can arrive on a socket concurrently with the application making calls to change active/passive modes on that socket. As your descriptions already note, these race conditions are already possible in Erlang today, and this patch neither improves nor worsens the situation, as that is not its focus. In fact, I would argue that while the explicit transitions of the socket's active mode between N, true and once that you've noted are clearly possible, they do not strike me as being all that useful in practice. Perhaps I've led a sheltered Erlang life but I personally have never seen code that freely mixes different active and passive modes willy-nilly on the same socket as you're suggesting — I fail to see what benefit an app would gain from doing so.

In practice, the active/passive modes are about flow control and are used to engage backpressure mechanisms in protocols that provide them. Today, for active modes we have only {active,true} mode with no flow control or backpressure possibilities, and {active,once} which allows only a relative drip of messages to come through. The introduction of {active,N} gives applications the ability to better tune the tradeoff between the flow of messages to a receiver and applying backpressure to the sender.

[krestenkrab]

Thanks Steve. I see it now. But I'll still argue that it's fairly complicated, and likely to cause confusion. Perhaps we should call it something other than {active, N} to signify that it's not the same. {machine_gun, N} come to mind; with a final tcp_recharge after the burst is done.

[krestenkrab]

Here's another thought: as a client if a stream socket, most likely I'm interested in receiving a limited number of bytes, not necessarily a limited number of MTU units of data; as would probably typically be the consequence of using {active, N}. So perhaps that would be a sensible semantics? What if the back pressure generated by this option was related to the amount of data, and not the number of data messages? However as a use of a socket accepting connections, I'd be interested in accepting a given number of new connections.

[vinoski]

I don't think a new name is needed since this new option fits perfectly with the other existing active options. On the spectrum of "burstiness" {active,N} neatly covers the range between {active,true} and {active,once} since the former provides a never-ending burst of messages while the latter allows a "burst" of one.

As for treating N as a byte count instead of a message count, I don't think that would be a good approach, for at least the following reasons:

• It differs completely from {active,true} and {active,once} semantics.
• It would disconnect the N option from the size of the controlling process's message queue, which is one of the primary resources managed by the active options.
• The active modes apply to more than just stream-oriented protocols.
• The recv calls in passive mode can already be used to receive specified numbers of bytes.

[rimmius]

Merged.

[krestenkrab]

Great!