Multiple connections on the same port

[MikeBishop]

@mikkelfj has raised questions about whether we might be able to maintain many connections on the same port. @ianswett has pointed out that other connection-ID-oriented protocols operate all connections on the same port to minimize the use of ephemeral ports. However, if any of those connections are to the same back end, there will be duplicate 5-tuples, and QUIC's current model doesn't support simultaneous handshakes with the same 5-tuple.

If we want to enable this, we would need to have the server's first flight continue using the client-chosen Connection ID, but contain the new Connection ID.

[janaiyengar]

Summarizing editors' discussion: We can use this new connection ID for only 1-RTT encrypted packets. We can further simplify the transport mechanism for server-chosen connection ID, by simply making it part of transport_params.

Note that #713 handles the case of server-chosen connection ID for Stateless Retries.

[igorlord]

That said, we should make sure to discourage the use a single client port for all of its QUIC traffic. Routers do EQMP based a on 5-tuple, and most of them will not be QUIC-aware for years.

[igorlord]

@janaiyengar, could you clarify what you mean by: "We can further simplify the transport mechanism for server-chosen connection ID, by simply making it part of transport_params"?

We are starting our load balancer implementation for IETF QUIC, so details like this are very pertinent.

[MikeBishop]

Basically, the reasoning in the meeting was this: If you want multiple handshakes sharing the same 5-tuple in parallel, then you have to be able to identify which handshake a given packet belongs to. That means the first packet from the server that gives you the server-chosen ID also has to have the client-chosen ID so it can be correlated. The client can start using the server-chosen ID after that, as currently defined.

If the server's choice of connection ID has to be carried in its first packet and that packet has to have the client-chosen connection ID, then it makes sense to just carry the new connection ID in the transport parameters, which are in that packet anyway for the normal case. (Of course, we also need a way to carry it in a HelloRetryRequest, but that's #713.)

[janaiyengar]

What Mike said. Thanks for speaking up about your implementation effort, Igor, that's useful to know. I think at this point, #713 and this issue are the only ones that we need to resolve... and actually for your load balancer implementation, you only really need this issue to be resolved. I'll spin up a PR soon.

[igorlord]

So the server's first 1-RTT packet would have client's connection ID and carry server's connection ID in transport params. But all subsequent server's packets will have server's connection ID? (Or will all subsequent server's packets still have client's connection ID, while client's 1-RTT packets will have Server's connection ID?)

In the former case (only the server's first 1-RTT packet has client's connection ID), what happens if that first packet is lost/reordered? Will the client respond with a Stateless Reset, when it receives server's second 1-RTT packet with server's connection ID?

[janaiyengar]

So, if we carry the server-chosen connection ID in transport params, we can be assured that the client will have this ID by the time it is trying to decrypt 1-RTT-encrypted packets.

So the rules then become:

• All 1-RTT encrypted packets use server-chosen connection ID.
• All other packets carry client-chosen connection ID (*)

(*) The exception to this rule is the Client Initial following a Stateless Retry Packet, but that does not matter from a load balancer's perspective, since the connection ID in Client Initial packets are supposed to be treated by load balancers as client-chosen.

Does that make sense?

[igorlord]

Ok, that makes sense.

One observation that such a multiplexing client would need to be especially careful with QUIC packets featuring unknown Connection IDs.

if we carry the server-chosen connection ID in transport params, we can be assured that the client will have this ID by the time it is trying to decrypt 1-RTT-encrypted packets

If the client receives 1-RTT-encrypted packet from the server before it received a packet with QUIC TLS transport params (server-chosen connection ID), quic-tls ("9.3. Receiving Out-of-Order Protected Frames") says that the client may store those packets and later decrypt them. If the client is in the process of negotiating multiple connections using the same 5-tuple, it will not be able to rely on a 5-tuple to identify a specific QUIC connection it is trying to negotiate and will need to do something more clever. That client would also need to be clever enough to know when it can and cannot send a stateless reset in response to a packet with a connection id it is not aware of.

[janaiyengar]

Good point. In this case, the client would have to maintain a shared buffer for undecrypted packets that arrive on the socket but cannot be assigned to a connection yet, or some such logic. That seems fine as the responsibility of a client that is choosing to multiplex connections over the same 5-tuple.

As for Stateless Reset, that is only sent by the server, and the connection ID is clear in that case.

[mikkelfj]

One observation that such a multiplexing client would need to be especially careful with QUIC packets featuring unknown Connection IDs.

Due to packet spoofing and port guessing this is/should also be a concern with 5-tuples during initial handshake - relying on connection IDs ought to signficantly reduce this concern because it is difficult to guess off path. But I agree that this must be considered regardless.

However, is it strictly relevant to buffer unknown packets. It is much simpler to drop them when there isn't yet an established context to hook them up to and it does protect against random noise/attacks. It will only be relevant in the rearly connection setup. I'm just guessing here, but for flow control it might not be a bad thing to register a dropped packet because clearly there is some performance issue unless it is trivial reordering - and multi-channel reordering typically happens in larger bulks I believe. So, optimizing performance under adverse conditions might not be the best option. A typical attacker could force dropped packets by overflowing network and router queues and here end-point buffering would add some reselience, but then, most end-point buffering would likely be from adversary.

[martinthomson]

With the recent changes to packet protection, packets for different connections are cryptographically distinct. However, that means a bunch of trial decryption for those packets from the server that are reordered. I think that we really want to make sure that we want this feature. It might be something we can defer.

[mikkelfj]

Possible solution:

• The Clear Text packets could carry the client connection ID unencrypted in both directions and expose the server chosen ID as a separate field to inform the client about future 1-RTT transmissions. It would make the server clear text header larger, but that is likely an acceptable cost.
• 1-RTT encrypted packets would carry the server chosen ID in both directions.
• If there is no connection ID in the packets, multiple connections on same port would not be supported.
• Some implementations or configurations may require the connection ID to always be present.
• 0-RTT must also be considered, which I have not done here.

This would simplify connection lookup when the connection ID is always present because there is no need for separate 5-tuple mappings, which require extra work for high speed interfaces such as netmap and might not be possible without on some non-UDP links without extra pre-header.