Proposal: change the default value of `initial_max_send_streams` to 100

[magurotuna]

Issue

Currently, the number of streams opened by the client before it receives the initial SETTINGS frame from the server is set to usize::MAX by default.

h2/src/client.rs

Line 645 in 122091a

initial_max_send_streams: usize::MAX,

This is valid according HTTP/2 spec (in fact the spec doesn't mention any explicit limit here). However, opening unlimited number of streams can lead to majority of them being rejected by the server with REFUSED_STREAM, if the server's SETTINGS_MAX_CONCURRENT_STREAMS is set to any lower value. This is simply not ideal and waste of resources.

Proposal

To better handle this issue, I propose that we start with reasonably low number of streams by default, rather than the maximum possible number. The initial number is only used until the client receives the initial SETTINGS frame, so it should be much more preferable to start with a low number in order to reduce the risk of REFUSED_STREAM errors even if it might end up with lower utilization for a very short time.

What is the reasonable default number? The good number should be as low as what most servers would accept. In fact, the spec recommends that the value should be no smaller than 100. To back up this recommendation, I've done a quick research on the values of MAX_CONCURRENT_STREAMS used by several popular servers. The result is as follows.

Server	MAX_CONCURRENT_STREAMS
google.com	100
youtube.com	100
x.com	100
facebook.com	100
www.amazon.com	100
www.apple.com	100
netflix.com	100
microsoft.com	100
cloudflare.com	100
fastly.com	100
hyper.rs	100
openai.com	128
www.rust-lang.org	128
crates.io	128
vercel.com	250
tokio.rs	256
deno.com	(unspecified)

So if the client starts with 100 streams at maximum, all of these servers will accept them and the client will not receive any REFUSED_STREAM errors. Therefore we can conclude that 100 is the reasonable default number of streams to start with.

Take a quick look at other HTTP/2 implementations

To further support the proposal, let's have a look at how other HTTP/2 client implementations are doing. In short, all of these three implementations adopt starting with the reasonably small number of streams.

nghttp2

nghttp2's default initial maximum number of concurrent streams is 100.
https://github.com/nghttp2/nghttp2/blob/837f0c67c7139aedfb0c31d252634c823bfa75ba/lib/includes/nghttp2/nghttp2.h#L2537-L2554

Go x/net/http2

Go also uses 100 as the the initial number of maximum concurrent strams.
https://cs.opensource.google/go/x/net/+/689bbc7005f6bbf9fac1a8333bf03436fa4b4b2a:http2/transport.go;l=56

One notable difference from nghttp2 is that Go will set 1,000 as the MAX_CONCURRENT_STREAMS if the server doesn't provide one, while nghttp2 sets inifinity in this case.

Python httpx

httpx starts with just 1.
https://github.com/encode/httpcore/blob/2fcd062df71555cc7de55774c6dc137551eb8692/httpcore/_async/http2.py#L117-L119

[seanmonstar]

This is a great write-up, thank you! I agree it makes sense to adjust the default. I only want to pause to make sure it's the right place to adjust the default.

It's worth calling out that for workloads where the server will NOT place a limit, changing this default here could be a negative. This would be places where both peers are trusted. My mind goes to things like linkerd2 proxies. (cc @hawkw @olix0r) The proper solution could actually be that those use cases should explicitly set a high initial value, instead of depending on the default, which perhaps should aim to protect the most common cases.

But I also vaguely feel like so far we've put those defaults in hyper or reqwest, and left h2 to simply implement the protocol, and otherwise not pick defaults.

[magurotuna]

It makes sense that changing the default value could have a negative impact in a rare situation.
Actually, I opened an issue to here instead of to hyper (or reqwest) because I found that the client builder in h2 provides a way to set the initial MAX_CONCURRENT_STREAMS value while hyper doesn't. Plus, it looks like hyper doesn't set any value for initial MAX_CONCURRENT_STREAMS when constructing h2 client, thus ending up using the default value specified in h2, i.e. usize::MAX.

So probably what we would like to do is:

• to have hyper provide a method on hyper::client::conn::http2::Builder that allows users to set the initial MAX_CONCURRENT_STREAMS value
• to have hyper call h2::client::Builder::initial_max_send_streams when constructing h2 client with the user-configured value, if any.
  • If no value is configured by the user, set the reasonable default value i.e. 100.
  • Or we could set the default value at one more upper level, i.e. reqwest.
• to have hyper_util provide a method on hyper_util::client::legacy::Builder that allows users to set the initial MAX_CONCURRENT_STREAMS value

I also feel that some changes still need to be made to h2, namely:

• improves the doc of h2::client::Builder::initial_max_send_streams to clearify that the default value is usize::MAX
• sets MAX_CONCURRENT_STREAMS to usize::MAX or whatever reasonable value (for example Go adopts 1,000) if the server's initial SETTINGS frame doesn't include MAX_CONCURRENT_STREAMS, assuming that the absence of MAX_CONCURRENT_STREAMS implies the server is willing to accept infinite number of concurrent streams
  • note that currently h2 keeps using the value set by h2::client::Builder::initial_max_send_streams if no specific value is supplied by the server

Do these sound reasonable to you? If they do, I'd be happy to work on these :)

[dswij]

But I also vaguely feel like so far we've put those defaults in hyper or reqwest, and left h2 to simply implement the protocol, and otherwise not pick defaults.

+1. h2 definitely should concern the protocol-level implementation only. The defaults should be in hyper.

Having arbitrary defaults everywhere is also a nightmare to handle, so it's good that we're avoiding that.

to have hyper call h2::client::Builder::initial_max_send_streams when constructing h2 client with the user-configured value, if any.

This sounds like it should be exposed in hyper

[magurotuna]

Thanks for your opinion @dswij.

This sounds like it should be exposed in hyper

What do you mean by this? If what you mean is that the users of hyper should be able to configure the initial value of max send streams, then I think it's covered by the first bullet:

• to have hyper provide a method on hyper::client::conn::http2::Builder that allows users to set the initial MAX_CONCURRENT_STREAMS value

[dswij]

What do you mean by this? If what you mean is that the users of hyper should be able to configure the initial value of max send streams, then I think it's covered by the first bullet

Yes :)

I'm completely on board with the first point. Just saying that regardless of changing the default value, this option should be configurable in hyper. So I see this as another improvement if we go down this route.

[magurotuna]

So I separated #732 into a couple of PRs. Now the part of setting a reasonable default value is moved to hyper. Here are the separated PRs:

• fix: set MAX_CONCURRENT_STREAMS to usize::MAX if no value is advertised initially #736
• feat(http2): add initial_max_send_streams method to HTTP/2 client builder hyper#3524
• feat(http2): add initial_max_send_streams method to client builder hyper-util#87

[seanmonstar]

Thanks a lot for doing that, your contributions have been wonderful to work with ♥️