Skip to content

node:http: give CONNECT and Upgrade tunnel sockets read backpressure - #43570

Closed
robobun wants to merge 4 commits into
mainfrom
robobun/d9c03192/http-tunnel-read-backpressure
Closed

robobun wants to merge 4 commits into
mainfrom
robobun/d9c03192/http-tunnel-read-backpressure

Conversation

@robobun

@robobun robobun commented Sep 19, 2026 •

Copy link
Copy Markdown
Collaborator

Problem

  • The socket that node:http gives to 'connect' and 'upgrade' listeners never stops its reads. A paused socket, or a pipe to a slow destination, buffers all that the peer sends: 256 MB sent, readableLength 256 MB. Node v26.3.0 holds 64 KB.
  • Cause: pause_socket() and resume_socket() return early in tunnel mode (src/runtime/server/NodeHTTPResponse.rs:465), and #onData ignores the result of push() (src/js/node/_http_server.ts:1573).

Fix

  • Like net.Socket: when push() returns false, handle.readStop() pauses the raw socket. _read() calls handle.readStart(), and resume() does while the buffer has room. pause() alone stops no reads, as in Node and node:net (net: keep the handle reading while a socket is paused, like node #39830).
  • Native hands chunks to JS in a task, so readStop() comes after the read loop, which reads on while recv() fills its 512 KB buffer. onData() pauses the socket while 512 KB wait for JS.
  • A pause exposed three faults, each fixed and tested: a 'drain' for every read pause or resume, tunnel bytes dropped after socket.end(), and a WebSocket that adopts a paused connection and never reads.
  • Verified: test/js/node/http/node-http-connect.test.ts, 13 new tests, 11 fail on main. Self-reviewed: 13 concerns raised, 10 addressed, 1 not reproduced, 2 are older bugs (notes).

Background

  • Tunnel mode (isConnectRequest): after a CONNECT or an accepted Upgrade, the HTTP parser passes all bytes to the JS socket unparsed.
  • us_socket_pause() takes the socket out of the poll set for reads. The read loop checks that after each chunk.
  • Flood prevention: the server pauses reads while a response has unsent bytes, and resumes them later. That resume now skips a tunnel that paused its own reads.
Notes

Repro. A 'connect' listener calls socket.pause() and answers 200. A net client then writes 256 MB in 64 KB chunks and waits for 'drain'. After 3 s (release builds, same machine):

client wrote socket.readableLength RSS
Node v26.3.0 3 MB 64 KB 45 MB
main (367d939) 256 MB 256 MB 300 MB
this branch 3 MB 64 KB 45 MB

More reader shapes, largest readableLength seen:

Node main this branch
pause() in the middle of the stream, after 128 MB at full speed 64 KB 127 MB (all the rest) 512 KB
pipe() to a Writable that takes one chunk per ms, 32 MB 82 KB 24 MB 959 KB
pause() on each chunk, resume() on the next setImmediate, 256 MB 64 KB 187 MB 959 KB

After resume() every byte arrives in all cases. The bound is the high water mark plus two reads of 512 KB: the read that fills the buffer, and one more that can be on its way to JS.

Why the cap is 512 KB (LIBUS_RECV_BUFFER_LENGTH). The read loop in loop.c reads again only after a read that filled its buffer (or for an EOF or error hint). Without the cap, a pause in the middle of a fast stream holds 1.5 to 3 MB: one turn of the loop takes up to six reads before the task with the first chunk runs. A tunnel with small messages never reaches the cap and pays no epoll_ctl. A bulk transfer pays one pause and one resume per loop turn. Release builds of main and of this branch, alternating runs: 4 GB to a flowing reader ran at 2.27 to 2.49 GB/s on main and 2.39 to 2.88 GB/s here. 100000 echo round trips of 1 KB ran at 60k to 70k per second on main and 55k to 69k here. Both are within the noise.

Node's rule. Node stops the handle only when push() returns false (stream_base_commons.js#L191-L198) and starts it from _read() (net.js#L779-L792). pause() alone stops it only for an onread socket. So a paused socket still reads up to the high water mark, and still sees the peer's FIN below it. #39830 gave node:net the same rule. resume() of this class already called into native before super.resume(). It now starts the reads only while the buffer is below the high water mark. The first version of this change started them on every resume(), and pipe() to a slow destination then held 4 to 12 MB, because pipe() calls resume() on every 'drain' while the buffer is still full.

The three faults that a pause exposed.

  • us_socket_pause() and us_socket_resume() arm the writable event. In tunnel mode every writable event reached onDrain(), which told JS, which emitted 'drain'. A pipe() source that waits for 'drain' then writes on into a socket that has not drained. With a reader that pauses on each chunk and 32 MB, the socket emitted 9 'drain' events (Node: 0). onDrain() now returns when no bytes were buffered. JS waits for a drain only when handle.write() returned false, and that is exactly when bytes are buffered. This also removes one 'drain' that main emits on every https tunnel.
  • HttpContext::onData drops all data on a socket that is shut down. A tunnel is half-open: a client that gets the server's FIN can still send, and Node delivers it (a 100 KB write after the server's end(): Node 100000 bytes, main 0, this branch 100000). With read backpressure, bytes that the peer sent long before can still be unread when end() runs, so without this change a reader that was behind at end() would get a cut stream and a clean 'end'. Bun.serve is unchanged (if constexpr (IsNodeHttp)).
  • wss.handleUpgrade() can run later than the 'upgrade' event (an async check of the request). Bytes that arrive in between go to the socket of the listener. At the high water mark the reads stop, and the WebSocket that adopts the connection inherited the pause. onSocketUpgraded now resumes the adopted socket.

Flood prevention. endFloodPreventionPause() is now the one place where flood prevention resumes reads (two copies before). It skips the resume when the tunnel paused the reads. In tunnel mode readStart() resumes the socket even while the pause from before the tunnel is still in place: Node reads a tunnel at once too (64 KB buffered while the response ahead is unread). socket.end() has a pause and resume of its own for kqueue. It leaves alone a tunnel that paused its reads.

End of stream. readStop() does nothing once onData() got the end of the stream. On kqueue the FIN comes with the last data, so the chunk tasks run after native has the EOF. A pause then would never end (no _read() reaches native after the end), and socket.end() would skip its kqueue cycle. This case is reasoned, not reproduced: it needs kqueue.

What fails without each part (checked by removing it, debug build):

  • no readStop()/readStart() (main): 11 of the 13 new tests.
  • readStart() on every resume(): the first test, resume() with a full buffer reads on (64 KB grows to 160 KB).
  • no cap on the queued bytes: the mid-stream test, 1.5 MB against the bound of 64 KB + 2 reads.
  • no change in onDrain(): the drains count of the tests (on main the https rows already see 1 'drain').
  • no gate in endFloodPreventionPause(): the flood prevention test, the buffer grows from about 100 KB to between 190 KB and 1 MB.
  • no change in HttpContext::onData: the end() test gets 128000 of 1048576 bytes.
  • no check in socket.end(): the end() test, the buffer grows from 64 KB to 160 KB.
  • no resume in onSocketUpgraded: the handleUpgrade() test times out.

Found next to this, not fixed here. Both are on main.

  • A tunnel socket that closes natively with unread bytes in its buffer drops them: #onClose calls destroy(), so 'close' comes after push(null) with no 'end', and stream.pipeline() never calls back. It needs a reader that is behind at the close (pipe() to a slower destination, for await). On main: the peer sends 8 MB and FIN, the server pipes to a slow sink and calls end() 300 ms later: Node gets 8388608 bytes and the callback, main gets 1373184 and no callback, this branch gets 8154112 and no callback. This change makes the window smaller in bytes, and it adds one order of events that reaches the bug: the server calls end() first and the peer keeps sending (main drops those bytes in uWS and ends cleanly, with nothing delivered). Bun's net.Socket does not destroy on a native close (SocketHandlers.close in net.ts). That is the model for a fix in #onClose, with its own questions (an error close, a socket that nobody reads).
  • socket.pause() on a CONNECT with a Content-Length also arms the request body buffer (do_pause has no tunnel guard, do_resume has one). From then on every tunnel byte is copied into that buffer and never drained: +211 MB RSS for 200 MB through the tunnel, on main and here.

A change for a client that breaks RFC 6455 4.1. Bun's builtin ws never hands the bytes that arrive before handleUpgrade() to the WebSocket. They stay in the buffer of the upgrade socket, on main and here, so a client that sends frames before the 101 loses them. On main the connection stays usable, because the socket takes all of them. Here the reads stop at the high water mark, inside a frame, so the WebSocket that adopts the connection starts in the middle of a frame and closes. With 300 KB of early frames and a hello after the 101, main echoes hello and this branch closes with 1006. With 30 KB both echo hello. The fix for both is to give the buffered bytes to the WebSocket at the adoption. It is not in this PR.

Tests. The five tests of the matrix run for http and https CONNECT. An open Upgrade tunnel runs the same code, so the Upgrade rows run the first test only. The tests stay serial: they share one thread and spend their time in TLS and copies, so concurrent runs would not shorten the file, and would push each test toward its 5 s limit under ASAN.

Self-review. The review found the resume() fault above, and in the tests: a wait helper that gave up silently, a flood prevention test that did not check that flood prevention held the reads, a list of sockets shared between tests, missing failure exits, and filler bytes that were not valid WebSocket frames. All are fixed. It also found the two bugs above. One more report, a pause of the request body of an Upgrade that outlives the switch to tunnel mode, came from the code alone. My attempt to reproduce it gives the same result on Node, so it is not counted.

Suites run with the debug build. node-http-connect.test.ts, node-http.test.ts, node-http-with-ws.test.ts, node-http-req-socket-pause.test.ts, node-http-backpressure.test.ts, node-http-backpressure-max.test.ts, node-http-server-socket-end-drain.test.ts, test/js/first_party/ws/ (ws, ws-proxy, ws-upgrade-events), serve.test.ts, and all 498 vendored test-http-* and test-https-* files in test/js/node/test/parallel. The failures are the same with a debug build of main: test-http-agent-keepalive.js and test-https-timeout.js (timing), should use correct error when using a root range port in serve.test.ts (the container runs as root), and tests should run on bun in node-http-connect.test.ts (its sub-suite takes about 5 s under a debug build, and the limit is 5 s).


no test proof · iteration 2 · platform-specific test(s) that do not run on this machine, deferring to CI, which covers all platforms: test/js/node/http/node-http-connect.test.ts

…is full

The socket of a 'connect' or 'upgrade' listener pauses the raw socket when
push() returns false and resumes it from _read(), like net.Socket. resume()
resumes it only while the buffer has room. Native hands each chunk to JS in
a task, so onData() also pauses the socket while 512 KB wait for their
task: the read loop runs before JS can answer.

onDrain() tells JS only when bytes were buffered, because a read pause or
resume arms the writable event too. A tunnel still receives data after
socket.end(). The end of flood prevention leaves a tunnel alone that paused
its own reads. A WebSocket that adopts the connection resumes it.
@robobun

robobun commented Sep 19, 2026 •

Copy link
Copy Markdown
Collaborator Author
Updated 6:19 PM PT - Sep 19th, 2026

❌ @robobun, your commit 1c6addd has 1 failures in Build #118679 (All Failures):


🧪   To try this PR locally:

bunx bun-pr 43570

That installs a local version of the PR into your bun-43570 executable, so you can run:

bun-43570 --bun

@coderabbitai

coderabbitai Bot commented Sep 19, 2026 •

Copy link
Copy Markdown
Contributor

Review Change StackReview Change Stack

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info
⚙️ Run configuration

Configuration used: Repository: oven-sh/bun/.coderabbit.yaml

Review profile: ASSERTIVE

Plan: Essentials

Run ID: 5326b6ec-7dcf-4fc5-a078-a067e26961cc

📥 Commits

Reviewing files that changed from the base of the PR and between dc10831 and fd0783d.

📒 Files selected for processing (1)
  • test/js/node/http/node-http-connect.test.ts

Included review availability: Your plan provides up to 10 included reviews per hour; 1 remains after this review.


Walkthrough

The PR adds bounded read-ahead and coordinated pause/resume handling for HTTP and HTTPS CONNECT tunnels. It updates native socket state, JavaScript stream controls, upgrade handling, and regression tests.

Changes

CONNECT tunnel backpressure

Layer / File(s) Summary
Native tunnel read state
packages/bun-uws/src/HttpContext.h, src/jsc/bindings/node/JSNodeHTTPServerSocket.*, src/jsc/bindings/NodeHTTP.cpp
Native sockets track tunnel pauses, queued bytes, end-of-stream state, flood-prevention pauses, and upgrade read release.
JavaScript stream controls
src/js/node/_http_server.ts, src/jsc/bindings/node/JSNodeHTTPServerSocketPrototype.cpp
Readable-buffer state controls native reads. The socket prototype exposes readStop() and readStart().
Tunnel and upgrade regression coverage
test/js/node/http/node-http-connect.test.ts
Tests cover bounded buffering, pause and resume behavior, slow destinations, termination ordering, payload integrity, and WebSocket upgrades.

Suggested reviewers: cirospaciari

Priority: ➖ Normal

🚥 Pre-merge checks | ✅ 4
✅ Passed checks (4 passed)
Check name Status Explanation
Linked Issues check ✅ Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check ✅ Passed Check skipped because no linked issues were found for this pull request.
Title check ✅ Passed The title clearly and concisely describes the primary change: adding read backpressure to node:http CONNECT and Upgrade tunnel sockets.
Description check ✅ Passed The description explains the problem, implementation, verification results, tests, and known limitations. It does not use the exact template headings, but it provides the required information in equiv…

Comment @coderabbitai help to get the list of available commands.

@robobun

robobun commented Sep 19, 2026 •

Copy link
Copy Markdown
Collaborator Author

Status

  • Reproduced on main (367d939) and on Bun 1.4.3-canary: a 'connect' listener calls socket.pause(), then a net client writes 256 MB. socket.readableLength reaches 256 MB and RSS 300 MB. Node v26.3.0 holds 64 KB and the client stalls after 3 MB. The Notes of the PR body have the numbers for the other reader shapes (pipe() to a slow destination, a pause in the middle of a stream).
  • With this PR the socket holds 64 KB and RSS is 45 MB. After resume() every byte arrives.
  • Test: bun bd test test/js/node/http/node-http-connect.test.ts. 11 of the 13 new tests fail on main.
  • CI: the diff is green. The new tests passed on every lane in both builds (Linux, musl, ASAN, macOS, Windows). The red jobs are tests that this PR does not touch:
    • Build 118679 (latest): 180 of 181 jobs passed. test/js/bun/module-graph/module-graph-gc.test.ts fails on windows 2019 x64 (a count of live ModuleGraph objects after GC, 3 against an expected 2). It passed on that lane in build 118595.
    • Build 118595 (same tree): all lanes passed except debian 13 x64-asan, where four leak tests timed out (inspect-error-leak, html-rewriter-leak, require-cache, sourcetextmodule-leak). The same tests failed on that lane in builds of other PRs at that time (118585, 118596). They pass in build 118679.

@claude claude Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nothing blocking. The comments below are optional suggestions. There is no need to push a fix for them before merging.

Beyond the inline findings, I also checked the lifetime of the self capture in the onData postTask now that didDeliverQueuedTunnelBytes runs after the JS callback: onClose() nulls socket first and only clears strongThis in its own later-queued task, so isTunnel() returns false and no freed us_socket_t is touched. The HttpContext::onData tunnel bypass falls through to the existing returnedData path, so the us_socket_ref/unref pair stays balanced. Both flood-prevention resume sites now go through the one endFloodPreventionPause helper, and releaseTunnelReadsForUpgrade clears both holds before resuming.

Extended reasoning...

Findings were reported inline, so this is only the ruled-out note. I read the native diff (JSNodeHTTPServerSocket.cpp/.h, Prototype.cpp, NodeHTTP.cpp, HttpContext.h) and the _http_server.ts changes. The postTask lambda already captured raw self before this PR; the new scope-exit call runs synchronously after callStoredCallback with no allocation in between, and every native path it reaches (applyTunnelReads -> isTunnel) checks socket non-null and !us_socket_is_closed, which onClose() guarantees by nulling socket before posting its own task. The HttpContext.h change only widens the condition on an early return; the fall-through path is the pre-existing one that unrefs via returnedData. The two former copies of the flood-prevention resume were replaced by one helper, so the SSL/non-SSL and spill/no-spill variants get the same gate.

Comment thread src/jsc/bindings/NodeHTTP.cpp Resolved
…T only

An open Upgrade tunnel runs the same code as a CONNECT tunnel. The other
tests of the matrix move 8 to 24 MiB each on one thread.

@claude claude Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code review found no issues

No high-confidence issues detected in this change.

Still open from earlier reviews (1):

  • Unresolved: 1 minor or pre-existing.

@claude claude Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code review found no issues

No high-confidence issues detected in this change.

Jarred-Sumner added a commit that referenced this pull request Sep 26, 2026
…inish, lifecycle) (#43557)

One pull request for the open `node:http` server pull requests. Each
root cause is fixed once, and each pull request's tests are carried
over. Node is the reference: every scenario was run under Node and under
Bun from one script, and the outputs were compared.

Fixes #4733
Fixes #18613
Fixes #40350
Fixes #43155
Fixes #43297
Fixes #43513
Fixes #43527
Fixes #25632
Fixes #31301
Fixes #43027
Fixes #43163
Fixes #43342
Fixes #43344
Fixes #43370
Fixes #43490
Fixes #43512
Fixes #43519

Each of these has a repro that is wrong on Bun 1.4.3, right on this
branch, and the same as Node.

| Issue | Not closed by this PR, because |
| --- | --- |
| #30501 (msal-node keeps Bun alive at exit) | Probably fixed. The repro
copies the teardown of msal-node. The package itself was not run. |
| #14430 (yarn: "does not support SSL") | Probably fixed.
`response.hasOwnProperty("socket")` is now `true`. yarn itself was not
run. |
| #39681 (`server.setTimeout` callback after destroy) | Probably fixed.
The repro is the deterministic case of #39686. The script in the issue
depends on timing and on Windows. |
| #43455 (`req.complete`, nine flows) | Partially addressed. Flows 1, 2,
3, 5 and 7 are fixed, and flow 6 was already right. Flow 8 (a socket
timeout while the body of an Upgrade request arrives) and flow 9 (a
request that `stream.pipeline()` destroyed never reports `complete`) are
not. Flow 4 differs only in `_readableState.ended`. |

### What changes for users

| Area | Before | After (same as Node) |
| --- | --- | --- |
| `req.pause()` | The socket stops at once. `req.complete` stays `false`
for a small body. | The body is received until the buffer is full. Then
the socket stops. |
| `res.end()` before the body arrives | `req` gets `'end'` and `'close'`
at once, and the body is lost | The request completes when its body
really ends |
| `res.destroy()` in the middle of a body | `'end'` with bytes missing |
`aborted`, then `ECONNRESET` |
| `socket.destroy()` inside the `'request'` listener | The body that
came with the head is dropped | That body is still delivered |
| `'finish'` and the `end()` callback | Fire when `end()` buffers the
bytes | Fire when the last bytes have left the socket |
| A response that closes the connection | The server half-closes and
waits for the peer | The socket closes right behind the FIN |
| `'drain'` after a later write flushed the backlog | Lost. `pipe(res)`
could hang. | Emitted |
| A pipelined request whose body continues after the previous response
ends | Body dropped, no response, `server.close()` hangs | Delivered |
| CONNECT and Upgrade tunnel sockets | Keep reading when paused or full
| Stop reading. `_read()` starts them again. |
| A tunnel write that waits for a drain when the client goes away | Its
callback, the callbacks of the writes behind it and the `end()` callback
never run | They run with an error before `'close'` |
| Upgrade request with a body, paused in its listener | The body flows
away | The request keeps its body |
| `ws` on a reused keep-alive socket | Writes after the Upgrade could
stall | Sent |
| A raw `socket.write()` behind a response that still drains (the 400
for a bad pipelined request, the reply of a `'clientError'` listener) |
Lands in the middle of that response | Sent after it |
| `server.close()` | Could report closed while connections were open |
Waits for every connection. An idle tunnel does not keep the process
alive. |
| `closeAllConnections()` on a listening server | Also stops the
listener and destroys tunnels and WebSockets | Destroys only the HTTP
connections |
| `Proxy-Connection: close` (node:http only) | Ignored. The connection
stays open. | Ends the connection, like `Connection: close` |
| A response larger than 16 KB, also in `Bun.serve` and over TLS | Up to
4 `send()` calls for each chunk. Slower than Node in most cases. | One
write for the writes of one tick. 1.1x to 2.7x the requests per second
of main, and faster than Node. |
| `socket.destroy()` and then `res.end()` in a listener | (this PR,
earlier) `req` ended as if it were complete | `'aborted'`, then
`ECONNRESET` |
| `emit('connection')` or http2 `allowHTTP1`: the response ends while
the listener still reads the body | The rest of the body is dropped |
The body is complete |
| `httpValidation: "relaxed"`, `Content-Length` or `Transfer-Encoding`
in trailers | Accepted | `HPE_INVALID_CONTENT_LENGTH`,
`HPE_INVALID_TRANSFER_ENCODING` |
| `req.complete` inside `'connect'`, and inside `'upgrade'` without a
body | `false` | `true` |
| `optimizeEmptyRequests`: `socket.parser.incoming` after the response |
Keeps the request alive on an idle connection | `null` |
| A HEAD or OPTIONS request with `Content-Length` | The body is dropped,
and `req.complete` is `true` before it comes | The request has its body
|
| `res.end(chunk)` after the client went away | `finished` and
`writableEnded` stay `false`, no `'prefinish'` | The response ends |
| An HTTP/1.0 request with an `Expect` header | `100 Continue`,
`'checkContinue'`, `'checkExpectation'` or a 417 | A plain `'request'` |
| The idle sweep of `close()` and `closeIdleConnections()` | Could
destroy a connection that was still receiving a request, or whose
response was still draining | Closes only idle connections |

### Design

| Piece | What it is |
| --- | --- |
| Request body state | `None / Pending / Complete / Aborted / Upgraded /
Detached`. Only the last chunk sets `Complete`. One function,
`leave_pending`, is the only other way out of `Pending`. |
| Read flow control | One path: `push()` returning false stops the
socket, `_read()` starts it. Both native pause buffers are removed: no
read is copied and replayed. |
| "This read is parsed" signal | `notifyWhenReadParsed()` sets a uws
state bit. uws delivers a `readParsed` event after the read. It replaces
a `setImmediate`. |
| Close during a parse | One uws bit defers a close to the end of the
current message. |
| Response finish | A response is finished when it has ended and the
socket has fully drained. |
| Idle connection | One rule, `HttpResponse::closeIfIdle()`. A
connection is idle when it receives no request (head or body) and no
response is in flight, queued or undrained. The sweep of `close()` and
`closeIdleConnections()` both use it. |
| Idle tunnel | A tunnel at read EOF with nothing left to send. uws
reports it to the server through the connection filter (`-3`, `+3`,
`-4`). It still counts for `'close'`, but it does not hold the event
loop, like a libuv handle in that state. |
| Server `'close'` | One native close promise per `listen()`. `close()`
records whether its sweep left nothing open. Then a `listen()` in the
same tick cannot hold `'close'` back, as in
`net.Server._emitCloseIfDrained`. |
| Raw socket writes | While uws holds response bytes (its buffer, the
zero-copy tail of a `res.write()`, the cork buffer), a raw write goes
through `AsyncSocket::write`, the path a 1xx line takes. So the order on
the wire is the order of the calls. |
| Upgrade verdict | One scanner and one verdict, shared by the parser
and the dispatcher. |
| llhttp | Updated from 9.3.0 to 9.4.2, as Node v26.5.0 vendors it, plus
one local patch (see below). Node v26.5.1 and later vendor 9.4.3. That
update is not in this PR. |

The parser changes also tighten request framing so that it agrees with
llhttp in more cases. There is no new API surface.

### A pause holds from the next read

The copy of the rest of a read (`nodeHttpPausedSpill`), its replay from
a posted task and the nested parse are removed. Like in Node, the rest
of the read that caused a pause is still parsed, and the socket stops at
the next read. usockets reads up to 512 KB in one call. libuv reads 64
KB.

| One paused, unread request (client sends 64 MB) | Bytes held |
| --- | --- |
| Node 25.6 | 131,018 |
| This pull request | 524,234 |

The price is in one case. A client sends 512 KB of small pipelined
requests (19,418 of them) and never reads. Each handler answers with its
own 64 KB body:

| Handler | Runtime | Requests dispatched | RSS |
| --- | --- | --- | --- |
| Answers at once | Node 26.3 | 2,425 | +177 MB |
| Answers at once | main | 41 | +9 MB |
| Answers at once | This PR | 2,425 | +171 MB |
| Answers one tick later | Node 26.3 | 4,850 | +336 MB |
| Answers one tick later | main | 2,426 | +181 MB |
| Answers one tick later | This PR | 4,850 | +330 MB |

Release builds on Linux x64. This PR now does what Node does. main held
fewer responses, mostly for a handler that answers at once. On macOS one
read can return all 512 KB. There, Bun 1.4.3 already reached +951 MB for
the handler that answers one tick later, and Node reached +1,294 MB.
`server.maxRequestsPerSocket` bounds it.

### Performance

#### Responses larger than 16 KB are faster, and now faster than Node

On main, a response that did not fit the 16 KB uWS cork buffer released
the cork. After that, each piece was its own `send()`: the buffered
head, the chunk-size line, the data, the `\r\n` and the last chunk. Over
TLS, each 2-byte piece was also its own record. Two changes fix that,
for `Bun.serve` and for node:http:

| Change | Effect |
| --- | --- |
| A write that does not fit goes out with the cork buffer and its
framing in one vectored write | No copy is added. Over TLS, the records
of all the pieces share the write batch that one `SSL_write` loop
already had. |
| The cork buffer holds 128 KB, up from 16 KB. Only a write of 16 KB or
less is copied into it, as before. | Several writes in one tick go out
in one write, like in Node. A longer write still goes out without a
copy. |

The bytes on the wire are the same. The vectored write uses `sendmsg()`
with the flags that `send()` uses.

Write syscalls for one response:

| Response | Node 26.3 | main | This PR |
| --- | --- | --- | --- |
| 4 x `res.write(16 KB)` | 1 | 16 | 1 |
| 40 x `res.write(2 KB)` | 1 | 16 | 1 |
| `res.end(64 KB)` | 1 | 2 | 1 |
| 256 KB file, `.pipe(res)` | 4 | 16 | 5 |

Throughput (req/s, the mean of 2 rounds). Node v26.3.0, main
`97246d044e`, this PR `fe0ed1fbea`, with the method below:

| Case | Node | main | This PR | main / Node | PR / Node | PR / main |
| --- | --- | --- | --- | --- | --- | --- |
| http, 4 x `res.write(16 KB)` | 17,735 | 6,983 | 19,160 | 0.39x | 1.08x
| 2.74x |
| https, 4 x `res.write(16 KB)` | 11,720 | 6,110 | 15,510 | 0.52x |
1.32x | 2.54x |
| http, 40 x `res.write(2 KB)` | 9,879 | 6,140 | 14,980 | 0.62x | 1.52x
| 2.44x |
| https, 40 x `res.write(2 KB)` | 6,981 | 5,541 | 11,780 | 0.79x | 1.69x
| 2.13x |
| http, `res.end(64 KB)` | 18,535 | 16,528 | 19,889 | 0.89x | 1.07x |
1.20x |
| http, 256 KB file `.pipe(res)` | 2,684 | 2,375 | 2,719 | 0.88x | 1.01x
| 1.15x |
| https, `res.end(64 KB)` | 11,894 | 14,586 | 16,426 | 1.23x | 1.38x |
1.13x |
| http, GET hello (control) | 55,994 | 70,989 | 71,958 | 1.27x | 1.29x |
1.01x |

main was slower than Node in six of these eight cases. This PR is faster
than Node in all eight.

`Bun.serve`, measured on `a771572a8d`, before the larger cork buffer
(req/s, the mean of 2 rounds):

| Case | main | PR | Change |
| --- | --- | --- | --- |
| Direct stream, 4 x 16 KB | 6,826 | 12,479 | +83% |
| 64 KB string | 16,944 | 20,145 | +19% |
| TLS, 64 KB string | 15,045 | 16,892 | +12% |
| hello (control) | 83,957 | 83,137 | -1.0% |

These runs are on loopback, where the kernel send buffer is 2.6 MB and
the work of the receiver runs inside `send()`. That is the best case for
fewer writes. A new connection over a real network takes about 46 KB in
its first write on Linux. The rest waits in the socket buffer, as it
would after separate writes.

#### Small responses are unchanged

A small response is already one `recvfrom` and one `sendto` on both
builds. `perf` puts 66% of the time of a hello-world server in the
kernel, on both builds.

CI release builds on Linux x64: main `97246d044e` (the merge base)
against this PR `8834cd0787`. Both use the same WebKit. The server runs
on one pinned core. `oha` sends 64 connections for 5 s after a 2 s
warm-up. There are 2 rounds, and the order of the builds alternates.
"Change" compares the means of the two rounds.

Framework servers from `bun-perf-tester` (req/s):

| Server | main, round 1 | main, round 2 | PR, round 1 | PR, round 2 |
Change |
| --- | --- | --- | --- | --- | --- |
| express | 49,803 | 51,103 | 49,968 | 50,263 | -0.7% |
| fastify | 61,214 | 61,105 | 60,705 | 60,668 | -0.8% |
| node:http | 70,934 | 71,405 | 73,178 | 71,053 | +1.3% |
| elysia | 84,696 | 85,096 | 85,027 | 84,882 | +0.1% |
| `Bun.serve` | 89,020 | 89,099 | 88,168 | 88,373 | -0.9% |

node:http paths that this PR changes (req/s):

| Case | main, round 1 | main, round 2 | PR, round 1 | PR, round 2 |
Change |
| --- | --- | --- | --- | --- | --- |
| GET hello | 70,226 | 70,341 | 70,151 | 72,359 | +1.4% |
| POST, 16 KB body | 47,446 | 47,789 | 48,191 | 48,785 | +1.8% |
| 64 KB response in four writes | 6,885 | 6,894 | 6,834 | 6,868 | -0.6%
|
| Pipelined keep-alive, depth 8 | 94,063 | 93,294 | 92,909 | 93,809 |
-0.3% |

p99 latency (ms), the higher of the two rounds:

| Server | main | PR |
| --- | --- | --- |
| express | 1.94 | 1.91 |
| fastify | 1.52 | 1.55 |
| node:http | 1.11 | 1.12 |
| elysia | 0.98 | 0.96 |
| `Bun.serve` | 0.82 | 0.83 |

RSS (MB), one pass of 8 s of load:

| Server | Build | Start | Under load | 5 s idle | 15 s idle |
| --- | --- | --- | --- | --- | --- |
| express | main | 39 | 94 | 61 | 57 |
| express | PR | 40 | 92 | 60 | 57 |
| fastify | main | 41 | 91 | 58 | 55 |
| fastify | PR | 41 | 91 | 58 | 55 |
| node:http | main | 20 | 64 | 43 | 40 |
| node:http | PR | 20 | 65 | 45 | 41 |
| elysia | main | 28 | 46 | 36 | 35 |
| elysia | PR | 29 | 46 | 37 | 36 |
| `Bun.serve` | main | 14 | 30 | 22 | 22 |
| `Bun.serve` | PR | 14 | 30 | 22 | 22 |

Every change is within 2%. fastify and `Bun.serve` hello are lower in
both rounds, by about 1%. `Bun.serve` hello shows the same -1.0% in the
control row above, so a small real cost there is possible. The RSS pass
ran at the same time as the throughput runs, on other cores. The commits
after `8834cd0787` change tests and add one version check to the
node:http dispatcher. They were not measured.

### Supersedes

| Theme | Pull requests |
| --- | --- |
| Request body | #43592 #43579 #38196 #43518 #43602 #43408 #43427 #43597
#43555 #43456 #43466 |
| Tunnels | #43570 #43485. #43596 is a duplicate of #43570. |
| Parser | #43182 #43161 #43326 #43327 #40505 #43363 #42532 #42194 |
| Response write | #39386 #43371 #43548 #43499 #43496 #43464 #42008
#43549 |
| Response finish | #40351 #43021 #41822 #43473 #42068 #35207 #43425
#43503 |
| Lifecycle | #43413 #39686 #43028 #42727 #42622 #42610 #35837 #35839
#37825 #37749 #43376 #35268 |
| JS API | #41691 #41738 #38036 #42462 #36527 #39718 #37964. #42947
merged on its own. |

The close drain, `resetAndDestroy()`, the pending write callback
handling and the response `'close'` ordering come from #42622 and #42727
by @steipete. The diagnosis and the tests for the stalled `ws` writes
come from his #42610.

Not included:

| Pull request | Reason |
| --- | --- |
| #33061 | main already enforces `headersTimeout` and `requestTimeout` |
| #41672 | It makes `http.createServer({ key, cert })` stop serving TLS.
That needs a product decision. |
| #37543 | A type refactor with no tests and no user-visible change |
| #35465 | It makes `http.Server` extend `net.Server`. Only the
prototype chains were joined. The `net.Server` constructor never ran, so
`_handle` and `_connections` were `undefined`, and
`_emitCloseIfDrained()` emitted `'close'` on a listening server. The
server is backed by uWS, not `node:net`. |
| The `AutoFlusher` removal in #42622 | It makes `flushHeaders()` flush
at once. That is a performance change with no relation to the rest. |

### Tests

| Check | Result on a debug build (macOS arm64) | Head |
| --- | --- | --- |
| Every test file that this PR touches (28 files) | 1,866 pass, 2 fail.
The 2 failures are `serve.test.ts` "bounds memory when proxying ... to a
stalled client". They fail the same way on a debug build of main. |
`83af4da4a3`, run before the last commit of main came in |
| `test/js/third_party/express` (9 files) and the `body-parser` test |
299 pass, 0 fail | `83af4da4a3`, run before the last commit of main came
in |
| Node 25.6 against Bun, 32 scenarios from two scripts (event order,
framing, lifecycle) | No regression against Bun 1.4.3 | `0ff1a23f61` |
| Every vendored Node `test-http-*` and `test-https-*` file, plus the
`test-net-*` and `test-tls-*` files for pause, write, end and close |
535 of 537 exit 0. `test-http-agent-keepalive.js` and
`test-https-timeout.js` fail on that debug build. Both pass on every CI
lane. | `daee05fcfd` (before the rebase) |
| The tests that depend on what the kernel takes in one send, on Windows
Server 2019 x64 and Windows 11 arm64 | pass | `3eef223328` (x64),
`a29289bcc1` (arm64) |
| CI build 120191 (Linux, macOS and Windows, release and ASAN) | every
lane passed | `daee05fcfd` (before the rebase) |

Each new test fails on Bun 1.4.3, or on the commit before its fix for a
fault that this branch introduced.

The two tests over the limit are `node-http-connect.test.ts` ("tests
should run on bun") and `node-http-syscall-fault.test.ts` ("racing a
queued drain"). Each starts a debug subprocess that needs more than 5 s
on this machine. Both pass on CI.

### Changes in the last push

The branch is rebased on main (`daee05fcfd` was the head before). It is
now linear.

Four regressions against main, each with a test that fails without its
fix:

| Case | main | Before this push | Now (same as Node) |
| --- | --- | --- | --- |
| `emit('connection')` or http2 `allowHTTP1`: `res.end()` on a request
that nobody reads | `'end'`, `'close'` | No events | `'end'`, `'close'`
|
| The same server, an unread 32 MB body | 0 bytes held | 32 MB held | 0
bytes held |
| A NUL in a header value with `httpValidation: "relaxed"` (client,
`HTTPParser`, `emit('connection')` server) | Accepted | The process
spins forever | `HPE_INVALID_HEADER_TOKEN` |
| `Connection: close`, body in the same read as the head, a 20 KB
response before the body is read | `'end'` with an empty body | No
events on `req` | `'end'` with the body, `'close'` |
| An empty line on an idle keep-alive connection, then `server.close()`
| 0 s | About 6 s | 0 s |

| Fix | Where |
| --- | --- |
| The finish listener of a fallback connection dumps an unread request,
like Node's `resOnFinish` | `http1_server_fallback.ts` |
| llhttp patch: `llhttp__internal__c_test_lenient_flags_20` is false for
a NUL. The relaxed state does not consume a NUL, and the next state sent
it back there. 9.4.3 has the same loop. | `llhttp.c`, noted in its
`README.md` |
| A node:http socket that `onData` is parsing gets the close gate of
`onData`, also when a large write released the cork | `HttpResponse.h`
`uncorkCompletedResponse()` |
| A read that starts no message leaves an idle connection idle |
`HttpContext.h` `onData` |

The open review threads are fixed in `8834cd0787`:
`closeAllConnections()`, `Proxy-Connection: close`, five comments cut to
one line, and the test of two overlapping listeners, which now waits on
events. With the generation gate in `emitCloseServer` removed, that test
fails in both cases. `AsyncSocketData` keeps its bools together, which
takes it from 56 to 48 bytes per socket.

`http.Server` no longer extends `net.Server` (see "Not included"). The
special case for it in `Ipc.ts` is gone too. `child.send(msg,
httpServer)` still throws `ERR_INVALID_HANDLE_TYPE`, and its test stays.

<details><summary>Changes since the first revision (2988a61)</summary>

Merged with main at `c8e1f6fa5b`. The one conflict was #43708
(`req.socket` emits `'end'` and `'error'`). Its state bit
`HTTP_NODE_PEER_ENDED` moved to bit 22, because bit 19 is
`HTTP_NODE_NOTIFY_READ_PARSED` here. Its 15 tests run in
`node-http-server-abort-events.test.ts` next to the tests of this branch
(103 pass).

CI on `2988a610c` had ten red tests from four causes. They are fixed:
- `ed882e5e99`: `write()` to a response without a body (HEAD, 204) does
not wait for unsent bytes.
- `811f817704`: an idle tunnel does not hold the event loop after
`server.close()`. Four vendored Node tests timed out on every platform.
- `0697deec11`, `d428824c08`, `7aec062d14`: the write callback tests use
a body that backs up a loopback socket, and accept what Winsock does.
- `c7af1c1615`: two tests from main asserted the old `close()` contract.

Review findings, each reproduced against Node v26.3.0 and fixed with a
test that fails without the fix:
- `d4d2b783ea`, `2cc79939bb`, `45141abca9`, `9a63dbc470`, `80a23a1822`:
the idle rule. A keep-alive connection is idle again when its body ends
after its response. A connection that owes a queued pipelined response,
that still receives a request head or body, or whose response still
drains is not idle.
- `87d8904aa3`, `7eca4f7806`: `close(cb)` followed by `listen()` in the
same tick reports `'close'`, also for an https server whose only
connection was idle.
- `3fd7b25382`: a paused pipelined request behind a response that still
drains stops the connection. The first revision read 512 MiB of 512 MiB
into memory.
- `2d1a5e2d8d`, `33e4683a8a`, `539cb41eb9`, `197c7dfc29`, `0490e3540b`:
raw socket writes stay behind every unsent response byte. The cases were
a CONNECT pipelined behind a response that still drains (its `200`
landed at offset 2.6 MB of a 64 MiB body), a zero-length tunnel write
(it hung the tunnel), the 400 replies above, the zero-copy tail of a
large `res.write()`, the cork buffer, and Windows 11, where the kernel
takes the whole response and refuses the next send.
- `0abd39c133`: an upgrade from the request's `'end'` listener keeps the
body bytes out of the WebSocket. The connection closed with 1006 right
after the 101.
- `5aafc61aa6`: the callback of a small `res.write()` that the kernel
refuses at the uncork runs on the drain. Reproduced on Windows 11 only.
- `a29289bcc1`: a tunnel write that waits for a drain settles its
callbacks when the connection closes.

Known differences from Node that this PR leaves:
- A handler that calls `res.end()` and then `server.close()` closes its
keep-alive connection at once. Node waits for the `keepAliveTimeout`.
- A pipelined Upgrade behind a response that still drains is served as a
plain request.
- An `end()` on a tunnel with no write pending, while the response
before the CONNECT still drains, closes both directions after the flush.
Node half-closes.
- A raw `req.socket.write(big)` and `req.socket.end()` with no
`res.end()` sends every byte but no FIN. main loses bytes here.
- A CONNECT socket that is given back with `server.emit('connection',
socket)` answers only the first of several pipelined requests. main
answers none.
- A large write from an `'upgrade'` listener stalls while the body of
that Upgrade request is still pending. A second `listen()` on a
listening server does not throw. Both are the same on main.
- A tunnel write that fails because the client went away fails its
callbacks but emits no `'error'`. Node emits `ECONNRESET`. A new
`'error'` could end a process that has no listener for it.

</details>

<!-- robobun:evidence:begin -->

---

**no test proof** · iteration 8 · platform-specific test(s) that do not
run on this machine, deferring to CI, which covers all platforms:
test/js/web/fetch/fetch.stream.test.ts, test/js/node/url/url.test.ts,
test/js/node/tls/tls-syscall-fault.test.ts,
test/js/node/net/node-net-server.test.ts,
test/js/node/http/node-http.test.ts,
test/js/node/http/node-http-syscall-fault.test.ts,
test/js/node/http/node-http-server-close-drain.test.ts,
test/js/node/http/node-http-connect.test.ts,
test/js/node/http/node-http-backpressure.test.ts,
test/js/node/child_process/child_process_ipc_handle.test.ts,
test/js/bun/http/serve.test.ts,
test/js/bun/http/serve-syscall-fault.test.ts,
test/js/bun/http/bun-server.test.ts

<!-- robobun:evidence:end -->

---------

Co-authored-by: Jarred Sumner <jarred@jarredsumner.com>
Co-authored-by: autofix-ci[bot] <114827586+autofix-ci[bot]@users.noreply.github.com>
@Jarred-Sumner

Copy link
Copy Markdown
Collaborator

Superseded by #43557, which is merged (5d5f03f). It fixes this once for the whole node:http server and carries the tests over.

@robobun

robobun commented Sep 26, 2026

Copy link
Copy Markdown
Collaborator Author

Confirmed on main at 5d5f03f with a debug build.

  • The repro of this PR (a paused 'connect' socket, 256 MB from the client): the socket holds 125 KB and the client stalls after 3 MB. Bun 1.4.3-canary.1+367d939d9 holds 256 MB.
  • pipe() to a slow destination holds 512 KB at most (24 MB before).
  • The 13 tests that were carried over pass: bun bd test test/js/node/http/node-http-connect.test.ts -t "read backpressure|handleUpgrade".
  • The copy of tunnel bytes for a CONNECT with a Content-Length (named in the notes here) is gone too.

One bug from the notes of this PR is still on main, so it has an issue now: #44034. A tunnel socket that closes with unread data drops that data and emits no 'end', and stream.pipeline() never calls back.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants