Repository navigation
Conversation
…ODY_ALREADY_USED The used-body guard added in #33118 covers Body.Value.Used but a Response whose ReadableStream body the handler already consumed stays Body.Value.Locked with a disturbed stream in the JS-side cache. The server only checked is_locked() on that stream, so a disturbed-but-unlocked stream (await resp.text(), or getReader()+read()+releaseLock()) fell through to do_render_stream and produced a silent empty 200, never invoking error(). Check is_disturbed() on the stream and route it to the same ERR_BODY_ALREADY_USED error path as Body.Value.Used.
|
Status: closed, and not reopened.
|
WalkthroughChangesResponse body reuse handling
Suggested reviewers: 🚥 Pre-merge checks | ✅ 4✅ Passed checks (4 passed)
Comment |
There was a problem hiding this comment.
LGTM — closes the ReadableStream gap in the existing ERR_BODY_ALREADY_USED handling from #33118.
What was reviewed:
is_disturbed()is a plain C++ field read (stream->m_disturbed, WebStreamsExports.cpp:115) — no user-JS re-entry or exception concerns.- Cleanup via
response_body_readable_stream_ref.deinit()matches the siblingSource::Invalidearly-return;is_aborted_or_ended()was already checked at the top of theLockedarm. - Error construction and
run_error_handlercall are byte-identical to the existingBody::Value::Usedarm. - Tests extend the existing file with an
it.eachmatrix (start/pull sources, partial-read+releaseLock) and assert exact error code/name/message.
Extended reasoning...
Overview
The PR adds a single is_disturbed() guard in do_render_with_body's Body::Value::Locked arm (RequestContext.rs:3167-3179), routing consumed-ReadableStream response bodies to the same ERR_BODY_ALREADY_USED error handler that already fires for Body::Value::Used (string/Blob/Uint8Array bodies, added in #33118). The test file gains an it.each matrix over five body kinds plus a partial-read case.
Security risks
None. This tightens behavior — a case that previously silently served an empty/truncated 200 now surfaces as a 500 via the user's error handler. No new input parsing, no auth/crypto, no untrusted-data handling.
Level of scrutiny
Low-to-medium. The change is 14 lines of Rust that mirror two adjacent sibling patterns exactly: the error object and run_error_handler call are copied from the Body::Value::Used arm at line 3122, and the .deinit() cleanup matches the Source::Invalid branch at line 3202. ReadableStream__isDisturbed (WebStreamsExports.cpp:115) is a dynamicDowncast + bool field read — it cannot throw or run user JS, so no exception-scope or re-entrancy concerns. The is_aborted_or_ended() guard at the top of the Locked arm (line 3145) already covers this path, matching the existing is_locked() branch which also doesn't re-check.
Other factors
- The new check is placed before the existing
is_locked()check, which is correct ordering: a fully-consumed stream is disturbed but not locked, andBODY_ALREADY_USEDis the more specific/helpful error. - Tests are in the right file, use
port: 0/await using, assert exact{code, name, message}, and the PR description confirms they fail on system bun and pass on the debug build. - The partial-read test explicitly guards against the old "serve the unread remainder" behavior with a comment and exact-body assertion.
- No prior reviewer comments to address; only a robobun status note in the timeline.
…42609) ### Problem - `Bun.serve()` sends a second `Response` around an already sent `ReadableStream` as a `200` with an empty body. The `error` handler does not run. - A native sink consumer (`Bun.serve()`, a `fetch()` upload, `Bun.write()`, `Bun.spawn()` stdin, S3, `HTMLRewriter`) pumps a JS-backed stream in `BunStreamSource.cpp`. At the end `rsisFinally` (`:853`) releases the reader and `directStreamOnClose` (`:662`) drops the lock: `locked === false`. A `type: "direct"` stream, so every async iterable body, is not disturbed either. `RequestContext.rs:3152` checks the lock alone. - Found by a comparison with node v26.3.0, not by a user report. ### Fix - `rsisBegin` and `readDirectStream` call the new `JSReadableStream::markConsumedAsBody()` once the pump holds the stream. The stream stays locked after the pump lets go. - Behaviour change: after such a consumer took a stream, `getReader()`, `tee()`, `pipeTo()` and `cancel()` fail with a `TypeError`. Body mixin methods (#42116) and natively wired sinks (#42477) already do this. Neither is released. - `pipeTo()`, async iteration and `Bun.readableStreamTo*()` do not reach these pumps and still unlock. - Verified: `web-stream-state.test.ts` (30 new tests fail on main), `body.test.ts` (12), `serve-reused-response.test.ts` (2). Self-reviewed: 3 concerns, 2 addressed, 1 out of scope (Notes). ### Background - A JSSink is a native sink (`HTTPResponseSink`, `FileSink`). `JSSink::assign_to_stream` pumps a stream into it. - `readStreamIntoSink` reads through a default reader. `readDirectStream` gives the sink to the `pull()` of a direct stream. - `m_consumedAsBody` (#42116) is a bit that `isReadableStreamLocked()` includes and nothing clears. The fetch spec never releases the reader of a body. <details><summary>Notes</summary> **The rule.** A consumer that takes a stream as a body keeps it locked: the body mixin methods (#42116), `to_any_blob` lifts (#42516), natively wired `ByteStream`/`FileReader` sinks (#42477), and now the two pumps. A reader at the stream level releases its lock as the Streams spec says. Checked on this branch: `pipeTo()`, `pipeThrough()`, `for await`, `getReader()` + `releaseLock()`, `Bun.readableStreamToText()`, `Bun.readableStreamToArrayBuffer()`, `stream.text()`, `stream.bytes()`, `stream.json()`, `stream.blob()` all end with `locked === false`. **Scope of the pumps.** `readStreamIntoSink` and `readDirectStream` have one caller, `assignToStream`, which only `JSSinkController__assignToStream` calls (Rust `JSSink::assign_to_stream`: `HTTPResponseSink` and its TLS and HTTP/3 siblings, `FetchRequestBodySink`, `NetworkSink`, `FileSink`, `RewriterPipe`). **State on main.** The lock state after the hand-off depends on how the pump ended. A clean end and a sink that closes early (client gone, aborted upload, child exited) go through `rsisFinish` and release the reader one microtask after the stream closes. A producer error goes through `rsisAbrupt`, which orphans the reader, so that stream stays locked. A direct stream is never disturbed. **Repro (the user-visible part).** ```js const stream = new ReadableStream({ start(c) { c.enqueue(new TextEncoder().encode("payload")); c.close(); } }); const responses = [new Response(stream), new Response(stream)]; await using server = Bun.serve({ port: 0, fetch: () => responses.shift(), error: e => new Response(e.code, { status: 500 }) }); for (let i = 0; i < 2; i++) { const r = await fetch(server.url); console.log(r.status, JSON.stringify(await r.text())); } // main: 200 "payload", 200 "" // branch: 200 "payload", 500 "ERR_STREAM_CANNOT_PIPE" (Stream already used, please create a new one) ``` For a direct stream `new Response(stream)` per request shows the same on main: `200 "hello"`, `200 ""`, `500`. On this branch the second `new Response(stream)` throws `Body object should not be disturbed or locked`. **Other observable changes.** `request.bodyUsed` after `fetch(request)` with an async iterable body is now `true` (was `false`, the stream was not disturbed). `Bun.readableStreamToText(stream)` on a stream a sink holds rejects with "ReadableStream has already been used" in place of "ReadableStream is locked" (same `ERR_INVALID_STATE`). **Bare streams.** `fetch(url, { body: stream })`, `Bun.write(path, stream)` and `Bun.spawn({ stdin: stream })` keep a bare stream locked too. Natively wired sources already do this for the same calls (`ReadableStream__lockNative` never unlocks, see "errors a Bun.file() stream whose file does not open" from #42477). **node v26.3.0.** `fetch(url, { method: "POST", body: jsStream, duplex: "half" })`: afterwards `locked === true` and `getReader()` throws. The same after an upload that an `AbortSignal` stopped. The other five consumers are Bun APIs with no Node counterpart. **The mark comes after the reader acquisition.** A stream that another reader already holds is not marked. `Bun.serve()`, `fetch()`, `Bun.write()` and `HTMLRewriter` reject such a stream before the pump. `Bun.spawn()` stdin checks only `is_disturbed` (`stdio.rs:388`, `:536`), so a locked stream reaches the pump there. **Out of scope (the concern not addressed).** `rsisFinally` calls `clearStreamControllerSlots` also when the pump never got the lock. `Bun.spawn({ stdin: stream })` with a stream the caller holds through `getReader()` reaches that: the caller's `reader.read()` then never settles. This is the same on main and on this branch. #41532 rejects a locked stdin stream before the pump. **Not covered here.** - A handler that drains or partly reads a stream with its own reader, releases it, and then returns it in a `Response` still gets a `200` with the empty or remaining body. #36110 covers that gate. - Handing the same Node `Readable` or generator object to a second consumer makes a new stream each time. **Changed tests.** Three tests in `bun-write.test.js` (from #42114) read the terminal state through `stream.getReader().closed`. One test in `streams.test.js` (from #33781) polled `ts.readable.locked` to wait for the sink teardown. Both observed the unlock as a means, not as the subject. They use `finished()` from `node:stream/promises` now. The `streams.test.js` test still proves the teardown ran: `desiredSize` reads `null` only after the controller slot is cleared, and reads `0` for a stream that is only closed. **Fail before.** With `src/` from main and `bun bd`: `web-stream-state.test.ts` 30 of 46 fail, `body.test.ts` 12 of 778 fail, `serve-reused-response.test.ts` 2 of 9 fail, and the three `bun-write.test.js` tests fail on the new `locked` assertion. With this branch all pass. On main only the `Bun.serve()` row of "the stdout of a running child" fails: the other four consumers wire a pipe-backed `FileReader` natively. **Built-in JS.** I searched `src/js` for code that touches a stream after a native sink took it. The only `stream.cancel()` on a web stream is `ReadableFromWeb._destroy`, which #42573 handles for the body mixin case. **Earlier reports of the same class.** #7001 (fixed in #7861) and #6860. **Suites run on the debug build.** `test/js/web/streams/{streams,streams-leak,readable-stream-blob-consumed,readable-stream-terminal-barrier-release,transform-stream-leak}`, `test/js/third_party/wpt-streams`, `test/js/web/fetch/{body,body-stream,body-stream-excess,body-clone,body-async-iterator,body-mixin-errors,blob-write,fetch,fetch.stream,fetch-backpressure,fetch-abort-stream-body,fetch-stream-cancel-leak,fetch-redirect,response}`, `test/js/web/request/request`, `test/js/bun/http/{serve,bun-server,serve-reused-response,serve-direct-readable-stream,serve-body-leak,serve-pending-promise-abort-leak,async-iterator-stream,serve-async-stream-client-abort,serve-error-handler-stream,serve-response-stream-sink-leak,serve-stream-body-error,serve-stream-reject-flush-leak}`, `test/js/bun/spawn/{spawn,spawn-stdin-readable-stream}`, `test/js/bun/io/bun-write`, `test/js/workerd/{html-rewriter,html-rewriter-leak}`, `test/js/bun/s3/{s3-stream-cancel-leak,s3-stream-error-gc,s3-upload-stream-gc,s3-write-to-file-sync-close,s3-connection-close}`, `test/js/node/stream/{node-stream,web-stream-state}`, `test/js/node/async_hooks/AsyncLocalStorage`, `test/regression/issue/07001`. The failures that remain also fail with `src/` from main in this container: IPv6, tests that need a non-root user, external hosts, and 5 s timeouts of the debug build. </details> <!-- robobun:evidence:begin --> --- **no test proof** · iteration 0 · platform-specific test(s) that do not run on this machine, deferring to CI, which covers all platforms: test/js/web/streams/streams.test.js, test/js/bun/io/bun-write.test.js <!-- robobun:evidence:end -->
…ven-sh#42609) ### Problem - `Bun.serve()` sends a second `Response` around an already sent `ReadableStream` as a `200` with an empty body. The `error` handler does not run. - A native sink consumer (`Bun.serve()`, a `fetch()` upload, `Bun.write()`, `Bun.spawn()` stdin, S3, `HTMLRewriter`) pumps a JS-backed stream in `BunStreamSource.cpp`. At the end `rsisFinally` (`:853`) releases the reader and `directStreamOnClose` (`:662`) drops the lock: `locked === false`. A `type: "direct"` stream, so every async iterable body, is not disturbed either. `RequestContext.rs:3152` checks the lock alone. - Found by a comparison with node v26.3.0, not by a user report. ### Fix - `rsisBegin` and `readDirectStream` call the new `JSReadableStream::markConsumedAsBody()` once the pump holds the stream. The stream stays locked after the pump lets go. - Behaviour change: after such a consumer took a stream, `getReader()`, `tee()`, `pipeTo()` and `cancel()` fail with a `TypeError`. Body mixin methods (oven-sh#42116) and natively wired sinks (oven-sh#42477) already do this. Neither is released. - `pipeTo()`, async iteration and `Bun.readableStreamTo*()` do not reach these pumps and still unlock. - Verified: `web-stream-state.test.ts` (30 new tests fail on main), `body.test.ts` (12), `serve-reused-response.test.ts` (2). Self-reviewed: 3 concerns, 2 addressed, 1 out of scope (Notes). ### Background - A JSSink is a native sink (`HTTPResponseSink`, `FileSink`). `JSSink::assign_to_stream` pumps a stream into it. - `readStreamIntoSink` reads through a default reader. `readDirectStream` gives the sink to the `pull()` of a direct stream. - `m_consumedAsBody` (oven-sh#42116) is a bit that `isReadableStreamLocked()` includes and nothing clears. The fetch spec never releases the reader of a body. <details><summary>Notes</summary> **The rule.** A consumer that takes a stream as a body keeps it locked: the body mixin methods (oven-sh#42116), `to_any_blob` lifts (oven-sh#42516), natively wired `ByteStream`/`FileReader` sinks (oven-sh#42477), and now the two pumps. A reader at the stream level releases its lock as the Streams spec says. Checked on this branch: `pipeTo()`, `pipeThrough()`, `for await`, `getReader()` + `releaseLock()`, `Bun.readableStreamToText()`, `Bun.readableStreamToArrayBuffer()`, `stream.text()`, `stream.bytes()`, `stream.json()`, `stream.blob()` all end with `locked === false`. **Scope of the pumps.** `readStreamIntoSink` and `readDirectStream` have one caller, `assignToStream`, which only `JSSinkController__assignToStream` calls (Rust `JSSink::assign_to_stream`: `HTTPResponseSink` and its TLS and HTTP/3 siblings, `FetchRequestBodySink`, `NetworkSink`, `FileSink`, `RewriterPipe`). **State on main.** The lock state after the hand-off depends on how the pump ended. A clean end and a sink that closes early (client gone, aborted upload, child exited) go through `rsisFinish` and release the reader one microtask after the stream closes. A producer error goes through `rsisAbrupt`, which orphans the reader, so that stream stays locked. A direct stream is never disturbed. **Repro (the user-visible part).** ```js const stream = new ReadableStream({ start(c) { c.enqueue(new TextEncoder().encode("payload")); c.close(); } }); const responses = [new Response(stream), new Response(stream)]; await using server = Bun.serve({ port: 0, fetch: () => responses.shift(), error: e => new Response(e.code, { status: 500 }) }); for (let i = 0; i < 2; i++) { const r = await fetch(server.url); console.log(r.status, JSON.stringify(await r.text())); } // main: 200 "payload", 200 "" // branch: 200 "payload", 500 "ERR_STREAM_CANNOT_PIPE" (Stream already used, please create a new one) ``` For a direct stream `new Response(stream)` per request shows the same on main: `200 "hello"`, `200 ""`, `500`. On this branch the second `new Response(stream)` throws `Body object should not be disturbed or locked`. **Other observable changes.** `request.bodyUsed` after `fetch(request)` with an async iterable body is now `true` (was `false`, the stream was not disturbed). `Bun.readableStreamToText(stream)` on a stream a sink holds rejects with "ReadableStream has already been used" in place of "ReadableStream is locked" (same `ERR_INVALID_STATE`). **Bare streams.** `fetch(url, { body: stream })`, `Bun.write(path, stream)` and `Bun.spawn({ stdin: stream })` keep a bare stream locked too. Natively wired sources already do this for the same calls (`ReadableStream__lockNative` never unlocks, see "errors a Bun.file() stream whose file does not open" from oven-sh#42477). **node v26.3.0.** `fetch(url, { method: "POST", body: jsStream, duplex: "half" })`: afterwards `locked === true` and `getReader()` throws. The same after an upload that an `AbortSignal` stopped. The other five consumers are Bun APIs with no Node counterpart. **The mark comes after the reader acquisition.** A stream that another reader already holds is not marked. `Bun.serve()`, `fetch()`, `Bun.write()` and `HTMLRewriter` reject such a stream before the pump. `Bun.spawn()` stdin checks only `is_disturbed` (`stdio.rs:388`, `:536`), so a locked stream reaches the pump there. **Out of scope (the concern not addressed).** `rsisFinally` calls `clearStreamControllerSlots` also when the pump never got the lock. `Bun.spawn({ stdin: stream })` with a stream the caller holds through `getReader()` reaches that: the caller's `reader.read()` then never settles. This is the same on main and on this branch. oven-sh#41532 rejects a locked stdin stream before the pump. **Not covered here.** - A handler that drains or partly reads a stream with its own reader, releases it, and then returns it in a `Response` still gets a `200` with the empty or remaining body. oven-sh#36110 covers that gate. - Handing the same Node `Readable` or generator object to a second consumer makes a new stream each time. **Changed tests.** Three tests in `bun-write.test.js` (from oven-sh#42114) read the terminal state through `stream.getReader().closed`. One test in `streams.test.js` (from oven-sh#33781) polled `ts.readable.locked` to wait for the sink teardown. Both observed the unlock as a means, not as the subject. They use `finished()` from `node:stream/promises` now. The `streams.test.js` test still proves the teardown ran: `desiredSize` reads `null` only after the controller slot is cleared, and reads `0` for a stream that is only closed. **Fail before.** With `src/` from main and `bun bd`: `web-stream-state.test.ts` 30 of 46 fail, `body.test.ts` 12 of 778 fail, `serve-reused-response.test.ts` 2 of 9 fail, and the three `bun-write.test.js` tests fail on the new `locked` assertion. With this branch all pass. On main only the `Bun.serve()` row of "the stdout of a running child" fails: the other four consumers wire a pipe-backed `FileReader` natively. **Built-in JS.** I searched `src/js` for code that touches a stream after a native sink took it. The only `stream.cancel()` on a web stream is `ReadableFromWeb._destroy`, which oven-sh#42573 handles for the body mixin case. **Earlier reports of the same class.** oven-sh#7001 (fixed in oven-sh#7861) and oven-sh#6860. **Suites run on the debug build.** `test/js/web/streams/{streams,streams-leak,readable-stream-blob-consumed,readable-stream-terminal-barrier-release,transform-stream-leak}`, `test/js/third_party/wpt-streams`, `test/js/web/fetch/{body,body-stream,body-stream-excess,body-clone,body-async-iterator,body-mixin-errors,blob-write,fetch,fetch.stream,fetch-backpressure,fetch-abort-stream-body,fetch-stream-cancel-leak,fetch-redirect,response}`, `test/js/web/request/request`, `test/js/bun/http/{serve,bun-server,serve-reused-response,serve-direct-readable-stream,serve-body-leak,serve-pending-promise-abort-leak,async-iterator-stream,serve-async-stream-client-abort,serve-error-handler-stream,serve-response-stream-sink-leak,serve-stream-body-error,serve-stream-reject-flush-leak}`, `test/js/bun/spawn/{spawn,spawn-stdin-readable-stream}`, `test/js/bun/io/bun-write`, `test/js/workerd/{html-rewriter,html-rewriter-leak}`, `test/js/bun/s3/{s3-stream-cancel-leak,s3-stream-error-gc,s3-upload-stream-gc,s3-write-to-file-sync-close,s3-connection-close}`, `test/js/node/stream/{node-stream,web-stream-state}`, `test/js/node/async_hooks/AsyncLocalStorage`, `test/regression/issue/07001`. The failures that remain also fail with `src/` from main in this container: IPv6, tests that need a non-root user, external hosts, and 5 s timeouts of the debug build. </details> <!-- robobun:evidence:begin --> --- **no test proof** · iteration 0 · platform-specific test(s) that do not run on this machine, deferring to CI, which covers all platforms: test/js/web/streams/streams.test.js, test/js/bun/io/bun-write.test.js <!-- robobun:evidence:end -->
…with the buffer (#38003) ### Problem - A native byte stream first read after its producer failed ends clean after 0 bytes. Sources: `fetch` body, `req.body`, `S3File.stream()`, `HTMLRewriter` output. `Bun.serve` answers an empty `200`. An S3 copy stores an empty object. - If a reader took part of the buffer, the process aborts: `panic: range end index 262144 out of range for slice of length 0` (`ByteStream::take_buffer`). - Cause: `ByteStream::append(Err)` stores the error, drops `buffer` and keeps `offset`. `on_start`, `to_any_blob` and `Bun.serve` read that as "ended, empty". ### Fix - A stored error makes `on_start` return `Start::Ready` and `to_any_blob` return `None`, so the first pull delivers it. `Bun.serve` hands it to its sink, and `end_chunk` calls `error()`. - `buffer` and `offset` become one `Buffered` value. Every method that empties the bytes resets the index. - Verified: six test files (Notes). 24 new tests fail without the change. - Self-reviewed: 2 concerns raised, 2 addressed. ### Background - `ByteStream` is the native source of a streamed body. - A pull copies into a view. Bytes that do not fit stay buffered. `offset` counts the ones already taken. - Weighed a store in `append(Err)` and a clamp in `take_buffer`. Six sites empty the buffer, and a clamp hides a wrong index. ### Downsides - A handler that returned such a body got a silent `200`. It now gets `error()` or the default 500. - One test of `pending.result` per stream start and per `Bun.serve` attach. Release `text`: +768 bytes. - Still open: `Bun.write(path, stream)` rejects after it truncated the file. A started body that did not fail loses its queued bytes (#33461, #42281). <details><summary>Notes</summary> **Origin and reach.** No user reported either symptom. A fuzzer found the panic, and a differential run against node v26.3.0 found the empty body. The panic ships since 1.4.0: #38656 gave `take_buffer` its `drain(..consumed)`. The review thread at #43410 (comment) named the stale `offset` on 09-19 and got the answer that the crash would have its own change with its own test. This is that change, inside this PR so that `ByteStream.rs` has one pull. **Three routes reach `take_buffer` with an index left behind an emptied buffer.** 1. `Bun.serve` renders a `Source::Bytes` body that a consumer started and let go (`res.body.getReader().releaseLock()`), after the producer failed. `do_render_with_body` calls `drain()`. 2. `Readable.fromWeb()` on such a stream. `on_start` took the buffer and kept the index, then `native-readable.ts` calls `drain()`. This one needs no failure. 3. The stream's own pull loop in text mode (`res.textStream()`). The adapter keeps its pull view there, so `nativeSourcePullImpl` (`BunStreamSource.cpp`) calls `handle.drain()` before each pull. A body that fails after a partial read is enough. No second consumer is involved. Sites that emptied `buffer` and kept `offset` on main: `append(Err)`, the buffer-action error arm of `on_data`, `on_cancel`, the stored-error arm of `to_buffered_value`, the `OwnedAndDone` arm of `on_start`, `to_any_blob`. `Buffered` offers `take`, `clear`, `copy_out`, `adopt` and `extend`, and nothing else writes the two fields. **Before and after** (release build of main, debug build of this branch): | script | main | this PR | | --- | --- | --- | | `HTMLRewriter` output, probed with `getReader().releaseLock()`, 400,007 bytes of output, then a handler throws, returned from `Bun.serve` | panic, exit 134 | `500`, `error()` got `boom` | | the same with exactly one pull view of output (262,144 bytes) | `200 ""` | `500`, `error()` got `boom` | | the same after only `void res.body` | `200 ""` | `500`, `error()` got `boom` | | `const body = res.body`, upstream resets, `for await (const c of body)` | ends cleanly after 0 of 100,000 bytes | rejects `ECONNRESET` (node: rejects) | | `res.textStream()` reader, one view read, upstream resets, next read | panic, exit 134 | the read rejects `ECONNRESET` | | `Readable.fromWeb()` on a probed rewriter output, clean end | panic, exit 134 | the bytes the native source still holds | | `req.body` read after the uploading client went away | `{ done: true }` | rejects `AbortError` | | `new Response(body).arrayBuffer()` / `bytes()` / `blob()` / `json()` on a body that failed while idle | resolve empty | reject `ECONNRESET` | | `new Bun.S3Client({}).file(key).stream()` (no credentials), then `for await`, a reader, `pipeTo`, `tee`, `new Response(s).bytes()` or `.blob()` | end cleanly after 0 bytes | reject `ERR_S3_MISSING_CREDENTIALS` | | `client.file(key).stream()` read after the download failed: a body cut short, a missing key (404), a denied key (403) | ends cleanly after 0 bytes | rejects `ConnectionClosed`, `NoSuchKey`, `AccessDenied` | | `client.write(key, new Response(s))` or `Bun.write(s3file, new Response(s))` with such a stream `s` | resolves 0, a `PUT` of 0 bytes replaces the object | rejects, no request goes out | | `fetch(url, { method: "POST", body: s })` | resolves 200, the server gets a complete POST with `Content-Length: 0` | rejects, the server gets no complete request | | `HTMLRewriter` output `.body` taken, a handler throws or the upstream resets, then the first read | `{ done: true }` | rejects with the handler's or the upstream's error | **Four sources, one script.** Each transfer announces 100 bytes, sends 50, and is cut by a FIN 20 ms later. The stream is taken at once and read 100 ms later: `(await fetch(url)).body`, `req.body` in `Bun.serve`, `s3.file(key).stream()`, `new HTMLRewriter().transform(await fetch(url)).body`. A release canary (367d939) and a debug build of main bd599f5 end all four cleanly with 0 bytes, 3 of 3 runs each. A debug build of this branch rejects all four, 3 of 3 runs, with the error the same read gets when it is made at once: `ECONNRESET`, `AbortError`, `S3Error ConnectionClosed`, `ECONNRESET`. **What stays open.** For a stream source, `Bun.write(path, source)` opens the destination with `O_TRUNC` before it reads the stream (`pipe_readable_stream_to_blob`, `Blob.rs:1448`). With a failed stream, or a `Response` or `Request` that wraps one, the write now rejects, and a file that held 777 bytes holds 0. Main does the same for a bare stream, and resolved 0 for the wrappers. Only `Bun.write(path, res)` with the fetch `Response` itself keeps the old bytes. An S3 destination keeps the old object, because no request goes out. This PR does not change when `Bun.write` truncates. Two behaviours change only in the state that used to abort: `on_start` and `resume` hand over the bytes no consumer took, not the whole buffer, so a prefix a reader already holds is not sent twice. The `to_write == 0` arm of `on_pull` is gone: it handled a buffer whose bytes were all taken but not dropped, which cannot exist now. The `HTMLRewriter` paces its input by the whole buffer, the taken prefix included (#38656). `Buffered::held_len()` keeps that. Counting only the untaken bytes made `html-rewriter.test.js` "a locked reader paces the input by its reads" fail, which is how the difference showed. **Self-review.** Two concerns survived: that the change may claim severity but not demand (so this body states that no user reported it and lists the routes), and that a rename in `ByteStream.rs` races the other open pulls on the file (it now ships inside this one). **Since the first revision of this PR.** Merged up to main. The `Blob.rs` hunk is gone: main rewrote `pipe_readable_stream_to_blob` and marks the pump's rejected promise handled there. `s3-write-to-file-sync-close.test.ts` now asserts the rejection instead of accepting both outcomes. `end_sink_with_pending_error` uses main's `detach_sink` plus `sink.end`. The fixture variants sit in main's `nativeBodies` table (#40596). The "failed after attach" control is dropped: main's `proxied-fetch-after-status` row covers it with #40596's contract (the connection closes without a complete message). One variant is new: the probed rewriter output from the first row above. **Tests.** `bun bd test` on Linux x64: - New or ported, failing without the change (24): `serve-stream-body-error.test.ts` (8), `body-mixin-errors.test.ts` (9), `bun-server.test.ts` (1), `fetch.stream.test.ts` (2), `s3-stream-error-gc.test.ts` (2), `html-rewriter.test.js` (2). Each was run against a debug build of main bd599f5, the first file against the release canary. - `s3-write-to-file-sync-close.test.ts` passes with and without the change: main already rejects that write, and the test now pins it. - After the rebase onto main (bd599f5): `serve-stream-body-error` 34 pass, `body-mixin-errors` 23, `s3-stream-error-gc` 4, `s3-write-to-file-sync-close` 1, `html-rewriter.test.js` 188, `bun-server` 81, `fetch.stream` 121. - This container gives the debug build about two cores. `bun-server` and `fetch.stream` were run with `--max-concurrency` lowered: with the default, their concurrent subprocess tests pass the 5 s limit, with and without the change. The "Content-Length response works (multiple parts)" rows of `fetch.stream` take 2.1 s to 4.9 s each on both builds, and in one run two of them passed the limit. - Earlier, before the rebase: `fetch-backpressure` 127, `body-stream` 9086, `body` 774, `streams` 624, `node-stream` 112. `serve.test.ts`: 325 pass. "root range port" and "/bun:info to loopback" fail on the release build of main too (the container runs as root). `bytestream-partial-read-error-fixture.ts` stages the failure with an `HTMLRewriter` whose handler throws on the second chunk, so every step runs on the JS thread in order. It waits for the handler to have run, checks that the first read is exactly one pull view, and requires the next read to reject with the handler's own error. A text-mode reader is required: a byte reader's full view retires the adapter's pending view, and the next pull then skips `drain()`. Against the released bun the fixture aborts with the panic in its header. `s3-stream-stored-error-fixture.ts` has two modes. With no credentials the download fails inside `stream()`, so every reader and sink runs against a stored failure and no timing is involved. For the failures on the wire, the stub closes the connection behind each answer. The read starts after the client has closed its end and one more request has made the round trip through the HTTP thread. The fixture writes to an absolute path: `Bun.write("out.bin", new Response(""))` with a relative path to a file that does not exist rejects with `ENOENT` from `truncate`, on main too. That is a separate defect. **Size.** `size` of the release binary: `text` 80,676,945 on main, 80,677,713 with the change. `data` and `bss` are equal. Measured before the rebase. **History.** The branch is rebased onto main at bd599f5: one commit for the change, and one for the tests of the S3 and `HTMLRewriter` sources. **Related open pulls.** #33461 and #36110 give opposite answers for a started body that did not fail (send everything, or refuse). #42281 keeps `Readable.fromWeb()` from taking a started native source. None of them touches `ByteStream.rs`. #43410 rewrites how `append(Err)` stores a JS error and needs a rebase after this. </details> <!-- robobun:evidence:begin --> --- **no test proof** · iteration 0 · platform-specific test(s) that do not run on this machine, deferring to CI, which covers all platforms: test/js/workerd/html-rewriter.test.js, test/js/web/fetch/fetch.stream.test.ts, test/js/bun/http/bun-server.test.ts <!-- robobun:evidence:end -->
|
Closing this PR because it has been inactive for more than 90 days. |
|
This PR stays closed. The stale sweep closed it, and #46528 lists it under "Skipped". I ran the cases again on The reported case is fixed
Why this diff is wrong today
What is still open
Notes: cases, results, probe diff, scriptCases. The handler makes a Response, does one action, and returns it. The body is
Builds. Local debug builds on Linux x64: GET on
GET on the #46528 branch at 3fd0627. The table is the same, with two differences. A Blob stream and a file stream answer USED after The check of this PR on
Probe on the #46528 branch at 3fd0627. 18 of 96 cases change. They are the 18 open cases, and each becomes USED. --- a/src/runtime/server/RequestContext.rs
+++ b/src/runtime/server/RequestContext.rs
@@ -3176,6 +3176,13 @@ impl<const SSL_ENABLED: bool, const DEBUG_MODE: bool, const MUX: bool>
return;
}
+ if stream.is_disturbed(global_this) {
+ this.response_body_readable_stream_ref
+ .with_mut(|s| s.deinit());
+ this.refuse_used_body();
+ return;
+ }
+
// A source that something has started feeds the stream's queue, so the stream is read like one script made.
let source = match stream.ptr {
readable_stream::Source::Bytes(_)The probe had no review. It shows which cases the rule reaches, and nothing else. Script// Matrix: a Bun.serve handler returns a Response whose body it already touched.
// Prints one line per (method, source, state). Run it with any bun binary.
import { tmpdir } from "node:os";
import { join } from "node:path";
const enc = new TextEncoder();
const upstream = Bun.serve({ port: 0, fetch: () => new Response("AAAABBBB") });
const tmpfile = join(tmpdir(), `matrix-36110-${process.pid}.txt`);
await Bun.write(tmpfile, "AAAABBBB");
const sources: Record<string, () => Promise<Response> | Response> = {
"string": () => new Response("AAAABBBB"),
"js-start": () =>
new Response(
new ReadableStream({
start(c) {
c.enqueue(enc.encode("AAAA"));
c.enqueue(enc.encode("BBBB"));
c.close();
},
}),
),
"js-pull": () => {
let i = 0;
return new Response(
new ReadableStream({
pull(c) {
if (i === 0) c.enqueue(enc.encode("AAAA"));
else if (i === 1) c.enqueue(enc.encode("BBBB"));
else c.close();
i++;
},
}),
);
},
"blob-stream": () => new Response(new Blob(["AAAA", "BBBB"]).stream()),
"file-stream": () => new Response(Bun.file(tmpfile).stream()),
"fetch": () => fetch(upstream.url),
};
const states: Record<string, (r: Response) => Promise<void>> = {
"fresh": async () => {},
"text-awaited": async r => {
await r.text();
},
"text-inflight": async r => {
r.text().catch(() => {});
},
"reader-released": async r => {
r.body!.getReader().releaseLock();
},
"read1-released": async r => {
const rd = r.body!.getReader();
await rd.read();
rd.releaseLock();
},
"drained-released": async r => {
const rd = r.body!.getReader();
while (!(await rd.read()).done);
rd.releaseLock();
},
"reader-held": async r => {
r.body!.getReader();
},
"cancelled": async r => {
await r.body!.cancel();
},
};
let lastError: any = null;
let lastBodyUsed: boolean | null = null;
const srv = Bun.serve({
port: 0,
async fetch(req) {
const u = new URL(req.url);
const r = await sources[u.searchParams.get("src")!]();
await states[u.searchParams.get("state")!](r);
lastBodyUsed = r.bodyUsed;
return r;
},
error(e: any) {
lastError = e;
return new Response("ERR:" + (e?.code ?? e?.name ?? String(e)), { status: 500 });
},
});
const only = process.argv[2] ? new Set(process.argv[2].split(",")) : null;
console.log("bun", Bun.version, Bun.revision.slice(0, 10));
for (const method of ["GET", "HEAD"]) {
for (const src of Object.keys(sources)) {
if (only && !only.has(src)) continue;
for (const state of Object.keys(states)) {
lastError = null;
lastBodyUsed = null;
let line: string;
try {
const res = await fetch(`${srv.url}?src=${src}&state=${state}`, { method, signal: AbortSignal.timeout(4000) });
const text = await res.text();
line = `${res.status} ${JSON.stringify(text)} cl=${res.headers.get("content-length")} te=${res.headers.get("transfer-encoding")}`;
} catch (e: any) {
line = `CLIENT-ERROR ${e?.code ?? e?.name}: ${String(e?.message).slice(0, 60)}`;
}
console.log(
`${method.padEnd(4)} ${src.padEnd(11)} ${state.padEnd(16)} -> ${line} | bodyUsed=${lastBodyUsed} error()=${lastError ? (lastError.code ?? lastError.name) + ": " + String(lastError.message).slice(0, 50) : "not called"}`,
);
}
}
}
srv.stop(true);
upstream.stop(true);
process.exit(0); |
Repro
String/Blob/Uint8Array/
Response.jsonbodies already 500 withERR_BODY_ALREADY_USEDhere (since #33118). A Response whose ReadableStream body the handler consumed was served as a silent empty200withContent-Length: 0, and a partially-read stream (getReader()+read()+releaseLock()) silently served only the unread remainder.Cause
After
await resp.text()on a stream body, the body staysBody::Value::Lockedwithaction = GetTextand the disturbed stream still in the Response's JS-sidestreamcache slot.do_render_with_bodyre-reads that stream viaget_body_readable_stream, but the Locked arm only checkedstream.is_locked(), notstream.is_disturbed(). A consumed stream is disturbed but not locked, so it fell through todo_render_streamand rendered as empty.Fix
Check
is_disturbed()on the response's ReadableStream indo_render_with_bodyand route to the sameERR_BODY_ALREADY_USEDerror-handler path as the existingBody::Value::Usedarm.Verification
Extended
test/js/bun/http/serve-reused-response.test.tsto cover the stream variants (start-source, pull-source, partial-read-then-releaseLock) alongside the existing string/blob cases.[review] gate passed · iteration 0 · 2 files touched
fails on main (without fix)
passes on PR (with fix)
diff hotspot
gate history · 1 passed · 0 rejected · iteration 0
evidence per changed file