Repository navigation
HTMLRewriter: stream through SinkHandle/SourceHandle; suspend async handlers instead of wait_for_promise - #36733
Conversation
|
Note Reviews pausedIt looks like this branch is under active development. To avoid overwhelming you with review comments due to an influx of new commits, CodeRabbit has automatically paused this review. You can configure this behavior by changing the Use the following commands to manage reviews:
Use the checkboxes below for quick actions:
WalkthroughChangesThe PR adds asynchronous HTMLRewriter handler suspension and resumption. It replaces buffered output with streaming HTMLRewriter suspension and streaming
Possibly related PRs
Suggested reviewers: 🚥 Pre-merge checks | ✅ 4✅ Passed checks (4 passed)
Comment |
|
Found 1 issue this PR may fix:
🤖 Generated with Claude Code |
|
This PR may be a duplicate of:
🤖 Generated with Claude Code |
|
On the duplicate-PR bot: #36697 (opened a few hours earlier, same streaming-input direction) still calls On #33261: this PR resolves work item 2 of that umbrella ( |
|
On the comment-cop flags: these are doc comments on public fields/fns and invariant notes on refcount pairing and the defer-metadata behavior, not workaround justifications. Every one of them is information the next reader would otherwise have to reconstruct (why |
There was a problem hiding this comment.
Actionable comments posted: 20
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Inline comments:
In @.github/workflows/lolhtml.yml:
- Around line 28-33: Update the test job’s concurrency group to include
github.event_name alongside github.workflow and github.ref, keeping cancellation
enabled so direct and workflow_call triggers use separate lanes. Add a
timeout-minutes value to the same test job to bound cargo test runtime.
In `@packages/bun-types/html-rewriter.d.ts`:
- Around line 185-196: Update the input-type predicate used by
HTMLRewriter.transform to recognize JSType::DataView before widening the
TypeScript declaration to Bun.BufferSource. Preserve the existing byte
extraction and handling for TypedArray and ArrayBuffer inputs so DataView
reaches the same transformation path instead of throwing “Expected Response or
Body.”
In `@patches/lolhtml/content-handler-suspension.patch`:
- Around line 3028-3032: Update the SuspendedPhase::None arm in
TransformStream::resume to return a RewritingError describing that resume was
called without an active suspension. Remove the success return and ensure the
misuse is reported in release builds, while preserving existing behavior for
genuinely suspended phases.
- Around line 2696-2705: Update the default TransformController methods
resume_suspended_token and resume_suspended_document_end to return a catchable
RewritingError instead of invoking unreachable!. Use
RewritingError::ContentHandlerError with a message identifying
TransformController and the missing suspension override, while preserving the
existing method signatures and behavior for implementations that support
suspension.
- Around line 2196-2216: Add multi-handler suspension tests alongside
end_tag_handler_suspension and document_end_suspension: register two handlers
for each path, have both suspend, then resume through the existing APIs and
assert each handler runs in order exactly once and the suspension count is two.
Cover both do_for_each_active_and_remove entry points without changing
production behavior.
- Around line 2549-2551: Strengthen the suspension assertions in both affected
tests near the output checks: replace each n > 0 assertion with the
deterministic exact suspension count, and update text_suspension_around_cdata to
assert the complete expected text-chunk sequence rather than checking only for a
chunk equal to "a".
- Around line 3007-3013: Extract the duplicated end/resume logic into helpers:
add buffered_chunk(&self) -> &[u8] for chunk selection, finish_or_park(&mut
self) -> Result<(), RewritingError> for finish dispatch and
SuspendedPhase::Finish handling, and commit_consumed(&mut self, chunk: &[u8],
consumed: usize) for flush, shift, and buffer-state bookkeeping. Replace both
call sites in end() and resume() while preserving their existing suspension and
consumption behavior.
- Around line 1023-1032: Update both feed-processing sites around the output
handler calls to store TextFeedSuspension only when the returned error is
RewritingError::Suspended; leave fatal ContentHandlerError and memory-limit
errors unsaved. Match the suspension variant, construct the suspension state and
copy rest only inside that branch, then propagate all other errors unchanged
through the existing result handling.
- Around line 2032-2046: Update the pump closure in drive to enforce a maximum
suspension/resume count, incrementing the existing suspensions counter and
panicking when the cap is exceeded before continuing the loop. Preserve the
current resume-until-Ok behavior below the cap and the existing handling for
unexpected rewriting errors.
- Around line 1278-1288: Update Doctype::into_bytes to add a debug assertion
that self.raw is not Modified before the existing original-bytes fallback,
preserving the current output behavior while surfacing invariant violations in
debug builds.
- Around line 314-331: Move the #[cold] annotation off map_suspension and apply
cold outlining only to the ActionError::Suspended handling branch inside it.
Keep map_suspension itself inlinable so the common other => other path remains
optimized on the hot path, while preserving the existing suspension bookkeeping
and returned error.
- Around line 1914-1928: Update the documentation for HtmlRewriter::end to
explicitly state that consuming the rewriter discards any pending
RewritingError::Suspended state and prevents resume from being called; direct
embedders using suspending document-end handlers to HtmlRewriter::end_mut
instead, while preserving the existing end_mut behavior.
In `@scripts/build/deps/lolhtml.ts`:
- Around line 36-43: Update fetchDep() to run git apply with LC_ALL=C and
verbose output while capturing stderr, then detect the exact skipped-patch
message for the requested patch path and fail instead of continuing. Ensure the
.ref file is only written after the patch has been successfully applied.
In `@src/runtime/api/html_rewriter.rs`:
- Around line 2279-2289: Update the error-result branch in the handler flow to
call record_handler_error with sink and result before returning
HandlerOutcome::Stop, while preserving the existing is_error/is_aggregate_error
condition and other paths unchanged.
- Around line 1008-1019: The error paths in init must release the output-body
reference before returning. In the handler-error handling around
take_handler_error and all early stream-error returns, detach the output
back-reference and call release_output_body_ref while _pipe_guard still keeps
the RewriterPipe alive; preserve the existing error propagation.
In `@src/runtime/webcore/Body.rs`:
- Around line 1081-1107: The comments at
src/runtime/webcore/Body.rs#L1081-L1107,
src/runtime/server/RequestContext.rs#L3256-L3266, and
src/runtime/server/RequestContext.rs#L3280-L3288 are overly verbose; condense
each to its single non-obvious fact. In the Body stream-feed logic, retain only
that bytes are fed to an already-created sole-consumer stream instead of closing
it empty; in the deferred metadata logic, retain only that metadata is deferred
so pre-first-byte failures reach error(); in the callback registration logic,
retain only that the registered callback owns a +1 reference released by
do_render_with_body_locked.
In `@test/js/workerd/html-rewriter-leak.test.ts`:
- Around line 262-274: Update the bounded polling closure around Bun.gc and the
final Promise.all(bodies) so it asserts that settled.length equals count after
the loop completes. Keep the terminal await only after this assertion, ensuring
an abandonment regression fails with a direct shortfall rather than hanging.
In `@test/js/workerd/html-rewriter.test.js`:
- Around line 1660-1663: Update the res.text() rejection assertion in the
transform response-body test to await the rejects matcher and require the
runtime’s exact disturbed-body error message, rather than using an unobserved
promise and bare toThrow().
- Around line 483-498: Replace the string-based schedule entries and eval usage
in the transform(string) test with scheduler function references, such as
process.nextTick and queueMicrotask. Invoke the selected function with the
promise resolver inside the async element handler while preserving the existing
test matrix and synchronous output assertion.
- Around line 728-737: Update the test’s handler completion coordination so the
`"resolves"` path signals and awaits handler completion after calling
`gate.resolve()`, rather than relying on the `/after` fetch to yield enough
event-loop turns. Keep the `/after` request assertion that the server remains
alive, and preserve the existing `handlerFinished` expectations for both settle
outcomes.
🪄 Autofix (Beta)
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: Path: .coderabbit.yaml
Review profile: ASSERTIVE
Plan: Pro
Run ID: a4e370a1-1bff-4e07-a545-2c5644e3cbc2
📒 Files selected for processing (27)
.github/workflows/lolhtml.ymldocs/runtime/html-rewriter.mdxpackages/bun-types/html-rewriter.d.tspatches/lolhtml/content-handler-suspension.patchscripts/build/deps/lolhtml.tssrc/codegen/generate-jssink.tssrc/jsc/VirtualMachine.rssrc/jsc/bindings/NativePromiseContext.hsrc/jsc/bindings/ZigGlobalObject.cppsrc/jsc/bindings/ZigGlobalObject.hsrc/jsc/bindings/headers.hsrc/jsc/bindings/webcore/streams/BunStreamSource.cppsrc/jsc/event_loop.rssrc/jsc/rare_data.rssrc/runtime/api/NativePromiseContext.rssrc/runtime/api/html_rewriter.rssrc/runtime/error.rssrc/runtime/server/RequestContext.rssrc/runtime/webcore.rssrc/runtime/webcore/Body.rssrc/runtime/webcore/Sink.rssrc/runtime/webcore/streams.rstest/integration/bun-types/fixture/index.tstest/js/workerd/html-rewriter-leak.test.tstest/js/workerd/html-rewriter.test.jstest/regression/issue/19219.test.tstest/regression/issue/21680.test.ts
💤 Files with no reviewable changes (2)
- src/runtime/error.rs
- src/runtime/webcore/Sink.rs
There was a problem hiding this comment.
Actionable comments posted: 1
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Inline comments:
In `@test/js/workerd/html-rewriter.test.js`:
- Around line 1655-1661: Update the DataView test in the transform(DataView)
case to allocate surrounding bytes before and after the HTML, then construct the
DataView using a nonzero byteOffset and exact byteLength for only the HTML
range. Keep the ArrayBuffer type assertion and verify the decoded output is
exclusively the rewritten HTML, excluding both surrounding regions.
🪄 Autofix (Beta)
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: Path: .coderabbit.yaml
Review profile: ASSERTIVE
Plan: Pro
Run ID: 647e513e-ce98-46cd-9a95-1092a5680c5c
📒 Files selected for processing (4)
.github/workflows/lolhtml.ymlsrc/runtime/api/html_rewriter.rstest/js/workerd/html-rewriter-leak.test.tstest/js/workerd/html-rewriter.test.js
There was a problem hiding this comment.
Beyond the inline findings, two adjacent memory-safety concerns were examined and ruled out this run: (1) SuspensionContext dual-ownership between the NativePromiseContext cell and the RareData cleanup hook cannot UAF — both take and abandon unregister the hook before freeing, and DeferredDerefTask::run reaches abandon only when the cell was collected unsettled; (2) the new +1 on RequestContext for on_receive_value in do_render_with_body is balanced by do_render_with_body_locked's RequestContextRef guard on every path including client abort.
Extended reasoning...
This is the third review pass; prior rounds (15:54, 16:41) surfaced five issues that were fixed in eabf7a1 and 03caa18. This run found one more refcount-balance gap (write() returning Owned after a handler synchronously cancels the output reader mid-feed()) plus an O(n) nit on remove_cleanup_hook. Two other refcount/UAF hypotheses adjacent to the reported leak were investigated and refuted — recording them so the next pass doesn't re-derive them. Not approving: this is a ~4000-line architectural rewrite of HTMLRewriter with a 3100-line lol-html fork, new refcount contracts across RewriterPipe/SuspensionContext/SinkHandle, and a user-visible behavior change to Bun.serve's Source::Bytes metadata timing that affects all proxied-fetch/S3 responses. The remaining inline finding is in REVIEW.md's most-blocked category and should be fixed before merge; the scope alone warrants a maintainer sign-off.
…ailure (#37915) ### Problem - Build lanes die in a dep fetch: `error: Failed to download after 5 attempts: https://github.com/oven-sh/lol-html/archive/725ce499....tar.gz`, `cause: fetch failed`. Build 93382 lost linux x64-asan on lolhtml. Build 93392 (a one-file PR) lost four lanes: x64-musl and x64-android on cares, mimalloc and the WebKit tarball, freebsd aarch64 on lolhtml, windows aarch64 on cares, libuv, mimalloc and WebKit. - Those are the downloads that miss the image's prefetch cache (everything else in the same logs says `using prefetch cache`): the deps whose pins moved after the images were baked in #34782 (lolhtml #36733, mimalloc #36431, c-ares #34007, libuv #36839, WebKit several times a week), plus WebKit on every lane other than linux arm64, because `prefetch-deps.ts` only enumerates the bake host's own target (handed off separately). Each build makes on the order of a hundred live github.com downloads. - `downloadWithRetry` (`scripts/build/download.ts:156`) made 5 attempts with 2+4+8+16s of backoff, about 30s in total. The logs show agent-wide outages longer than that: on the x64-musl lane three downloads that started together failed on all five attempts over roughly two minutes, after lolhtml had succeeded on the same agent seconds earlier; on the freebsd lane lolhtml failed five times in a row while cares and mimalloc went through and WebKit succeeded on its second try. - `BuildError.format()` (`scripts/build/error.ts:33`) prints one level of cause. node's fetch throws `TypeError: fetch failed` and keeps the real error (DNS, connect timeout, reset) in `.cause`, so every one of these logs says only `cause: fetch failed`, and the retry lines say nothing about what failed. ### Fix - `downloadRetry`: 10 attempts, backoff doubling from 2s and capped at 30s, 180s of backoff in total instead of 30s. Exported as a `RetryPolicy` (optional last parameter of `downloadWithRetry`) so the test can run the production attempt count with the backoff zeroed. - 408 and 429 are retried along with 5xx and network errors. Other 4xx still fail on the first attempt and are still thrown unwrapped, which `prefetch-deps.ts` relies on to tell a 404 (variant not published) from a transient failure. - Each retry line names the failure it is retrying, e.g. `retry 2/10 in 2000ms (fetch failed: other side closed)`, and `format()` prints the whole cause chain through the new `describeError()`, so the next one of these says what the network did. - Why here: the prefetch cache goes stale by design as soon as a pin moves, and WebKit moves faster than images get rebaked (the images were rebaked on July 21 and this came back within two weeks; #30095 was an earlier rebake for the same symptom), so the live path is permanently on every build's critical path and has to outlast the outages CI actually sees. The wider window only costs time while github.com is actually down, when the lane would otherwise have failed; build-bun's step timeout is 60 minutes. - Verified with `test/internal/build-download-retry.test.ts` (`bun bd test`, 7 pass). It drives the real `downloadWithRetry` against a local server that drops connections or returns scripted statuses, checks the retry lines and `format()` output, and pins the schedule's total backoff at two minutes or more. Against the previous `download.ts`/`error.ts` the file fails at import (`downloadRetry` and `describeError` did not exist); each behavioral case is something the old loop did not do (10 attempts, 429 retried, reason on the retry line, cause chain in `format()`). - Also ran the loop under node, the runtime CI builds with, against a dropping server: retry lines read `(fetch failed: other side closed)` and the final report prints `cause: fetch failed: other side closed`. `bunx tsc -p scripts/build/tsconfig.json` reports nothing for these files. ### Background - Dep fetching: configure emits one ninja `dep_fetch` edge per vendored dep, which runs `scripts/build/fetch-cli.ts`; that calls `downloadWithRetry` on `https://github.com/<repo>/archive/<commit>.tar.gz`, and `fetchPrebuilt` uses the same function for release tarballs such as WebKit. Both URL kinds start with a 302 from github.com (to codeload.github.com and objects.githubusercontent.com respectively), which is why one github.com problem takes out both kinds at once. - Prefetch cache: CI images run `scripts/prefetch-deps.ts` at bake time, storing each tarball under `/opt/bun-prefetch/by-url/<sha256(url)>`. `downloadWithRetry` looks there before touching the network, so a dep is served from the image only while its pinned URL is the one that was current at bake time; anything bumped later downloads live until the next `[publish images]` rebake. - `BuildError` is the build system's error type; `fetch-cli.ts` and `build.ts` print failures through its `format()`, which produces the `error:` / `hint:` / `cause:` lines seen in the build log.
### What does this PR do? `new HTMLRewriter().transform(new Response(Bun.file(f)))` no longer reads the whole file inside `transform()`. A streamed input (file, `fetch()` body, `ReadableStream`) is now paced by whoever reads the output, the way `fetch()` paces its own body stream: - `transform()` feeds at most one upstream chunk before returning (a document that arrives in one chunk still finishes inside the call, so small pages keep `Content-Length` and work as static routes). - While something is positioned to read the output — a native sink (`Bun.serve`, a second rewriter, spawn stdin), a JS reader, or a `.text()`/`.arrayBuffer()`/`Bun.write` consumer — the input is held once that reader is 16 KiB behind and resumed by its drain signal, so a slow reader never accumulates the document. - While nothing is, the rewrite still runs every handler to the end (from the event loop, one chunk per turn) and buffers the output until read, as it always has — an unread transform is a side effect callers rely on, and a `fetch()` input must not sit on a paused connection forever. Two things sized memory to the input before. lol_html's `preallocated_parsing_buffer_size` was set to the body length; that buffer only holds the unparsed tail of one write and grows on demand, so this reserved the whole body up front, ate most of the `u32::MAX` memory budget, and for bodies ≥ 4 GiB tripped lol_html's `debug_assert!` (which is why only debug/ASAN builds panicked and release canaries "worked"). It now uses lol_html's default. And since the streaming rewrite (#36733) nothing paced the input before the output stream existed, while regular-file reads are synchronous on POSIX — so `transform()` read the entire file and buffered its rewritten output before returning (~6.5 GB RSS for a 4.2 GB file). Fixes the new steady state depends on: output is handed to the stream once per lol_html call rather than per fragment; `finish()` passes the response headers to `Value::resolve` (`.blob().type`); terminal paths drop the input's GC edges after their end handlers/body resolution rather than before, and pipe entry points hold a ref across their work; `FileReader` pins its parent for the duration of a read (buffered-reader ref hooks) and pauses the reader on backpressure; the per-loop read scratch buffer is only lent to the outermost read loop (a nested file read started from a handler no longer corrupts the outer document); `ByteStream::drain`/`take_buffer` honour `offset` (`textStream()` re-delivered a consumed prefix); `Bun.write` installs its consumer before signalling `on_start_buffering`. ### How did you verify your code works? `test/js/workerd/html-rewriter.test.js` gains a "streamed input pacing" block: one consumer table (15 consumers, byte-exact) over a held file input, held `fetch()` input, an idle locked reader, unread transforms over file/fetch inputs running to completion in a child that must exit, single-chunk `Content-Length`, blob type, `textStream()`, a handler consuming another held transform, and sparse 256 MiB / 5 GiB files whose `transform()` must not move RSS. Seven of those fail with `USE_SYSTEM_BUN=1` (the read-ahead ones, plus the `.body`-touched-but-unread hang and JS-reader stall that already existed). Ran that file (also under `BUN_JSC_collectContinuously`), html-rewriter-leak/end-error/doctype, bun-write, body-stream, fetch.stream, stream-fast-path, streams, shell pipe tests on a debug+ASAN build; `cargo check` for `x86_64-pc-windows-msvc`. --------- Co-authored-by: autofix-ci[bot] <114827586+autofix-ci[bot]@users.noreply.github.com>
…tions (#42464) ### Problem - `CallSite.getFunction()` (the `Error.prepareStackTrace` API) returned the frame's callee unchecked. JSC has callees user code could never have called: a host function, a builtin, an async or generator body function, or no function at all (wasm and program frames). - Five shapes crash 1.4.2, canary and main. After one `await`, calling what `getFunction()` returns gives `panic(main thread): Segmentation fault at address 0x32`. A wasm frame faults at `0x5` on the read. - Cause: `CallSite::finishCreation` (`src/jsc/bindings/CallSite.cpp:34`) only checks the strictness of the frame's code block. `JSCStackFrame` leaves that null for host and hidden builtin frames. ### Fix - `getFunction()` reports `undefined` unless the callee is a function user code could have called. A user function gets itself, including the CommonJS wrapper. - Caller frames keep their function: hiding a callee does not set `Flags::IsStrict`. `isToplevel()` reads a new flag computed from the real callee, so its answers do not change. - Deleted: the function-inspecting block in `callSiteProtoFuncIsToplevel`, and its `JSBoundFunction.h` include. - Verified: four new blocks in `test/js/node/v8/capture-stack-trace.test.js`, all failing on 1.4.2 and with `src/` reverted. 627 pass across the stack-trace suites. ### Background - JSC compiles an async function or a generator into a wrapper (what the user named) and a body that takes five internal arguments. The synchronous prefix runs in the body too, so the wrapper is never a frame's callee. - `createCallSitesFromFrames` (`FormatStackTraceForJS.cpp:844`) copies `isStrict()` down the frame list. JSC shows host frames V8 omits (`Reflect.construct` under a transpiled `super()`), so a cascade from a hidden callee would blank callers node reports. - JSC frames carry no receiver, so `isToplevel()` judges by the callee. <details><summary>Notes</summary> #### The five crashes A fuzz ledger reported the async body and three native promise reactions. A review of the first draft of this patch found the wasm frame. Each crashes release 1.4.2 (exit 139), and each exits cleanly with this change. | shape | 1.4.2 | with this change | | --- | --- | --- | | async function body after `await` | SEGV `0x32` | `undefined` | | generator body after `next()` | SEGV `0x32` | `undefined` | | wasm frame, on read alone | SEGV `0x5` | `undefined` | | HTMLRewriter suspension reaction | SEGV `0x12` | `undefined` | | `Bun.serve` `error()` reaction, and the `Bun.plugin` onLoad reaction | SEGV `0x12` / `0x1A` | `undefined` | The first fixture, as CommonJS so that the top frame is sloppy: ```js async function f() { await 1; Error.prepareStackTrace = (e, sites) => sites; return new Error().stack[0].getFunction(); } f().then(g => { console.log(g === f, g.length); g(); }); ``` 1.4.2 prints `false 5` and then segfaults. `0x32` is an encoded `undefined` (`0xa`) plus `0x28`, the offset of a generator internal field. Under ASAN the frames are `llint_op_put_internal_field <- llint_op_call_ignore_result <- llint_call_javascript <- JSC::MicrotaskCall::tryCallWithArguments`, so the body function wrote a generator field into a plain `undefined`. The wasm crash has three more entry shapes. Each is the same null callee, each segfaults 1.4.2 at `0x5`, and each exits cleanly with this change, also under ASAN: the wasm frame as the first call site of a trap (`WebAssembly.RuntimeError`, which is what an error reporter walks), a wasm stack overflow whose `RangeError` carries only wasm frames, and a wasm frame that JSPI resumed. The test covers the import-callback shape. The reaction crashes all reach the same place. A handler from `Zig::GlobalObject::thenable()` runs user JS synchronously: a later HTMLRewriter element handler, `Bun.serve`'s `error()`, or a getter on a plugin's onLoad result. That user JS keeps the nameless native function `getFunction()` reported, and calls it. The handler then reads its trailing argument as a native context. 12 of the 54 `PromiseFunctions` handlers decode a raw pointer from a JS number, so the class is worse than a bad cast. The host-function arm closes all of them, because `getFunction()` is the only way user code could name one. #### How old each route is The unchecked hand-out itself is old. Release 1.3.14 has the same logic in `CallSite.cpp`, `thenable()` is already `ImplementationVisibility::Public` there, and `Bun__NativePromiseContext__take` already uses `uncheckedDowncast` (#28613). The fuzz ledger reports that the wasm shape crashes 1.3.14 too. The HTMLRewriter route is new in 1.4.0, and the ledger measured it: 1.3.14 is clean, 1.4.0 to 1.4.2 crash at `0x12`. The cause is #36733, not an engine upgrade. Before it, the rewriter handled an async handler with `vm.wait_for_promise()`, which spins the event loop in place, so no native reaction existed. #36733 suspends the rewrite and resumes it through `Bun__HTMLRewriter__onHandlerResolve`, which put a reaction frame on the stack for the first time. The `Bun.serve` `error()` route and the `Bun.plugin` onLoad route use machinery that predates 1.3.14 (the onLoad reaction is from 2022), so they are probably older than that window. I read that from history and did not run it: I have no 1.3.14 binary here. #### What node returns Measured on node v26.3.0 with the same fixtures. | frame | node | 1.4.2 | with this change | | --- | --- | --- | --- | | sloppy user function | itself | itself | itself | | CommonJS module wrapper | itself | itself | itself | | `Array.prototype.map` | undefined | the `map` function | undefined | | program frame under `eval` | undefined | a non-function `JSCallee` object | undefined | | async or generator body | the wrapper | the body (length 5) | undefined | | wasm frame | a `Number` | crash | undefined | | any caller below the above | see 2. below | itself | itself | Two divergences remain, both deliberate: 1. Node returns the wrapper for an async or generator frame. JSC keeps no back pointer from a body function to its wrapper, so the wrapper cannot be recovered. `getFunctionName()` still reports the user's name. JSC applies the same policy in `callerGetter` (`FunctionPrototype.cpp`). 2. Node blanks the caller frames below a builtin frame, because V8 compiles its builtins strict. Bun does not, and this change does not start. I measured that trigger: it moves a 22-shape host and builtin probe from 4/22 to 22/22 against node. It also blanks callers that node reports wherever JSC shows a host frame V8 hides, and `Reflect.construct` under Babel's `_callSuper` is one of those. It is a separable hunk. Say so if you want it. #### isToplevel() `m_function` has two readers, `getFunction()` and `isToplevel()`. The first push of this PR narrowed what `m_function` holds and did not account for the second reader. A review comment about dead branches in `callSiteProtoFuncIsToplevel` made me measure it: on a 16-row probe, `isToplevel()` flipped from `false` to `true` on 5 rows, the `Array.prototype.map` frame and every async or generator frame. `isToplevel()` now reads `Flags::IsSloppyFunctionCall`, which `finishCreation` computes from the real callee: the frame is sloppy and its callee is a `JSFunction` that is not a host function. All 16 rows match 1.4.2 again, and four new `...IsToplevel=false` test rows pin it. Those rows fail on the first push of this PR. The replaced block had a `JSBoundFunction` arm, a host-function arm, a `jsExecutable()` arm and an `InternalFunction` arm. With `m_function` narrowed, only the `jsExecutable()` arm could still run. The `JSBoundFunction` arm never could: a bound-function thunk is `ImplementationVisibility::Private` and never gets a `CallSite`, which I checked on both the `prepareStackTrace` and the `captureStackTrace` path. #### Dropped from the first draft The first draft cascaded from every hidden callee and carried three hardening hunks. A review plus my own measurements cut all four: - The cascade blanked `getFunction()` for a plain sloppy caller that both node and 1.4.2 return. - `ImplementationVisibility::Private` on the `thenable()` handlers hides a useless `at unknown` frame. With `BUN_JSC_showPrivateScriptsInStackTraces=0` it also flips `typeof err.stack` from `"string"` to `"undefined"` for three `Bun.plugin` onLoad validation errors, whose only frame was the reaction. It needs the empty-frame-vector case to produce a header-only stack first, the way `FormatStackTraceForJS.cpp:234` already does for the `captureStackTrace` path. Worth doing, but not here. - A `dynamicDowncast` in `Bun__NativePromiseContext__take`. #40203 is open and already touches that function. - Two `dynamicDowncast` guards in the module loader onLoad reactions. With this change nothing can reach them, and an unreachable, untested guard is not worth the lines. #### Suites run, on the debug ASAN build `capture-stack-trace.test.js` 50/50. The stack-trace set is 627 pass, 0 fail: `test/js/node/v8`, `vm.test.ts`, `vm-sourceUrl.test.ts`, `util.test.js`, `test/js/bun/sourcemap`, `reportError.test.ts`, `console-log.test.ts`, `23022-stack-trace-iterator`, `circular-error-stack`, `circular-error-stack-edge-cases`, `fix-bindings-stack-trace`, `prepare-stack-trace-crash`. `stack.test.ts` and `013880.test.ts` pass on their own. `node:test` `test-error-prepare-stack-trace.js`, `test-util-getcallsites-preparestacktrace.js` and `test-shadow-realm-prepare-stack-trace.js` exit 0. Earlier drafts, whose reach was wider, also cleared the html-rewriter (207), plugin and resolve (506), streams (549), serve (295) and `node:http` (372) suites. Two failures in that area are not from this change, and each now has its own session. `test/regression/issue/013880.test.ts` leaks a global `Error.prepareStackTrace` into the shared test process and breaks 21 tests in `capture-stack-trace.test.js`. It reproduces on release 1.4.2: that file alone passes, and the two files together fail 21. `test/js/node/vm/script-leak.test.ts` sets an `isASAN` RSS threshold but no `isASAN` timeout, so it needs 25s against a 5s default on any `bun bd` build. #### Flake check The four new blocks passed 5 runs in a row, and 3 more with 24 CPU burners running, to stand in for a slow ASAN lane. Each reaction fixture needs a genuinely pending promise to make the native reaction exist. An already-resolved promise and a `queueMicrotask`-resolved promise both fail to install the suspension, which I checked, so the fixtures use `setTimeout(resolve, 1)` inside the fixture process. The test itself only awaits the process exit. Each reaction fixture also asserts `reactionFrameSeen=true`, so `leaked=undefined` cannot pass because the frame stopped appearing. </details> <!-- robobun:evidence:begin --> --- **[human-review]** gate passed · iteration 1 · 4 files touched <details><summary>fails on main (without fix)</summary> ```console ASAN without fix: 4 FAILED $ BUN_DEBUG_QUIET_LOGS=1 bun scripts/build.ts --profile=debug --quiet test "--reporter=junit" "--reporter-outfile=/tmp/pr_gate.xml" "test/js/node/v8/capture-stack-trace.test.js" bun test v1.4.3 (6a92015) test/js/node/v8/capture-stack-trace.test.js: (pass) Regular .stack [24.55ms] (pass) throw inside Error.prepareStackTrace doesnt crash [40.55ms] (pass) capture stack trace [12.79ms] (pass) capture stack trace with message [14.33ms] (pass) capture stack trace with constructor [11.08ms] (pass) capture stack trace limit [43.64ms] (pass) prepare stack trace [22.91ms] (pass) capture stack trace second argument [35.13ms] (pass) capture stack trace edge cases [23.74ms] (pass) Error.captureStackTrace installs .stack as non-enumerable [39.79ms] (pass) prepare stack trace call sites [24.06ms] (pass) sanity check [26.96ms] (pass) CallFrame isEval works as expected [16.18ms] (pass) CallFrame isTopLevel returns false for Function constructor [17.42ms] (pass) CallFrame.p.getThisgetFunction: strict/sloppy mode interaction [17.16ms] (pass) CallFrame.p.isConstructor [6.29ms] (pass) CallFrame.p.isNative [7.07ms] 726 | stderr: "pipe", 727 | }); 728 | 729 | const [stdout, st ... (truncated) release without fix: all passed bun test v1.4.3-canary.1 (8a7f62b) test/js/node/v8/capture-stack-trace.test.js: (pass) Regular .stack [0.14ms] (pass) throw inside Error.prepareStackTrace doesnt crash [0.09ms] (pass) capture stack trace [0.09ms] (pass) capture stack trace with message [0.06ms] (pass) capture stack trace with constructor [0.06ms] (pass) capture stack trace limit [0.19ms] (pass) prepare stack trace [0.11ms] (pass) capture stack trace second argument [0.17ms] (pass) capture stack trace edge cases [0.10ms] (pass) Error.captureStackTrace installs .stack as non-enumerable [0.22ms] (pass) prepare stack trace call sites [0.14ms] (pass) sanity check [0.11ms] (pass) CallFrame isEval works as expected [0.09ms] (pass) CallFrame isTopLevel returns false for Function constructor [0.09ms] (pass) CallFrame.p.getThisgetFunction: strict/sloppy mode interaction [0.10ms] (pass) CallFrame.p.isConstructor [0.03ms] (pass) CallFrame.p.isNative [0.03ms] (pass) CallFrame.p.getFunction does not expose the module loader reaction [9.47ms] (pass) CallFrame.p.getFunction does not expose the HTMLRewriter reaction [11.12ms] (pass) CallFrame.p.getFunction does not expose the serve reject reaction [11.29ms] (pass ... (truncated) ``` </details> <details><summary>passes on PR (with fix)</summary> ```console ASAN with fix: all passed $ BUN_DEBUG_QUIET_LOGS=1 bun scripts/build.ts --profile=debug --quiet test "--reporter=junit" "--reporter-outfile=/tmp/pr_gate.xml" "test/js/node/v8/capture-stack-trace.test.js" bun test v1.4.3 (6a92015) test/js/node/v8/capture-stack-trace.test.js: (pass) Regular .stack [22.42ms] (pass) throw inside Error.prepareStackTrace doesnt crash [33.22ms] (pass) capture stack trace [9.60ms] (pass) capture stack trace with message [10.80ms] (pass) capture stack trace with constructor [8.87ms] (pass) capture stack trace limit [32.05ms] (pass) prepare stack trace [17.69ms] (pass) capture stack trace second argument [24.93ms] (pass) capture stack trace edge cases [18.24ms] (pass) Error.captureStackTrace installs .stack as non-enumerable [30.12ms] (pass) prepare stack trace call sites [19.79ms] (pass) sanity check [21.41ms] (pass) CallFrame isEval works as expected [11.96ms] (pass) CallFrame isTopLevel returns false for Function constructor [12.53ms] (pass) CallFrame.p.getThisgetFunction: strict/sloppy mode interaction [13.89ms] (pass) CallFrame.p.isConstructor [5.20ms] (pass) CallFrame.p.isNative [5.26ms] (pass) CallFrame.p.getFunction does not expose the serve reject reaction ... (truncated) release with fix: all passed $ bun scripts/build.ts --profile=release [configured] bun-profile → bun (stripped) in 710ms (unchanged) ninja: Entering directory `/workspace/bun/build/release' [1/9] cxx obj/src/jsc/bindings/ZigGlobalObject.cpp.o [2/9] cxx obj/unified/UnifiedSource-src_jsc_bindings-1.cpp.o [3/9] cxx obj/unified/UnifiedSource-src_jsc_bindings-0.cpp.o [4/9] gen cpp.rs (cppbind) [4/9] cargo bun_runtime → libbun_runtime.a ^[[1m^[[92m Compiling^[[0m bun_core v0.0.0 (/workspace/bun/src/bun_core) ^[[1m^[[92m Compiling^[[0m bun_errno v0.0.0 (/workspace/bun/src/errno) ^[[1m^[[92m Compiling^[[0m bun_ptr v0.0.0 (/workspace/bun/src/ptr) ^[[1m^[[92m Compiling^[[0m bun_boringssl_sys v0.0.0 (/workspace/bun/src/boringssl_sys) ^[[1m^[[92m Compiling^[[0m bun_safety v0.0.0 (/workspace/bun/src/safety) ^[[1m^[[92m Compiling^[[0m bun_base64 v0.0.0 (/workspace/bun/src/base64) ^[[1m^[[92m Compiling^[[0m bun_cares_sys v0.0.0 (/workspace/bun/src/cares_sys) ^[[1m^[[92m Compiling^[[0m bun_zlib_sys v0.0.0 (/workspace/bun/src/zlib_sys) ^[[1m^[[92m Compiling^[[0m bun_zstd v0.0.0 (/workspace/bun/src/zstd) ^[[1m^[[92m Compiling^[[0m bun_picohttp v0.0.0 (/workspace/bun/src/picohttp) ^[[1m^[[92m Compiling^[[0m bun_brotli v ... (truncated) ``` </details> <details><summary>diff hotspot</summary> ``` src/jsc/bindings/CallSite.cpp | 31 +++- src/jsc/bindings/CallSite.h | 2 + src/jsc/bindings/CallSitePrototype.cpp | 21 +-- test/js/node/v8/capture-stack-trace.test.js | 264 ++++++++++++++++++++++++++++ 4 files changed, 297 insertions(+), 21 deletions(-) ``` </details> **gate history** · 3 passed · 0 rejected · iteration 1 <details><summary>evidence per changed file</summary> ``` file reads edits tests src/jsc/bindings/CallSite.cpp 5 11 34 src/jsc/bindings/CallSite.h 2 3 33 src/jsc/bindings/CallSitePrototype.cpp 2 2 33 test/js/node/v8/capture-stack-trace.test.js 6 10 33 ``` </details> **root cause** · written by the author bot `CallSite` treated a frame as non-strict whenever it lacked a CodeBlock with a strict executable, so native frames, builtin JS frames, and the internal async/generator body functions that JSC runs after the first suspension all had their callee stored and handed back through `getFunction()`; calling one of those internal callees with caller-chosen arguments let plain JavaScript write generator internal fields into arbitrary objects, which segfaulted the interpreter. The fix narrows the callee exposure in one place: the function is only stored when it is a `JSFunction` whose `FunctionExecuta… <!-- robobun:evidence:end -->
…tions (oven-sh#42464) ### Problem - `CallSite.getFunction()` (the `Error.prepareStackTrace` API) returned the frame's callee unchecked. JSC has callees user code could never have called: a host function, a builtin, an async or generator body function, or no function at all (wasm and program frames). - Five shapes crash 1.4.2, canary and main. After one `await`, calling what `getFunction()` returns gives `panic(main thread): Segmentation fault at address 0x32`. A wasm frame faults at `0x5` on the read. - Cause: `CallSite::finishCreation` (`src/jsc/bindings/CallSite.cpp:34`) only checks the strictness of the frame's code block. `JSCStackFrame` leaves that null for host and hidden builtin frames. ### Fix - `getFunction()` reports `undefined` unless the callee is a function user code could have called. A user function gets itself, including the CommonJS wrapper. - Caller frames keep their function: hiding a callee does not set `Flags::IsStrict`. `isToplevel()` reads a new flag computed from the real callee, so its answers do not change. - Deleted: the function-inspecting block in `callSiteProtoFuncIsToplevel`, and its `JSBoundFunction.h` include. - Verified: four new blocks in `test/js/node/v8/capture-stack-trace.test.js`, all failing on 1.4.2 and with `src/` reverted. 627 pass across the stack-trace suites. ### Background - JSC compiles an async function or a generator into a wrapper (what the user named) and a body that takes five internal arguments. The synchronous prefix runs in the body too, so the wrapper is never a frame's callee. - `createCallSitesFromFrames` (`FormatStackTraceForJS.cpp:844`) copies `isStrict()` down the frame list. JSC shows host frames V8 omits (`Reflect.construct` under a transpiled `super()`), so a cascade from a hidden callee would blank callers node reports. - JSC frames carry no receiver, so `isToplevel()` judges by the callee. <details><summary>Notes</summary> #### The five crashes A fuzz ledger reported the async body and three native promise reactions. A review of the first draft of this patch found the wasm frame. Each crashes release 1.4.2 (exit 139), and each exits cleanly with this change. | shape | 1.4.2 | with this change | | --- | --- | --- | | async function body after `await` | SEGV `0x32` | `undefined` | | generator body after `next()` | SEGV `0x32` | `undefined` | | wasm frame, on read alone | SEGV `0x5` | `undefined` | | HTMLRewriter suspension reaction | SEGV `0x12` | `undefined` | | `Bun.serve` `error()` reaction, and the `Bun.plugin` onLoad reaction | SEGV `0x12` / `0x1A` | `undefined` | The first fixture, as CommonJS so that the top frame is sloppy: ```js async function f() { await 1; Error.prepareStackTrace = (e, sites) => sites; return new Error().stack[0].getFunction(); } f().then(g => { console.log(g === f, g.length); g(); }); ``` 1.4.2 prints `false 5` and then segfaults. `0x32` is an encoded `undefined` (`0xa`) plus `0x28`, the offset of a generator internal field. Under ASAN the frames are `llint_op_put_internal_field <- llint_op_call_ignore_result <- llint_call_javascript <- JSC::MicrotaskCall::tryCallWithArguments`, so the body function wrote a generator field into a plain `undefined`. The wasm crash has three more entry shapes. Each is the same null callee, each segfaults 1.4.2 at `0x5`, and each exits cleanly with this change, also under ASAN: the wasm frame as the first call site of a trap (`WebAssembly.RuntimeError`, which is what an error reporter walks), a wasm stack overflow whose `RangeError` carries only wasm frames, and a wasm frame that JSPI resumed. The test covers the import-callback shape. The reaction crashes all reach the same place. A handler from `Zig::GlobalObject::thenable()` runs user JS synchronously: a later HTMLRewriter element handler, `Bun.serve`'s `error()`, or a getter on a plugin's onLoad result. That user JS keeps the nameless native function `getFunction()` reported, and calls it. The handler then reads its trailing argument as a native context. 12 of the 54 `PromiseFunctions` handlers decode a raw pointer from a JS number, so the class is worse than a bad cast. The host-function arm closes all of them, because `getFunction()` is the only way user code could name one. #### How old each route is The unchecked hand-out itself is old. Release 1.3.14 has the same logic in `CallSite.cpp`, `thenable()` is already `ImplementationVisibility::Public` there, and `Bun__NativePromiseContext__take` already uses `uncheckedDowncast` (oven-sh#28613). The fuzz ledger reports that the wasm shape crashes 1.3.14 too. The HTMLRewriter route is new in 1.4.0, and the ledger measured it: 1.3.14 is clean, 1.4.0 to 1.4.2 crash at `0x12`. The cause is oven-sh#36733, not an engine upgrade. Before it, the rewriter handled an async handler with `vm.wait_for_promise()`, which spins the event loop in place, so no native reaction existed. oven-sh#36733 suspends the rewrite and resumes it through `Bun__HTMLRewriter__onHandlerResolve`, which put a reaction frame on the stack for the first time. The `Bun.serve` `error()` route and the `Bun.plugin` onLoad route use machinery that predates 1.3.14 (the onLoad reaction is from 2022), so they are probably older than that window. I read that from history and did not run it: I have no 1.3.14 binary here. #### What node returns Measured on node v26.3.0 with the same fixtures. | frame | node | 1.4.2 | with this change | | --- | --- | --- | --- | | sloppy user function | itself | itself | itself | | CommonJS module wrapper | itself | itself | itself | | `Array.prototype.map` | undefined | the `map` function | undefined | | program frame under `eval` | undefined | a non-function `JSCallee` object | undefined | | async or generator body | the wrapper | the body (length 5) | undefined | | wasm frame | a `Number` | crash | undefined | | any caller below the above | see 2. below | itself | itself | Two divergences remain, both deliberate: 1. Node returns the wrapper for an async or generator frame. JSC keeps no back pointer from a body function to its wrapper, so the wrapper cannot be recovered. `getFunctionName()` still reports the user's name. JSC applies the same policy in `callerGetter` (`FunctionPrototype.cpp`). 2. Node blanks the caller frames below a builtin frame, because V8 compiles its builtins strict. Bun does not, and this change does not start. I measured that trigger: it moves a 22-shape host and builtin probe from 4/22 to 22/22 against node. It also blanks callers that node reports wherever JSC shows a host frame V8 hides, and `Reflect.construct` under Babel's `_callSuper` is one of those. It is a separable hunk. Say so if you want it. #### isToplevel() `m_function` has two readers, `getFunction()` and `isToplevel()`. The first push of this PR narrowed what `m_function` holds and did not account for the second reader. A review comment about dead branches in `callSiteProtoFuncIsToplevel` made me measure it: on a 16-row probe, `isToplevel()` flipped from `false` to `true` on 5 rows, the `Array.prototype.map` frame and every async or generator frame. `isToplevel()` now reads `Flags::IsSloppyFunctionCall`, which `finishCreation` computes from the real callee: the frame is sloppy and its callee is a `JSFunction` that is not a host function. All 16 rows match 1.4.2 again, and four new `...IsToplevel=false` test rows pin it. Those rows fail on the first push of this PR. The replaced block had a `JSBoundFunction` arm, a host-function arm, a `jsExecutable()` arm and an `InternalFunction` arm. With `m_function` narrowed, only the `jsExecutable()` arm could still run. The `JSBoundFunction` arm never could: a bound-function thunk is `ImplementationVisibility::Private` and never gets a `CallSite`, which I checked on both the `prepareStackTrace` and the `captureStackTrace` path. #### Dropped from the first draft The first draft cascaded from every hidden callee and carried three hardening hunks. A review plus my own measurements cut all four: - The cascade blanked `getFunction()` for a plain sloppy caller that both node and 1.4.2 return. - `ImplementationVisibility::Private` on the `thenable()` handlers hides a useless `at unknown` frame. With `BUN_JSC_showPrivateScriptsInStackTraces=0` it also flips `typeof err.stack` from `"string"` to `"undefined"` for three `Bun.plugin` onLoad validation errors, whose only frame was the reaction. It needs the empty-frame-vector case to produce a header-only stack first, the way `FormatStackTraceForJS.cpp:234` already does for the `captureStackTrace` path. Worth doing, but not here. - A `dynamicDowncast` in `Bun__NativePromiseContext__take`. oven-sh#40203 is open and already touches that function. - Two `dynamicDowncast` guards in the module loader onLoad reactions. With this change nothing can reach them, and an unreachable, untested guard is not worth the lines. #### Suites run, on the debug ASAN build `capture-stack-trace.test.js` 50/50. The stack-trace set is 627 pass, 0 fail: `test/js/node/v8`, `vm.test.ts`, `vm-sourceUrl.test.ts`, `util.test.js`, `test/js/bun/sourcemap`, `reportError.test.ts`, `console-log.test.ts`, `23022-stack-trace-iterator`, `circular-error-stack`, `circular-error-stack-edge-cases`, `fix-bindings-stack-trace`, `prepare-stack-trace-crash`. `stack.test.ts` and `013880.test.ts` pass on their own. `node:test` `test-error-prepare-stack-trace.js`, `test-util-getcallsites-preparestacktrace.js` and `test-shadow-realm-prepare-stack-trace.js` exit 0. Earlier drafts, whose reach was wider, also cleared the html-rewriter (207), plugin and resolve (506), streams (549), serve (295) and `node:http` (372) suites. Two failures in that area are not from this change, and each now has its own session. `test/regression/issue/013880.test.ts` leaks a global `Error.prepareStackTrace` into the shared test process and breaks 21 tests in `capture-stack-trace.test.js`. It reproduces on release 1.4.2: that file alone passes, and the two files together fail 21. `test/js/node/vm/script-leak.test.ts` sets an `isASAN` RSS threshold but no `isASAN` timeout, so it needs 25s against a 5s default on any `bun bd` build. #### Flake check The four new blocks passed 5 runs in a row, and 3 more with 24 CPU burners running, to stand in for a slow ASAN lane. Each reaction fixture needs a genuinely pending promise to make the native reaction exist. An already-resolved promise and a `queueMicrotask`-resolved promise both fail to install the suspension, which I checked, so the fixtures use `setTimeout(resolve, 1)` inside the fixture process. The test itself only awaits the process exit. Each reaction fixture also asserts `reactionFrameSeen=true`, so `leaked=undefined` cannot pass because the frame stopped appearing. </details> <!-- robobun:evidence:begin --> --- **[human-review]** gate passed · iteration 1 · 4 files touched <details><summary>fails on main (without fix)</summary> ```console ASAN without fix: 4 FAILED $ BUN_DEBUG_QUIET_LOGS=1 bun scripts/build.ts --profile=debug --quiet test "--reporter=junit" "--reporter-outfile=/tmp/pr_gate.xml" "test/js/node/v8/capture-stack-trace.test.js" bun test v1.4.3 (6a92015) test/js/node/v8/capture-stack-trace.test.js: (pass) Regular .stack [24.55ms] (pass) throw inside Error.prepareStackTrace doesnt crash [40.55ms] (pass) capture stack trace [12.79ms] (pass) capture stack trace with message [14.33ms] (pass) capture stack trace with constructor [11.08ms] (pass) capture stack trace limit [43.64ms] (pass) prepare stack trace [22.91ms] (pass) capture stack trace second argument [35.13ms] (pass) capture stack trace edge cases [23.74ms] (pass) Error.captureStackTrace installs .stack as non-enumerable [39.79ms] (pass) prepare stack trace call sites [24.06ms] (pass) sanity check [26.96ms] (pass) CallFrame isEval works as expected [16.18ms] (pass) CallFrame isTopLevel returns false for Function constructor [17.42ms] (pass) CallFrame.p.getThisgetFunction: strict/sloppy mode interaction [17.16ms] (pass) CallFrame.p.isConstructor [6.29ms] (pass) CallFrame.p.isNative [7.07ms] 726 | stderr: "pipe", 727 | }); 728 | 729 | const [stdout, st ... (truncated) release without fix: all passed bun test v1.4.3-canary.1 (8a7f62b) test/js/node/v8/capture-stack-trace.test.js: (pass) Regular .stack [0.14ms] (pass) throw inside Error.prepareStackTrace doesnt crash [0.09ms] (pass) capture stack trace [0.09ms] (pass) capture stack trace with message [0.06ms] (pass) capture stack trace with constructor [0.06ms] (pass) capture stack trace limit [0.19ms] (pass) prepare stack trace [0.11ms] (pass) capture stack trace second argument [0.17ms] (pass) capture stack trace edge cases [0.10ms] (pass) Error.captureStackTrace installs .stack as non-enumerable [0.22ms] (pass) prepare stack trace call sites [0.14ms] (pass) sanity check [0.11ms] (pass) CallFrame isEval works as expected [0.09ms] (pass) CallFrame isTopLevel returns false for Function constructor [0.09ms] (pass) CallFrame.p.getThisgetFunction: strict/sloppy mode interaction [0.10ms] (pass) CallFrame.p.isConstructor [0.03ms] (pass) CallFrame.p.isNative [0.03ms] (pass) CallFrame.p.getFunction does not expose the module loader reaction [9.47ms] (pass) CallFrame.p.getFunction does not expose the HTMLRewriter reaction [11.12ms] (pass) CallFrame.p.getFunction does not expose the serve reject reaction [11.29ms] (pass ... (truncated) ``` </details> <details><summary>passes on PR (with fix)</summary> ```console ASAN with fix: all passed $ BUN_DEBUG_QUIET_LOGS=1 bun scripts/build.ts --profile=debug --quiet test "--reporter=junit" "--reporter-outfile=/tmp/pr_gate.xml" "test/js/node/v8/capture-stack-trace.test.js" bun test v1.4.3 (6a92015) test/js/node/v8/capture-stack-trace.test.js: (pass) Regular .stack [22.42ms] (pass) throw inside Error.prepareStackTrace doesnt crash [33.22ms] (pass) capture stack trace [9.60ms] (pass) capture stack trace with message [10.80ms] (pass) capture stack trace with constructor [8.87ms] (pass) capture stack trace limit [32.05ms] (pass) prepare stack trace [17.69ms] (pass) capture stack trace second argument [24.93ms] (pass) capture stack trace edge cases [18.24ms] (pass) Error.captureStackTrace installs .stack as non-enumerable [30.12ms] (pass) prepare stack trace call sites [19.79ms] (pass) sanity check [21.41ms] (pass) CallFrame isEval works as expected [11.96ms] (pass) CallFrame isTopLevel returns false for Function constructor [12.53ms] (pass) CallFrame.p.getThisgetFunction: strict/sloppy mode interaction [13.89ms] (pass) CallFrame.p.isConstructor [5.20ms] (pass) CallFrame.p.isNative [5.26ms] (pass) CallFrame.p.getFunction does not expose the serve reject reaction ... (truncated) release with fix: all passed $ bun scripts/build.ts --profile=release [configured] bun-profile → bun (stripped) in 710ms (unchanged) ninja: Entering directory `/workspace/bun/build/release' [1/9] cxx obj/src/jsc/bindings/ZigGlobalObject.cpp.o [2/9] cxx obj/unified/UnifiedSource-src_jsc_bindings-1.cpp.o [3/9] cxx obj/unified/UnifiedSource-src_jsc_bindings-0.cpp.o [4/9] gen cpp.rs (cppbind) [4/9] cargo bun_runtime → libbun_runtime.a ^[[1m^[[92m Compiling^[[0m bun_core v0.0.0 (/workspace/bun/src/bun_core) ^[[1m^[[92m Compiling^[[0m bun_errno v0.0.0 (/workspace/bun/src/errno) ^[[1m^[[92m Compiling^[[0m bun_ptr v0.0.0 (/workspace/bun/src/ptr) ^[[1m^[[92m Compiling^[[0m bun_boringssl_sys v0.0.0 (/workspace/bun/src/boringssl_sys) ^[[1m^[[92m Compiling^[[0m bun_safety v0.0.0 (/workspace/bun/src/safety) ^[[1m^[[92m Compiling^[[0m bun_base64 v0.0.0 (/workspace/bun/src/base64) ^[[1m^[[92m Compiling^[[0m bun_cares_sys v0.0.0 (/workspace/bun/src/cares_sys) ^[[1m^[[92m Compiling^[[0m bun_zlib_sys v0.0.0 (/workspace/bun/src/zlib_sys) ^[[1m^[[92m Compiling^[[0m bun_zstd v0.0.0 (/workspace/bun/src/zstd) ^[[1m^[[92m Compiling^[[0m bun_picohttp v0.0.0 (/workspace/bun/src/picohttp) ^[[1m^[[92m Compiling^[[0m bun_brotli v ... (truncated) ``` </details> <details><summary>diff hotspot</summary> ``` src/jsc/bindings/CallSite.cpp | 31 +++- src/jsc/bindings/CallSite.h | 2 + src/jsc/bindings/CallSitePrototype.cpp | 21 +-- test/js/node/v8/capture-stack-trace.test.js | 264 ++++++++++++++++++++++++++++ 4 files changed, 297 insertions(+), 21 deletions(-) ``` </details> **gate history** · 3 passed · 0 rejected · iteration 1 <details><summary>evidence per changed file</summary> ``` file reads edits tests src/jsc/bindings/CallSite.cpp 5 11 34 src/jsc/bindings/CallSite.h 2 3 33 src/jsc/bindings/CallSitePrototype.cpp 2 2 33 test/js/node/v8/capture-stack-trace.test.js 6 10 33 ``` </details> **root cause** · written by the author bot `CallSite` treated a frame as non-strict whenever it lacked a CodeBlock with a strict executable, so native frames, builtin JS frames, and the internal async/generator body functions that JSC runs after the first suspension all had their callee stored and handed back through `getFunction()`; calling one of those internal callees with caller-chosen arguments let plain JavaScript write generator internal fields into arbitrary objects, which segfaulted the interpreter. The fix narrows the callee exposure in one place: the function is only stored when it is a `JSFunction` whose `FunctionExecuta… <!-- robobun:evidence:end -->
A handler can return Err(SuspensionRequest) to park the current rewritable unit on the heap and exit write()/end() with the non-poisoning RewritingError::Suspended; HtmlRewriter::resume() continues from a StateMachineBookmark. - take_owned() on each rewritable unit deep-copies borrowed byte slices so the unit outlives the write() stack frame - Arena::shift advances a start offset instead of memmoving the tail (a suspended rewrite re-feeds its unconsumed tail on every resume) - the pending captured-text flush is hoisted from Dispatcher::handle_tag into the lexer actions so every suspension point has a uniform shape - end_mut() is the non-consuming end() for a rewriter that may suspend in a document-end handler Bun uses this so an async JS content handler can suspend the rewrite and resume it from a promise reaction instead of spinning a nested event loop inside write(). See oven-sh/bun#36733.
|
The seventh |
A handler can return Err(SuspensionRequest) to park the current rewritable unit on the heap and exit write()/end() with the non-poisoning RewritingError::Suspended; HtmlRewriter::resume() continues from a StateMachineBookmark. - take_owned() on each rewritable unit deep-copies borrowed byte slices so the unit outlives the write() stack frame - Arena::shift advances a start offset instead of memmoving the tail (a suspended rewrite re-feeds its unconsumed tail on every resume) - the pending captured-text flush is hoisted from Dispatcher::handle_tag into the lexer actions so every suspension point has a uniform shape - end_mut() is the non-consuming end() for a rewriter that may suspend in a document-end handler Bun uses this so an async JS content handler can suspend the rewrite and resume it from a promise reaction instead of spinning a nested event loop inside write(). See oven-sh/bun#36733.
What
HTMLRewriter.transform()now streams: input body chunks flow throughlol-htmlinto an outputByteStreamas they arrive, with backpressure propagated end-to-end via theSinkHandle/SourceHandlepattern from #36087. Async content handlers no longer nest the event loop.Why
BufferOutputSinkfully buffered the source body viaValueBuffererbefore a singlerewriter.write()+end(), andhandler_callbackspunvm.wait_for_promise()six native frames deep inside lol-html for any handler that returned a Promise. That meant handlers fired only at source-end (TTFB = full download),.body/Bun.servesaw an empty stream (#6068, #19305), JS-backedReadableStreaminputs were rejected outright (#11758, #14216), and the nested loop was a known hazard class (deadlocks, ready-poll clobbering, pending-exception leaks).lol-html could not be suspended before because its tokens are stack locals borrowing a stack-local lexeme; returning from
write()destroys them, and an async handler must be able to mutate the element after itsawait.lol-html fork (
oven-sh/lol-html, branchbun)A handler returns
Err(SuspensionRequest)to suspend. The in-flight unit is deep-copied onto the heap,write()/end()return the non-poisoningRewritingError::Suspended, andHtmlRewriter::resume()continues from aStateMachineBookmark. The pending captured-text flush is hoisted fromDispatcher::handle_taginto the lexer actions so every suspension point has a uniform shape.Arena::shiftadvances a start offset instead of memmoving the tail (a suspended rewrite re-feeds its unconsumed tail on every resume). 23 in-crate tests added; the new.github/workflows/lolhtml.ymlruns the fork's owncargo testat the pinned commit, gated onscripts/build/deps/lolhtml.ts. The fork is agithub-archivesource (no patch file), so rebasing onto a new upstream tag is agit rebase --ontoin the fork plus a commit bump here.Bun side (
RewriterPipereplacesBufferOutputSink)HTMLRewriterTransformJS cell (not user-visible) owns the pipe; its GC finalizer frees it, and nothing pins anything. Liveness is plain GC edges: the output Response'stransformWriteBarrier slot and the.then()context of a suspended handler's (or the JS pump's) promise reach the cell, the cell's five slots root the Response, the input/output streams, the pending flush promise, and a captured handler error, and a wired native source'sownerWriteBarrier slot (new on the generated NewSource cells) points back at the cell, so I/O that roots the source (a FetchTasklet, a FileReader's read refs, a reader on the output stream) roots the rewrite for exactly the window the rawSinkHandle/producer backrefs are wired. A handler promise collected without settling letsfinalizedefer to an event-loop task (abandon_suspension) that rejects the body before freeing; the pipe holds one native+1on the Response (released inDrop) so that task can still reach the body. No intrusive refcount, noStrongfields on the pipe;finish()frees the boxed lol-html state machine eagerly, andfail()/output-cancel close the upstream producer instead of draining it to EOF.Response's body isLocked(PendingValue { task: pipe, on_start_streaming, on_readable_stream_available, producer: SourceHandle::HTMLRewriter }), theFetchTasklet::to_body_valueshape. TheByteStreamis created lazily when a consumer reads.bodyor a body-mixin method; until then, rewriter output buffers in aVec<u8>handed over asDrainResult::Owned.FetchTasklet::start_request_stream. NativeByteStream/FileReadersources wirebyte_stream.sink = SinkHandle::HTMLRewriter+lock_native+drain(); other stream kinds go throughJSSink::<RewriterPipe>::assign_to_stream(newHTMLRewriterSinkcodegen entry). Materialized bodies (InternalBlob/bytesBlob/WTFStringImpl) feed synchronously.RewriterPipe::writefeeds one chunk throughrewriter.write()(output chunks push viaByteStream::on_data). If the output is paused or a handler suspended,writereturnsWritable::Backpressure;ByteStream::resume→SourceHandle::HTMLRewriter::on_ready→pipe.resume()drainspending_inputtheninput_source.ready().handler_callbackreturnsHandlerOutcome::{Continue, Stop, Suspend}. On a pending Promise it runs one microtask checkpoint (process.nextTickthen promise jobs, never the loop); a genuinely pending Promise suspends. The JS wrapper is retargeted at the heap-parked token so post-awaitmutations land where they should. The.then()context is the Transform cell itself (the reactions recover the pipe viafrom_js), so a handler promise collected without settling abandons the parked rewrite instead of leaking it.handler_erroron the pipe replaces the stackCell+unhandled_pending_rejection_to_captureoverride. A handler error on a streaming input now rejects the body with the real error instead ofThe rewriter has been stopped..AttributeIteratorholds a backref to theElementplus an index instead of a boxedslice::Iter, sofor (const [k, v] of el.attributes) { await ... }keeps working across a suspension.Deletions
ValueBufferer(~430 lines ofBody.rs) and its host-fn exports;SinkHandle::ValueBufferer+SinkWriteFn;JSSink<ArrayBufferSink>::detach_self;crate::Error::{StreamAlreadyUsed, InvalidStream, UnsupportedStreamType};NativePromiseContext::Tag::BodyValueBufferer(ordinal 4 reused forHTMLRewriterSuspension); the twoBun__BodyValueBufferer__*PromiseFunctions(slots reused forBun__HTMLRewriter__onHandler{Resolve,Reject}).Behavior changes
transform(string)/transform(ArrayBuffer)throw fromtransform(). EveryResponseinput rejects its output body instead. Five existing tests that pinned the old timing-dependent split are updated. Input-body errors (already-failed or aborted body) still throw synchronously fromtransform().transform(string)/transform(ArrayBuffer)throw aTypeError(pass a Response and await its body) instead of spinning. A Promise that settles within a microtask checkpoint still works.unhandledRejectioninstead of being captured and thrown fromtransform().transform()types corrected:Bun.BufferSourcereturnsArrayBuffer(it always has at runtime);Blobremoved from the overload (it threw at runtime).Bun.servewith an HTMLRewriter-produced response body defers status/headers until the first body byte or clean end, so a handler that fails before emitting any bytes is routed to the server'''serror()hook instead of committing200 OKthen force-closing the connection. Headers never preceded the first byte on this path before either (the old implementation buffered the whole rewrite). All other native-ByteStream bodies (proxiedfetch(), S3, spawn stdout) keep sending status/headers immediately, and JSReadableStreambodies (do_render_stream) are unchanged.docs/runtime/html-rewriter.mdxandpackages/bun-types/html-rewriter.d.tscover 1-4.Verification
test/js/workerd/html-rewriter.test.js: 107 pass, 0 fail under ASAN debug and underBUN_JSC_validateExceptionChecks=1. New coverage: async element/text/comment/doctype/onEndTag/document-end handlers,Bun.gc(true)while an element is heap-parked across anawait, re-suspension by a second handler on the same element, nestedtransform()inside a suspended handler, strict document-order across 8 awaiting handlers,Bun.servewith a live client, client abort mid-suspension, the consumer matrix for a pending output body, JS-backed andtype:'direct'ReadableStreaminputs.test/js/workerd/html-rewriter-leak.test.ts: protected-object +Responsecount regression over 120 suspending rewrites; a never-settling handler rejects the body instead of leaking.works with payload of type direct→ERR_STREAM_CANNOT_PIPEon the released bun.bun-typesgreen,cargo clippy -p bun_runtimeclean.vendor/lolhtml/.refre-fetch of the fork verified through the real ninja edge.Fixes #11758
Fixes #14216
Fixes #6068
Fixes #19305
Closes #33243 (same lol-html fork, different input layer), Closes #35324 (ResumableSink, deleted in #36087), Closes #32988 (per-chunk ValueBufferer callback).
no test proof · iteration 20 · Platform-specific test(s) that do not run on this machine. Deferring to CI, which covers all platforms: test/js/bun/io/bun-write.test.js test/js/node/process/process.test.js test/js/workerd/html-rewriter-leak.test.ts