Skip to content

HTMLRewriter: use the lol_html Rust crate directly instead of its C API - #33048

Merged
Jarred-Sumner merged 1 commit into
mainfrom
farm/13b38596/lolhtml-rust-crate-bindings
Jun 29, 2026
Merged

Jarred-Sumner merged 1 commit into
mainfrom
farm/13b38596/lolhtml-rust-crate-bindings

Conversation

@robobun

@robobun robobun commented Jun 29, 2026 •

Copy link
Copy Markdown
Collaborator

Precursor to #32988, at @Jarred-Sumner's direction there:

This code uses far too much unsafe. Instead, let's refactor our bindings to use the Rust LOLHTML crate directly instead of going through their C API.

Problem

Bun's runtime is Rust, but its lol-html bindings still went through lol-html's C API: a 1,299-line FFI layer (src/lolhtml_sys/, 72 extern "C" declarations) wrapping opaque handles, plus extern "C" output-sink and content-handler trampolines that manufacture aliased &mut references from userdata pointers, and a take-once thread-local last-error buffer (the direct cause of two correctness findings in #32988's review). That arrangement is a leftover from the Zig runtime, which had no other option; from Rust it is a Rust -> extern "C" -> Rust round trip for no benefit, and it is the structural reason HTMLRewriter code has been so unsafe-heavy.

lol_html (v2.7.2) is already vendored at vendor/lolhtml and was already in the cargo dependency graph (the c-api crate we consumed is a thin wrapper over it), so using it directly is a dependency-graph no-op.

What changed

  • src/lolhtml_sys/ is deleted entirely: 1,340 lines, 74 unsafe blocks, all 72 extern "C" declarations. lol_html = { path = "vendor/lolhtml" } becomes a direct workspace dependency of bun_runtime and bun_bundler; the vendor fetch step is unchanged.
  • src/runtime/api/html_rewriter.rs builds a lol_html::Settings from the selectors and handlers accumulated on LOLHTMLContext and constructs HtmlRewriter directly. Handler registration is safe closures capturing raw NonNull<Handler> pointers into the Rc<RefCell<LOLHTMLContext>> that already outlives the rewriter (the same invariant the C userdata relied on, now visible in Rust). The output sink is a safe impl lol_html::OutputSink for SinkRef(*mut BufferOutputSink); lol-html signals completion with a zero-length final chunk. HtmlRewriter::end(self) consumes, so the sink holds the rewriter behind a raw heap pointer read into a local before each call (the same re-entrancy discipline the file already uses) and never calls end after a failed write.
  • The eight JS wrapper classes keep their exact shape: a Cell<*mut X> handle that is only valid inside its content handler and is nulled before the handler returns. The handle is now a real lol_html::html_content type behind one documented lifetime-erasure helper (cell_get), and every wrapper method is a safe call into it, because the lol_html content operations are safe Rust.
  • The take-once thread-local error channel is gone. lol_html's methods return typed errors (RewritingError, SelectorError, TagNameError, AttributeNameError, CommentTextError), so error text comes from the error the failing call returned, never a global consumed as a side channel.
  • src/bundler/HTMLScanner.rs and src/bundler/linker_context/generateCompileResultForHtmlChunk.rs migrate the same way.
  • The two patches/lolhtml/* patches only served the now-unused c-api crate and are removed, as is the orphaned DISABLE_LOLHTML feature flag (its only reader was the deleted FFI crate).
  • HTMLRewriterLoader (239 lines) is deleted outright rather than migrated. It is a pub Zig-port carryover with zero callers: nothing constructs one, it is absent from the JSSink codegen registry (src/codegen/generate-jssink.ts), and no .rs/.ts/.cpp/.h/.zig file references it outside its own definition. Being pub is what kept it off the dead_code lint. Review (claude, then CodeRabbit) found three distinct latent memory-safety bugs in it, none reachable; deleting the class fixes all three at the root instead of hardening code that cannot run.

Net: 26 files, +877 / -2489.

One behavioral fix: onEndTag() no longer leaks its callback

This is the only user-visible behavior change, and it falls out of the model change rather than being a goal.

The C API binds an element's end-tag handler as a raw (function pointer, userdata) pair, so Bun heap-allocated a per-call handler holding a JSValue::protect() GC root on the callback and handed the raw pointer to lol-html. Nothing ever freed it on the success path (the only free was the registration-error branch), so every element.onEndTag(fn) call permanently leaked one protected JS function, plus everything its closure captured, for the life of the process. A long-running Bun.serve + HTMLRewriter + onEndTag grows without bound.

lol_html's Rust API instead takes an owning Box<dyn FnOnce(..)>. The closure owns the protection; lol-html owns the closure and drops it (releasing the root) whether or not the end tag is ever reached. The leak is structurally impossible.

Covered by an exact-count regression test in test/js/workerd/html-rewriter-leak.test.ts using bun:jsc's heapStats().protectedObjectTypeCounts (no RSS threshold, stable on debug builds): two passes of 400 onEndTag registrations after a baseline leave 800 extra protected functions on main and 0 with this change.

Every other HTMLRewriter and bundler-HTML test passes. Besides the new leak file, the only test change is four added cases in html-rewriter.test.js covering the element.tagName, endTag.name, and comment.text setters, which had no coverage anywhere in the tree; no existing test is modified.

Verification

  • cargo check -p bun_runtime -p bun_bundler, cargo clippy -p bun_runtime -p bun_bundler, and cargo fmt --all --check: clean.
  • All five HTMLRewriter test files, 80 pass / 0 fail (75 pre-existing, unmodified, plus the new leak test and four new setter tests):
    bun bd test test/js/workerd/html-rewriter.test.js test/js/workerd/html-rewriter-end-error.test.ts test/js/workerd/html-rewriter-leak.test.ts test/js/web/html/html-rewriter-doctype.test.ts test/regression/issue/htmlrewriter-additional-bugs.test.ts
  • The new leak test on main (fail-before): expect(after - before).toBe(0) receives 800.
  • All three bundler HTML test files, 30 pass / 0 fail:
    bun bd test test/bundler/bundler_html.test.ts test/bundler/bundler_html_server.test.ts test/bundler/html-import-manifest.test.ts
  • grep -rn bun_lolhtml_sys --include='*.rs' --include='*.toml' . outside vendor/: nothing.
  • grep 'extern "C"' src/runtime/api/html_rewriter.rs src/bundler/HTMLScanner.rs: nothing.

Honest unsafe accounting

The unsafe that was about the lol-html boundary is gone: the 74-block FFI layer, every trampoline, every opaque-handle call. src/bundler/HTMLScanner.rs drops from 13 unsafe blocks to 3.

html_rewriter.rs itself still contains unsafe, and it is worth being precise about what that is, because almost none of it is about lol-html anymore:

  1. One documented lifetime-erasure helper (cell_get) at the handler-to-JS-wrapper boundary. A JS object cannot carry a Rust borrow lifetime, so a handler's &mut Element<'r, 't> has to be parked as a raw pointer for the duration of the JS callback and nulled before the handler returns; that is inherent to exposing lol-html handles to JS at all, and it is now one place instead of being baked into 72 FFI signatures. Every call site obeys its ordering rule: a setter given a raw JSValue runs the value's ToString (arbitrary user JS, which can re-enter cell_get on the same wrapper) before the borrow is taken, through one shared setter_utf8_arg helper that carries the rule.
  2. The pre-existing JsClass refcount and *mut Response plumbing, which is independent of which lol-html API is used and is unchanged by this PR.

The practical payoff lands in #32988: its streaming change was fighting the C boundary (four review rounds found five real bugs, two of them caused directly by the take-once error buffer and the opaque-handle poisoning model). Rebased on this, that diff sheds the classes of unsafe that made it hard to get right.

Relationship to #32988

#32988 (make HTMLRewriter actually stream) stays open and will be rebased on top of this once it lands. The two are deliberately separate so each is reviewable on its own: this one is the binding migration, and that one is the streaming behavior change.

@coderabbitai

coderabbitai Bot commented Jun 29, 2026 •

Copy link
Copy Markdown
Contributor

Review Change Stack

Warning

Review limit reached

@robobun, you've reached your PR review limit, so we couldn't start this review.

Next review available in: 5 minutes

Enable usage-based reviews in Billing to review now. Otherwise, wait until the next included review is available.
You're only billed for reviews past your plan's rate limits ($0.25/file).

How can I continue?

After more reviews become available, a review can be triggered using the @coderabbitai review command as a PR comment. Alternatively, push new commits to this PR.

To avoid repeated limits, reduce automatic review volume by pausing incremental auto-reviews earlier, using label-based review opt-in, excluding WIP or generated PR titles, or requesting reviews manually when the PR is ready. If your team needs uninterrupted high-volume reviews, an organization admin can enable usage-based reviews.

How do review limits work?

CodeRabbit enforces per-developer PR review limits for each organization. Most developers receive the normal plan review availability.

For paid Pro and Pro+ PR reviews, CodeRabbit uses adaptive limits for sustained high-volume activity. When a developer's recent PR review activity reaches the 95th percentile or higher among CodeRabbit users, additional reviews become available more gradually as earlier reviews age out of the rolling window.

Please refer docs for additional details.

Review details
⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: ASSERTIVE

Plan: Pro

Run ID: 4141d660-2080-4f70-b554-1bf91e27ecc7

📥 Commits

Reviewing files that changed from the base of the PR and between 81ccde3 and 8a683c7.

⛔ Files ignored due to path filters (1)
  • Cargo.lock is excluded by !**/*.lock
📒 Files selected for processing (25)
  • .github/workflows/clippy.yml
  • Cargo.toml
  • patches/lolhtml/0001-rlib-only.patch
  • patches/lolhtml/0002-quiet-build-script-linker-warning.patch
  • rustfmt.toml
  • scripts/build/bun.ts
  • scripts/build/cargo-config.ts
  • scripts/build/deps/lolhtml.ts
  • scripts/build/rust.ts
  • scripts/rust-miri.ts
  • src/bun_core/feature_flags.rs
  • src/bundler/Cargo.toml
  • src/bundler/HTMLScanner.rs
  • src/bundler/bundle_v2.rs
  • src/bundler/linker_context/generateCompileResultForHtmlChunk.rs
  • src/lolhtml_sys/Cargo.toml
  • src/lolhtml_sys/lib.rs
  • src/lolhtml_sys/lol_html.rs
  • src/runtime/Cargo.toml
  • src/runtime/api.rs
  • src/runtime/api/html_rewriter.rs
  • src/runtime/api/lolhtml_jsc.rs
  • test/js/workerd/html-rewriter-leak.test.ts
  • test/js/workerd/html-rewriter.test.js
  • test/leaksan.supp

Walkthrough

Removes the bun_lolhtml_sys C-FFI wrapper crate entirely. lol_html is now vendored at vendor/lolhtml and declared as a direct workspace path dependency. HTMLScanner, generateCompileResultForHtmlChunk, and html_rewriter.rs are rewritten to use the lol_html Rust API, replacing all FFI handle/callback plumbing with Rust closures and safe typed wrappers.

Changes

Remove bun_lolhtml_sys FFI layer; use lol_html Rust API directly

Layer / File(s) Summary
Workspace and build system wiring
Cargo.toml, scripts/build/deps/lolhtml.ts, scripts/rust-miri.ts, src/bun_core/feature_flags.rs, src/bundler/Cargo.toml, src/runtime/Cargo.toml, .github/workflows/clippy.yml, rustfmt.toml, scripts/build/bun.ts, scripts/build/cargo-config.ts, scripts/build/rust.ts, test/leaksan.supp
Removes src/lolhtml_sys from workspace members, removes bun_lolhtml_sys workspace dependency, adds lol_html = { path = "vendor/lolhtml" } as a vendored workspace dependency, removes patches from the lolhtml build dep config, updates miri's Cargo.toml path gate, removes DISABLE_LOLHTML feature flag, and updates associated build/CI comments.
Delete bun_lolhtml_sys crate and lolhtml_jsc bridge
src/lolhtml_sys/Cargo.toml, src/lolhtml_sys/lib.rs, src/lolhtml_sys/lol_html.rs, src/runtime/api/lolhtml_jsc.rs, src/runtime/api.rs
Deletes the bun_lolhtml_sys package manifest, all FFI bindings and opaque handle types in lol_html.rs, and the lolhtml_jsc JSC bridge (HTMLString-to-JSValue). Removes the lolhtml_jsc public module declaration from api.rs.
HTMLScanner migrated to lol_html Rust API
src/bundler/HTMLScanner.rs
Rewrites HTMLScanner/HTMLProcessor to build element_content_handlers closures for lol_html::HtmlRewriter; changes TagHandler field types from &'static [u8] to &'static str; updates HTMLProcessorHandler trait and on_tag signatures to Element<'_, '_>; replaces C-API "last error" retrieval with RewritingError::to_string().
generateCompileResultForHtmlChunk migrated
src/bundler/linker_context/generateCompileResultForHtmlChunk.rs
Updates imports to lol_html interfaces; adds set_attribute UTF-8 helper; changes HTMLLoader::on_tag parameter to Element<'_, '_>; replaces extern "C" end-tag Directive handlers with register_end_tag_handler closures using HandlerResult; updates cold-path injection to use String-based helpers.
LOLHTMLContext, build_settings, and handler closure construction
src/runtime/api/html_rewriter.rs (lines 1–406)
Introduces Raw* type aliases and cell_get helper; rewrites LOLHTMLContext to store selector and handler boxes; rewrites build_settings to produce fresh lol-html handler vectors via NonNull closures; updates lol_content_ops! macro and HTMLRewriter lifecycle wiring to use the new context.
HTMLRewriterLoader, BufferOutputSink, and output sink wiring
src/runtime/api/html_rewriter.rs (lines 550–1230)
Rewrites HTMLRewriterLoader to store a heap-allocated lol_html::HtmlRewriter pointer using null-then-take semantics for write_bytes/end/finalize; introduces LoaderSinkRef and SinkRef OutputSink impls; refactors BufferOutputSink init, run_output_sink, and drop around the new heap rewriter lifecycle.
JS wrapper types updated to Raw* pointer types
src/runtime/api/html_rewriter.rs (lines 1249–2531)
Updates DocumentHandler, ElementHandler, TextChunk, DocType, DocEnd, Comment, EndTag, AttributeIterator, and Element wrappers to use Raw* pointer types and cell_get; rewrites attribute getters/setters with utf8_or_throw and opt_string_to_js_or_null; rewrites on_end_tag_ closure registration via erased raw-pointer callbacks.
Tests and leak suppressions
test/js/workerd/html-rewriter-leak.test.ts, test/js/workerd/html-rewriter.test.js, test/leaksan.supp
Adds onEndTag callback GC regression test using heapStats; adds setter coercion and detached-wrapper behavior tests for tagName, endTag.name, and comment.text; removes the now-obsolete lolhtml_sys.lol_html.HTMLString.toString leak suppression.

Possibly related PRs

  • oven-sh/bun#30876: Introduces the scripts/rust-miri.ts cargo-miri runner that this PR modifies to point at vendor/lolhtml/Cargo.toml instead of vendor/lolhtml/c-api/Cargo.toml.
  • oven-sh/bun#32927: Modifies BufferOutputSink::on_finished_buffering error propagation in the same html_rewriter.rs file that this PR substantially refactors.

Suggested reviewers

  • dylan-conway
  • Jarred-Sumner
🚥 Pre-merge checks | ✅ 4
✅ Passed checks (4 passed)
Check name Status Explanation
Title check ✅ Passed The title clearly states the main change: switching HTMLRewriter to the lol_html Rust crate instead of the C API.
Linked Issues check ✅ Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check ✅ Passed Check skipped because no linked issues were found for this pull request.
Description check ✅ Passed The description clearly explains the change and includes verification details, even though it doesn't use the repo's exact template headings.

Comment @coderabbitai help to get the list of available commands.

@robobun

robobun commented Jun 29, 2026 •

Copy link
Copy Markdown
Collaborator Author
Updated 3:56 AM PT - Jun 29th, 2026

❌ @robobun, your commit 8a683c7 has 2 failures in Build #66697 (All Failures):


🧪   To try this PR locally:

bunx bun-pr 33048

That installs a local version of the PR into your bun-33048 executable, so you can run:

bun-33048 --bun

@github-actions

Copy link
Copy Markdown
Contributor

Found 1 issue this PR may fix:

  1. HTMLRewriter: thrown handler exceptions leak protected Exception roots #31804 - This PR structurally eliminates the thread-local error channel whose missing unprotect() call caused the exception root leak; the new code uses direct return values for error propagation instead.

If this is helpful, copy the block below into the PR description to auto-close this issue on merge.

Fixes #31804

🤖 Generated with Claude Code

@robobun
robobun force-pushed the farm/13b38596/lolhtml-rust-crate-bindings branch from 88d64dd to 5a4a8b9 Compare June 29, 2026 06:32
@robobun

robobun commented Jun 29, 2026

Copy link
Copy Markdown
Collaborator Author

Good suggestion from the issue finder, but after checking I am not going to claim #31804, because it is not actually fixed by this PR.

#31804 is about thrown handler exceptions leaking a protected root. Its mechanism is the JS-exception capture path in handler_callback (exc_value.protect() written into the VM's capture slot), which is a different channel from lol-html's take-once thread-local last-error buffer that this PR removes. Both exc_value.protect() sites exist, unchanged, before and after this change (2 on main, 2 here), and the lol-html last-error is a String/BunString, never a protected JSValue, so eliminating it cannot release a GC root. I'd rather not auto-close an open leak report on a conflation.

For what it is worth, this PR does fix a different, previously unreported leak that also falls out of the binding change: every successful element.onEndTag(callback) call permanently leaked a JSValue::protect() root on its callback (the C API handed lol-html a raw (fn pointer, userdata) pair that nothing ever freed; the Rust crate takes an owning Box<dyn FnOnce> that is dropped). That one is covered by a new exact-count regression test in test/js/workerd/html-rewriter-leak.test.ts: 800 leaked protected functions on main, 0 here. The description now documents it.

Comment thread src/runtime/api.rs Resolved
Comment thread src/runtime/api/html_rewriter.rs Resolved
@robobun
robobun force-pushed the farm/13b38596/lolhtml-rust-crate-bindings branch from 5a4a8b9 to af8f092 Compare June 29, 2026 06:51
Comment thread src/runtime/api/html_rewriter.rs Resolved
@robobun
robobun force-pushed the farm/13b38596/lolhtml-rust-crate-bindings branch from af8f092 to 81ccde3 Compare June 29, 2026 07:37

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 7

🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In `@Cargo.toml`:
- Around line 105-108: `vendor/lolhtml` is only materialized by the
ninja/bootstrap flow, so plain Cargo commands can still fail on a fresh
checkout. Add or wire up a direct bootstrap path for the `lol_html` path
dependency in the main Cargo setup so `cargo check -p ...` and rust-analyzer can
fetch/prepare `vendor/lolhtml` before resolution; use the existing
`clone-lolhtml` source in the build scripts and the `lol_html = { path =
"vendor/lolhtml" }` dependency as the key symbols to update.

In `@src/bundler/linker_context/generateCompileResultForHtmlChunk.rs`:
- Around line 97-107: The set_attribute helper in
generateCompileResultForHtmlChunk currently panics when either attribute name or
value is not UTF-8, which makes HTML rewriting crash on byte-oriented paths such
as import_record.path.pretty. Update the rewrite path in the HTML chunk
generation flow to convert these bytes into a valid string/URL-safe
representation before calling Element::set_attribute, or change set_attribute
and its callers to return a typed error that is propagated instead of panicking.
Use the existing set_attribute helper and the import_record.path.pretty call
sites as the main places to adjust.

In `@src/runtime/api/html_rewriter.rs`:
- Around line 702-710: The write_bytes path in HtmlRewriter is vulnerable to
re-entrant finalize/done freeing the active rewriter while write() is still
running. Update write_bytes so it temporarily nulls or takes self.rewriter
before calling (*self.rewriter).write(bytes), then restores the pointer on
success or destroys the taken rewriter on failure; this should prevent
LoaderSinkRef::handle_chunk -> write_to_destination -> done/finalize from
dropping the live HtmlRewriter during the write call.
- Around line 744-759: HtmlRewriter::end currently ignores the result of the
underlying rewriter shutdown and can still call done() afterward, which may
double-complete the sink. In end(), capture the return value from the unsafe
bun_core::heap::take(rewriter).end() call, and if it fails, propagate that
RewritingError through self.fail() or return it instead of proceeding. Also
ensure self.done() is only reached when the final empty-chunk path from
LoaderSinkRef::handle_chunk() has not already completed the stream, so end()
does not signal success twice.
- Around line 2295-2301: The has_attribute_() path currently throws on invalid
UTF-8 because it uses utf8_or_throw(), unlike get_attribute_() which handles
non-UTF-8 names non-throwingly. Update has_attribute_() in html_rewriter.rs to
treat an invalid ZigString attribute name as a simple false result, preserving
non-throwing hasAttribute() behavior. Keep the existing el lookup and
JSValue::FALSE fallback, but replace the UTF-8 conversion/error path so no
HTMLRewriterError is surfaced for malformed attribute names.

In `@test/js/workerd/html-rewriter-leak.test.ts`:
- Around line 1-43: This regression test is in the wrong place; move the
onEndTag callback leak coverage into the existing HTMLRewriter test module
instead of keeping a new sibling file. Add the test case to the current
HTMLRewriter suite that already covers this behavior, and keep the existing
helper logic like rewriteWithEndTagHandlers and protectedFunctions intact while
relocating it so the module’s coverage stays centralized.

In `@test/js/workerd/html-rewriter.test.js`:
- Around line 1227-1249: The detached-wrapper no-op/no-coercion test only covers
element.tagName, but the same behavior also needs to be verified for endTag.name
and comment.text. Add similar detached-wrapper assertions in
html-rewriter.test.js using the existing HTMLRewriter handler patterns so that
setters on detached endTag and comment wrappers also return without coercing
toString and leave the properties unchanged.
🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

  • Push a commit to this branch (recommended)
  • Create a new PR with the fixes

ℹ️ Review info
⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: ASSERTIVE

Plan: Pro

Run ID: 87d7aa59-f09f-44db-b99e-0e8561372333

📥 Commits

Reviewing files that changed from the base of the PR and between f789198 and 81ccde3.

⛔ Files ignored due to path filters (1)
  • Cargo.lock is excluded by !**/*.lock
📒 Files selected for processing (25)
  • .github/workflows/clippy.yml
  • Cargo.toml
  • patches/lolhtml/0001-rlib-only.patch
  • patches/lolhtml/0002-quiet-build-script-linker-warning.patch
  • rustfmt.toml
  • scripts/build/bun.ts
  • scripts/build/cargo-config.ts
  • scripts/build/deps/lolhtml.ts
  • scripts/build/rust.ts
  • scripts/rust-miri.ts
  • src/bun_core/feature_flags.rs
  • src/bundler/Cargo.toml
  • src/bundler/HTMLScanner.rs
  • src/bundler/bundle_v2.rs
  • src/bundler/linker_context/generateCompileResultForHtmlChunk.rs
  • src/lolhtml_sys/Cargo.toml
  • src/lolhtml_sys/lib.rs
  • src/lolhtml_sys/lol_html.rs
  • src/runtime/Cargo.toml
  • src/runtime/api.rs
  • src/runtime/api/html_rewriter.rs
  • src/runtime/api/lolhtml_jsc.rs
  • test/js/workerd/html-rewriter-leak.test.ts
  • test/js/workerd/html-rewriter.test.js
  • test/leaksan.supp
💤 Files with no reviewable changes (9)
  • patches/lolhtml/0001-rlib-only.patch
  • src/bun_core/feature_flags.rs
  • src/runtime/api/lolhtml_jsc.rs
  • src/lolhtml_sys/Cargo.toml
  • patches/lolhtml/0002-quiet-build-script-linker-warning.patch
  • src/lolhtml_sys/lib.rs
  • test/leaksan.supp
  • src/lolhtml_sys/lol_html.rs
  • src/runtime/api.rs

Comment thread Cargo.toml Resolved
Comment thread src/runtime/api/html_rewriter.rs Resolved
Comment thread src/runtime/api/html_rewriter.rs Resolved
Comment thread src/runtime/api/html_rewriter.rs Resolved
Comment thread test/js/workerd/html-rewriter.test.js Resolved
Bun's runtime is Rust, but its lol-html bindings still went through
lol-html's C API: a 1,299-line FFI layer (src/lolhtml_sys, 72
extern "C" declarations) wrapping opaque handles, plus extern "C"
output-sink and handler trampolines that manufacture &mut references
from userdata pointers, and a take-once thread-local last-error buffer.
That arrangement is a leftover from the Zig runtime, which had no other
option; from Rust it is a Rust -> extern "C" -> Rust round trip for no
benefit, and it is why HTMLRewriter code has been so unsafe-heavy.

Switch the runtime and the bundler to the lol_html Rust crate (v2.7.2,
already vendored at vendor/lolhtml and already in the cargo dependency
graph via the c-api path dep). Every existing HTMLRewriter and
bundler-HTML test passes unchanged.

What this removes:
- src/lolhtml_sys/ entirely (1,340 lines, 74 unsafe blocks, all 72
  extern "C" declarations).
- Every extern "C" trampoline in the two consumers; zero remain.
- The take-once thread-local last-error channel. lol_html's methods
  return typed errors (RewritingError, SelectorError, TagNameError,
  AttributeNameError, CommentTextError), so error text now comes from
  the error the failing call returned, not a global consumed as a side
  channel.
- The two vendor patches, which only served the now-unused c-api crate.
- The orphaned DISABLE_LOLHTML feature flag (its only reader was the
  deleted FFI crate).

What replaces it:
- html_rewriter.rs builds a lol_html::Settings from the selectors and
  handlers accumulated on LOLHTMLContext and constructs HtmlRewriter
  directly. Handler registration is safe closures capturing raw
  NonNull<Handler> pointers into the Rc<RefCell<LOLHTMLContext>> that
  already outlives the rewriter. The output sink is a safe
  impl lol_html::OutputSink for SinkRef(*mut BufferOutputSink);
  lol-html signals completion with a zero-length final chunk.
  HtmlRewriter::end(self) consumes, so the sink holds the rewriter
  behind a raw heap pointer read into a local before each call (the
  same re-entrancy discipline the file already uses) and never calls
  end after a failed write.
- The eight JS wrapper classes keep their exact shape: a Cell<*mut X>
  handle valid only inside its handler, nulled before the handler
  returns. The handle is now a real lol_html type behind a single,
  documented lifetime-erasure helper (cell_get); every wrapper method
  is a safe call into it, because the lol_html content operations are
  safe Rust. The unsafe that remains in html_rewriter.rs is the
  pre-existing JsClass refcount and FFI plumbing, which is independent
  of which lol-html API is used.
- src/bundler/HTMLScanner.rs and generateCompileResultForHtmlChunk.rs
  migrate the same way (HTMLScanner drops from 13 unsafe blocks to 3).
- lol_html = { path = "vendor/lolhtml" } becomes a direct workspace
  dependency; the vendor fetch step is unchanged.

One user-visible fix falls out of the model change: element.onEndTag()
no longer leaks its callback. The C API bound end-tag handlers as a raw
(function pointer, userdata) pair, so Bun heap-allocated a per-call
handler holding a JSValue::protect() GC root on the callback and never
freed it on the success path. Every onEndTag call leaked one protected
function (and everything its closure captured) for the life of the
process. lol_html's Rust API takes an owning FnOnce box, which is
dropped, releasing the root, whether or not the end tag is ever
reached. Covered by a new exact-count regression test in
test/js/workerd/html-rewriter-leak.test.ts.
@robobun
robobun force-pushed the farm/13b38596/lolhtml-rust-crate-bindings branch from 81ccde3 to 8a683c7 Compare June 29, 2026 08:02
@robobun

robobun commented Jun 29, 2026

Copy link
Copy Markdown
Collaborator Author

CodeRabbit's full review came back with seven findings. All seven are now closed; summary for anyone triaging.

Acted on (three findings, two changes):

  • The re-entrant write() use-after-free and the end() double-complete were both inside HTMLRewriterLoader, which has zero callers: nothing constructs one, it is absent from the JSSink codegen registry (src/codegen/generate-jssink.ts), and no .rs/.ts/.cpp/.h/.zig file references it by name. Being pub is what kept it off the dead_code lint. It is a Zig-port carryover that was never wired up, and three reviews have now found three distinct latent bugs in it, so I deleted the whole class (243 lines) in 8a683c7 rather than harden code that cannot run. cargo check -p bun_runtime is clean and the full build links.
  • Extended the detached-wrapper setter test to cover all three wrapper types (element.tagName, endTag.name, comment.text), not just tagName.

Declined with evidence (four): the vendor/lolhtml bootstrap requirement, the bundler set_attribute panic, and hasAttribute's behavior on a non-UTF-8 name are all byte-for-byte unchanged from main, and html-rewriter-leak.test.ts is not a new file (it exists on the merge base and already held an HTMLRewriter leak test). Each decline carries its evidence in the thread, and CodeRabbit re-checked and withdrew all four.

At 8a683c7: 80 pass / 0 fail across all five HTMLRewriter test files, 30 pass / 0 fail across the bundler HTML suites.

@claude claude Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

All earlier review threads are resolved and the bug hunt found nothing new, but this is a large rewrite of the HTMLRewriter unsafe/lifetime-erasure layer and the live BufferOutputSink re-entrancy path — worth a human pass before merge.

Extended reasoning...

Overview

This PR replaces Bun's lol-html C-API FFI layer (src/lolhtml_sys/, ~1,300 lines, 72 extern "C" declarations) with direct use of the lol_html Rust crate. It rewrites src/runtime/api/html_rewriter.rs (handler closure construction, build_settings, cell_get lifetime erasure, SinkRef output sink, onEndTag ownership), migrates src/bundler/HTMLScanner.rs and generateCompileResultForHtmlChunk.rs the same way, deletes the dead HTMLRewriterLoader class outright, and updates the workspace/build wiring (Cargo.toml, vendor patches, build scripts). Net +877/−2489 across 26 files. One intentional behavior change: element.onEndTag() no longer leaks its protected callback, with a new exact-count regression test.

Security risks

No new attack surface. The change is internal to how Bun calls lol-html; the JS-visible HTMLRewriter API is unchanged. The remaining unsafe is concentrated in one documented cell_get lifetime-erasure helper plus pre-existing JsClass/*mut Response plumbing. The PR actually removes the take-once thread-local error channel and the aliased-&mut trampoline pattern that motivated the rewrite. I see no injection, auth, or data-exposure surface here.

Level of scrutiny

High. This is production-critical runtime code on a memory-safety boundary: lifetime erasure to 'static, raw NonNull captures into boxed handler interiors, a heap-allocated HtmlRewriter reached through raw pointers to avoid aliased &mut during output-sink re-entry, and a consuming end() whose ordering with the field null-out matters for double-free correctness. The reasoning in the PR is careful and well-documented, but it is exactly the kind of reasoning a maintainer should sign off on rather than a bot.

Other factors

Three rounds of automated review (mine and CodeRabbit's) raised nine threads; all are resolved — two by code changes (the setter_utf8_arg ordering fix, the dead LSan suppression), one by deleting HTMLRewriterLoader entirely (which mooted three latent-bug findings in unreachable code), and the rest withdrawn as pre-existing/non-regressions with evidence. The full HTMLRewriter and bundler-HTML test suites pass (80 + 30 tests), and the new leak test demonstrably fails on main. The PR was undertaken at Jarred-Sumner's explicit direction in #32988, so he is the natural reviewer. Given the scope and the unsafe-Rust subtlety, I'm deferring rather than approving.

@robobun

robobun commented Jun 29, 2026

Copy link
Copy Markdown
Collaborator Author

Ready for a human pass; rolling up the state.

Review: every thread across the three rounds of automated review is resolved. The details are in the threads and in my earlier summary comment. The final automated pass found nothing new and defers to a maintainer, which is the right call for an unsafe-Rust rewrite of this size.

CI on 8a683c7a44 (build 66697): 282 jobs pass. The only failures are test/js/node/test/parallel/test-net-connect-memleak.js on the two linux-x64-musl lanes, which is #33044 ("fails on half of PR builds on linux-x64-musl since June 28 ~23:00 UTC"), filed hours before this commit existed. The identical failure also hit the previous build of this PR at a materially different commit (af8f09277b, before the HTMLRewriterLoader deletion), so it is independent of this diff, and a node:net socket GC test shares no code path with the HTMLRewriter changes here. I'm not re-triggering against a documented 50%-of-builds flake. If the two still-queued darwin-aarch64 lanes go red, that lane has hit the same artifact download timed out infra failure on every build of both of these PRs.

Verification commands and results are in the PR description. #32988 rebases on top of this once it lands.

@robobun

robobun commented Jun 29, 2026

Copy link
Copy Markdown
Collaborator Author

Build 66697 has finished. Final result on 8a683c7a44: 283 jobs pass, 3 fail, and all three are unrelated to this diff.

  • test/js/node/test/parallel/test-net-connect-memleak.js on the two linux-x64-musl lanes: CI: test-net-connect-memleak.js fails on half of PR builds on linux-x64-musl since June 28 ~23:00 UTC #33044, as covered above.

  • test/cli/install/bun-create.test.ts on one darwin-aarch64 lane. My earlier guess that this lane would hit the artifact-download infra failure was wrong; it ran, and the failing assertion is a transient GitHub API error on a live network fetch:

    expect(err).not.toContain("error:")
    Received: "[github] GET dylan-conway/create-test ... error: An internal error occurred (HTTPError)"
    

    That test hits the live GitHub API by design (it has an isGithubRateLimited escape hatch at bun-create.test.ts:133), but a generic 5xx does not match the rate-limit guard. The identical sibling darwin-aarch64 shard and every other platform's lane passed it, 12 of the 13 tests in the file passed on the same agent, and this PR touches neither that test nor any of bun create's implementation (git diff main...HEAD -- test/cli/install/bun-create.test.ts src/install/ src/runtime/cli/ is empty).

So the diff is green; the red is the tracked musl flake plus a one-off GitHub API error, neither on a code path this PR touches. Review state and verification are in the comments above.

@robobun

robobun commented Jul 20, 2026

Copy link
Copy Markdown
Collaborator Author

The RSS leak test added here went flaky on the Windows lanes (~31 MB false positive from a mimalloc segment decommit landing between the two point samples); #34786 switches it to compare peak RSS over each window.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants