Repository navigation
HTMLRewriter: use the lol_html Rust crate directly instead of its C API - #33048
Conversation
|
Warning Review limit reached
Next review available in: 5 minutes Enable usage-based reviews in Billing to review now. Otherwise, wait until the next included review is available. How can I continue?After more reviews become available, a review can be triggered using the To avoid repeated limits, reduce automatic review volume by pausing incremental auto-reviews earlier, using label-based review opt-in, excluding WIP or generated PR titles, or requesting reviews manually when the PR is ready. If your team needs uninterrupted high-volume reviews, an organization admin can enable usage-based reviews. How do review limits work?CodeRabbit enforces per-developer PR review limits for each organization. Most developers receive the normal plan review availability. For paid Pro and Pro+ PR reviews, CodeRabbit uses adaptive limits for sustained high-volume activity. When a developer's recent PR review activity reaches the 95th percentile or higher among CodeRabbit users, additional reviews become available more gradually as earlier reviews age out of the rolling window. Please refer docs for additional details. Review details⚙️ Run configurationConfiguration used: Path: .coderabbit.yaml Review profile: ASSERTIVE Plan: Pro Run ID: ⛔ Files ignored due to path filters (1)
📒 Files selected for processing (25)
WalkthroughRemoves the ChangesRemove bun_lolhtml_sys FFI layer; use lol_html Rust API directly
Possibly related PRs
Suggested reviewers
🚥 Pre-merge checks | ✅ 4✅ Passed checks (4 passed)
Comment |
|
Updated 3:56 AM PT - Jun 29th, 2026
❌ @robobun, your commit 8a683c7 has 2 failures in
🧪 To try this PR locally: bunx bun-pr 33048That installs a local version of the PR into your bun-33048 --bun |
|
Found 1 issue this PR may fix:
🤖 Generated with Claude Code |
88d64dd to
5a4a8b9
Compare
|
Good suggestion from the issue finder, but after checking I am not going to claim #31804, because it is not actually fixed by this PR. #31804 is about thrown handler exceptions leaking a protected root. Its mechanism is the JS-exception capture path in For what it is worth, this PR does fix a different, previously unreported leak that also falls out of the binding change: every successful |
5a4a8b9 to
af8f092
Compare
af8f092 to
81ccde3
Compare
There was a problem hiding this comment.
Actionable comments posted: 7
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Inline comments:
In `@Cargo.toml`:
- Around line 105-108: `vendor/lolhtml` is only materialized by the
ninja/bootstrap flow, so plain Cargo commands can still fail on a fresh
checkout. Add or wire up a direct bootstrap path for the `lol_html` path
dependency in the main Cargo setup so `cargo check -p ...` and rust-analyzer can
fetch/prepare `vendor/lolhtml` before resolution; use the existing
`clone-lolhtml` source in the build scripts and the `lol_html = { path =
"vendor/lolhtml" }` dependency as the key symbols to update.
In `@src/bundler/linker_context/generateCompileResultForHtmlChunk.rs`:
- Around line 97-107: The set_attribute helper in
generateCompileResultForHtmlChunk currently panics when either attribute name or
value is not UTF-8, which makes HTML rewriting crash on byte-oriented paths such
as import_record.path.pretty. Update the rewrite path in the HTML chunk
generation flow to convert these bytes into a valid string/URL-safe
representation before calling Element::set_attribute, or change set_attribute
and its callers to return a typed error that is propagated instead of panicking.
Use the existing set_attribute helper and the import_record.path.pretty call
sites as the main places to adjust.
In `@src/runtime/api/html_rewriter.rs`:
- Around line 702-710: The write_bytes path in HtmlRewriter is vulnerable to
re-entrant finalize/done freeing the active rewriter while write() is still
running. Update write_bytes so it temporarily nulls or takes self.rewriter
before calling (*self.rewriter).write(bytes), then restores the pointer on
success or destroys the taken rewriter on failure; this should prevent
LoaderSinkRef::handle_chunk -> write_to_destination -> done/finalize from
dropping the live HtmlRewriter during the write call.
- Around line 744-759: HtmlRewriter::end currently ignores the result of the
underlying rewriter shutdown and can still call done() afterward, which may
double-complete the sink. In end(), capture the return value from the unsafe
bun_core::heap::take(rewriter).end() call, and if it fails, propagate that
RewritingError through self.fail() or return it instead of proceeding. Also
ensure self.done() is only reached when the final empty-chunk path from
LoaderSinkRef::handle_chunk() has not already completed the stream, so end()
does not signal success twice.
- Around line 2295-2301: The has_attribute_() path currently throws on invalid
UTF-8 because it uses utf8_or_throw(), unlike get_attribute_() which handles
non-UTF-8 names non-throwingly. Update has_attribute_() in html_rewriter.rs to
treat an invalid ZigString attribute name as a simple false result, preserving
non-throwing hasAttribute() behavior. Keep the existing el lookup and
JSValue::FALSE fallback, but replace the UTF-8 conversion/error path so no
HTMLRewriterError is surfaced for malformed attribute names.
In `@test/js/workerd/html-rewriter-leak.test.ts`:
- Around line 1-43: This regression test is in the wrong place; move the
onEndTag callback leak coverage into the existing HTMLRewriter test module
instead of keeping a new sibling file. Add the test case to the current
HTMLRewriter suite that already covers this behavior, and keep the existing
helper logic like rewriteWithEndTagHandlers and protectedFunctions intact while
relocating it so the module’s coverage stays centralized.
In `@test/js/workerd/html-rewriter.test.js`:
- Around line 1227-1249: The detached-wrapper no-op/no-coercion test only covers
element.tagName, but the same behavior also needs to be verified for endTag.name
and comment.text. Add similar detached-wrapper assertions in
html-rewriter.test.js using the existing HTMLRewriter handler patterns so that
setters on detached endTag and comment wrappers also return without coercing
toString and leave the properties unchanged.
🪄 Autofix (Beta)
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: Path: .coderabbit.yaml
Review profile: ASSERTIVE
Plan: Pro
Run ID: 87d7aa59-f09f-44db-b99e-0e8561372333
⛔ Files ignored due to path filters (1)
Cargo.lockis excluded by!**/*.lock
📒 Files selected for processing (25)
.github/workflows/clippy.ymlCargo.tomlpatches/lolhtml/0001-rlib-only.patchpatches/lolhtml/0002-quiet-build-script-linker-warning.patchrustfmt.tomlscripts/build/bun.tsscripts/build/cargo-config.tsscripts/build/deps/lolhtml.tsscripts/build/rust.tsscripts/rust-miri.tssrc/bun_core/feature_flags.rssrc/bundler/Cargo.tomlsrc/bundler/HTMLScanner.rssrc/bundler/bundle_v2.rssrc/bundler/linker_context/generateCompileResultForHtmlChunk.rssrc/lolhtml_sys/Cargo.tomlsrc/lolhtml_sys/lib.rssrc/lolhtml_sys/lol_html.rssrc/runtime/Cargo.tomlsrc/runtime/api.rssrc/runtime/api/html_rewriter.rssrc/runtime/api/lolhtml_jsc.rstest/js/workerd/html-rewriter-leak.test.tstest/js/workerd/html-rewriter.test.jstest/leaksan.supp
💤 Files with no reviewable changes (9)
- patches/lolhtml/0001-rlib-only.patch
- src/bun_core/feature_flags.rs
- src/runtime/api/lolhtml_jsc.rs
- src/lolhtml_sys/Cargo.toml
- patches/lolhtml/0002-quiet-build-script-linker-warning.patch
- src/lolhtml_sys/lib.rs
- test/leaksan.supp
- src/lolhtml_sys/lol_html.rs
- src/runtime/api.rs
Bun's runtime is Rust, but its lol-html bindings still went through
lol-html's C API: a 1,299-line FFI layer (src/lolhtml_sys, 72
extern "C" declarations) wrapping opaque handles, plus extern "C"
output-sink and handler trampolines that manufacture &mut references
from userdata pointers, and a take-once thread-local last-error buffer.
That arrangement is a leftover from the Zig runtime, which had no other
option; from Rust it is a Rust -> extern "C" -> Rust round trip for no
benefit, and it is why HTMLRewriter code has been so unsafe-heavy.
Switch the runtime and the bundler to the lol_html Rust crate (v2.7.2,
already vendored at vendor/lolhtml and already in the cargo dependency
graph via the c-api path dep). Every existing HTMLRewriter and
bundler-HTML test passes unchanged.
What this removes:
- src/lolhtml_sys/ entirely (1,340 lines, 74 unsafe blocks, all 72
extern "C" declarations).
- Every extern "C" trampoline in the two consumers; zero remain.
- The take-once thread-local last-error channel. lol_html's methods
return typed errors (RewritingError, SelectorError, TagNameError,
AttributeNameError, CommentTextError), so error text now comes from
the error the failing call returned, not a global consumed as a side
channel.
- The two vendor patches, which only served the now-unused c-api crate.
- The orphaned DISABLE_LOLHTML feature flag (its only reader was the
deleted FFI crate).
What replaces it:
- html_rewriter.rs builds a lol_html::Settings from the selectors and
handlers accumulated on LOLHTMLContext and constructs HtmlRewriter
directly. Handler registration is safe closures capturing raw
NonNull<Handler> pointers into the Rc<RefCell<LOLHTMLContext>> that
already outlives the rewriter. The output sink is a safe
impl lol_html::OutputSink for SinkRef(*mut BufferOutputSink);
lol-html signals completion with a zero-length final chunk.
HtmlRewriter::end(self) consumes, so the sink holds the rewriter
behind a raw heap pointer read into a local before each call (the
same re-entrancy discipline the file already uses) and never calls
end after a failed write.
- The eight JS wrapper classes keep their exact shape: a Cell<*mut X>
handle valid only inside its handler, nulled before the handler
returns. The handle is now a real lol_html type behind a single,
documented lifetime-erasure helper (cell_get); every wrapper method
is a safe call into it, because the lol_html content operations are
safe Rust. The unsafe that remains in html_rewriter.rs is the
pre-existing JsClass refcount and FFI plumbing, which is independent
of which lol-html API is used.
- src/bundler/HTMLScanner.rs and generateCompileResultForHtmlChunk.rs
migrate the same way (HTMLScanner drops from 13 unsafe blocks to 3).
- lol_html = { path = "vendor/lolhtml" } becomes a direct workspace
dependency; the vendor fetch step is unchanged.
One user-visible fix falls out of the model change: element.onEndTag()
no longer leaks its callback. The C API bound end-tag handlers as a raw
(function pointer, userdata) pair, so Bun heap-allocated a per-call
handler holding a JSValue::protect() GC root on the callback and never
freed it on the success path. Every onEndTag call leaked one protected
function (and everything its closure captured) for the life of the
process. lol_html's Rust API takes an owning FnOnce box, which is
dropped, releasing the root, whether or not the end tag is ever
reached. Covered by a new exact-count regression test in
test/js/workerd/html-rewriter-leak.test.ts.
81ccde3 to
8a683c7
Compare
|
CodeRabbit's full review came back with seven findings. All seven are now closed; summary for anyone triaging. Acted on (three findings, two changes):
Declined with evidence (four): the At 8a683c7: 80 pass / 0 fail across all five HTMLRewriter test files, 30 pass / 0 fail across the bundler HTML suites. |
There was a problem hiding this comment.
All earlier review threads are resolved and the bug hunt found nothing new, but this is a large rewrite of the HTMLRewriter unsafe/lifetime-erasure layer and the live BufferOutputSink re-entrancy path — worth a human pass before merge.
Extended reasoning...
Overview
This PR replaces Bun's lol-html C-API FFI layer (src/lolhtml_sys/, ~1,300 lines, 72 extern "C" declarations) with direct use of the lol_html Rust crate. It rewrites src/runtime/api/html_rewriter.rs (handler closure construction, build_settings, cell_get lifetime erasure, SinkRef output sink, onEndTag ownership), migrates src/bundler/HTMLScanner.rs and generateCompileResultForHtmlChunk.rs the same way, deletes the dead HTMLRewriterLoader class outright, and updates the workspace/build wiring (Cargo.toml, vendor patches, build scripts). Net +877/−2489 across 26 files. One intentional behavior change: element.onEndTag() no longer leaks its protected callback, with a new exact-count regression test.
Security risks
No new attack surface. The change is internal to how Bun calls lol-html; the JS-visible HTMLRewriter API is unchanged. The remaining unsafe is concentrated in one documented cell_get lifetime-erasure helper plus pre-existing JsClass/*mut Response plumbing. The PR actually removes the take-once thread-local error channel and the aliased-&mut trampoline pattern that motivated the rewrite. I see no injection, auth, or data-exposure surface here.
Level of scrutiny
High. This is production-critical runtime code on a memory-safety boundary: lifetime erasure to 'static, raw NonNull captures into boxed handler interiors, a heap-allocated HtmlRewriter reached through raw pointers to avoid aliased &mut during output-sink re-entry, and a consuming end() whose ordering with the field null-out matters for double-free correctness. The reasoning in the PR is careful and well-documented, but it is exactly the kind of reasoning a maintainer should sign off on rather than a bot.
Other factors
Three rounds of automated review (mine and CodeRabbit's) raised nine threads; all are resolved — two by code changes (the setter_utf8_arg ordering fix, the dead LSan suppression), one by deleting HTMLRewriterLoader entirely (which mooted three latent-bug findings in unreachable code), and the rest withdrawn as pre-existing/non-regressions with evidence. The full HTMLRewriter and bundler-HTML test suites pass (80 + 30 tests), and the new leak test demonstrably fails on main. The PR was undertaken at Jarred-Sumner's explicit direction in #32988, so he is the natural reviewer. Given the scope and the unsafe-Rust subtlety, I'm deferring rather than approving.
|
Ready for a human pass; rolling up the state. Review: every thread across the three rounds of automated review is resolved. The details are in the threads and in my earlier summary comment. The final automated pass found nothing new and defers to a maintainer, which is the right call for an unsafe-Rust rewrite of this size. CI on Verification commands and results are in the PR description. #32988 rebases on top of this once it lands. |
|
Build 66697 has finished. Final result on
So the diff is green; the red is the tracked musl flake plus a one-off GitHub API error, neither on a code path this PR touches. Review state and verification are in the comments above. |
|
The RSS leak test added here went flaky on the Windows lanes (~31 MB false positive from a mimalloc segment decommit landing between the two point samples); #34786 switches it to compare peak RSS over each window. |
Precursor to #32988, at @Jarred-Sumner's direction there:
Problem
Bun's runtime is Rust, but its lol-html bindings still went through lol-html's C API: a 1,299-line FFI layer (
src/lolhtml_sys/, 72extern "C"declarations) wrapping opaque handles, plusextern "C"output-sink and content-handler trampolines that manufacture aliased&mutreferences from userdata pointers, and a take-once thread-local last-error buffer (the direct cause of two correctness findings in #32988's review). That arrangement is a leftover from the Zig runtime, which had no other option; from Rust it is a Rust ->extern "C"-> Rust round trip for no benefit, and it is the structural reason HTMLRewriter code has been sounsafe-heavy.lol_html(v2.7.2) is already vendored atvendor/lolhtmland was already in the cargo dependency graph (the c-api crate we consumed is a thin wrapper over it), so using it directly is a dependency-graph no-op.What changed
src/lolhtml_sys/is deleted entirely: 1,340 lines, 74unsafeblocks, all 72extern "C"declarations.lol_html = { path = "vendor/lolhtml" }becomes a direct workspace dependency ofbun_runtimeandbun_bundler; the vendor fetch step is unchanged.src/runtime/api/html_rewriter.rsbuilds alol_html::Settingsfrom the selectors and handlers accumulated onLOLHTMLContextand constructsHtmlRewriterdirectly. Handler registration is safe closures capturing rawNonNull<Handler>pointers into theRc<RefCell<LOLHTMLContext>>that already outlives the rewriter (the same invariant the C userdata relied on, now visible in Rust). The output sink is a safeimpl lol_html::OutputSink for SinkRef(*mut BufferOutputSink); lol-html signals completion with a zero-length final chunk.HtmlRewriter::end(self)consumes, so the sink holds the rewriter behind a raw heap pointer read into a local before each call (the same re-entrancy discipline the file already uses) and never callsendafter a failedwrite.Cell<*mut X>handle that is only valid inside its content handler and is nulled before the handler returns. The handle is now a reallol_html::html_contenttype behind one documented lifetime-erasure helper (cell_get), and every wrapper method is a safe call into it, because thelol_htmlcontent operations are safe Rust.lol_html's methods return typed errors (RewritingError,SelectorError,TagNameError,AttributeNameError,CommentTextError), so error text comes from the error the failing call returned, never a global consumed as a side channel.src/bundler/HTMLScanner.rsandsrc/bundler/linker_context/generateCompileResultForHtmlChunk.rsmigrate the same way.patches/lolhtml/*patches only served the now-unused c-api crate and are removed, as is the orphanedDISABLE_LOLHTMLfeature flag (its only reader was the deleted FFI crate).HTMLRewriterLoader(239 lines) is deleted outright rather than migrated. It is apubZig-port carryover with zero callers: nothing constructs one, it is absent from the JSSink codegen registry (src/codegen/generate-jssink.ts), and no.rs/.ts/.cpp/.h/.zigfile references it outside its own definition. Beingpubis what kept it off thedead_codelint. Review (claude, then CodeRabbit) found three distinct latent memory-safety bugs in it, none reachable; deleting the class fixes all three at the root instead of hardening code that cannot run.Net: 26 files, +877 / -2489.
One behavioral fix:
onEndTag()no longer leaks its callbackThis is the only user-visible behavior change, and it falls out of the model change rather than being a goal.
The C API binds an element's end-tag handler as a raw
(function pointer, userdata)pair, so Bun heap-allocated a per-call handler holding aJSValue::protect()GC root on the callback and handed the raw pointer to lol-html. Nothing ever freed it on the success path (the only free was the registration-error branch), so everyelement.onEndTag(fn)call permanently leaked one protected JS function, plus everything its closure captured, for the life of the process. A long-runningBun.serve+HTMLRewriter+onEndTaggrows without bound.lol_html's Rust API instead takes an owningBox<dyn FnOnce(..)>. The closure owns the protection; lol-html owns the closure and drops it (releasing the root) whether or not the end tag is ever reached. The leak is structurally impossible.Covered by an exact-count regression test in
test/js/workerd/html-rewriter-leak.test.tsusingbun:jsc'sheapStats().protectedObjectTypeCounts(no RSS threshold, stable on debug builds): two passes of 400onEndTagregistrations after a baseline leave 800 extra protected functions onmainand 0 with this change.Every other HTMLRewriter and bundler-HTML test passes. Besides the new leak file, the only test change is four added cases in
html-rewriter.test.jscovering theelement.tagName,endTag.name, andcomment.textsetters, which had no coverage anywhere in the tree; no existing test is modified.Verification
cargo check -p bun_runtime -p bun_bundler,cargo clippy -p bun_runtime -p bun_bundler, andcargo fmt --all --check: clean.bun bd test test/js/workerd/html-rewriter.test.js test/js/workerd/html-rewriter-end-error.test.ts test/js/workerd/html-rewriter-leak.test.ts test/js/web/html/html-rewriter-doctype.test.ts test/regression/issue/htmlrewriter-additional-bugs.test.tsmain(fail-before):expect(after - before).toBe(0)receives 800.bun bd test test/bundler/bundler_html.test.ts test/bundler/bundler_html_server.test.ts test/bundler/html-import-manifest.test.tsgrep -rn bun_lolhtml_sys --include='*.rs' --include='*.toml' .outsidevendor/: nothing.grep 'extern "C"' src/runtime/api/html_rewriter.rs src/bundler/HTMLScanner.rs: nothing.Honest unsafe accounting
The
unsafethat was about the lol-html boundary is gone: the 74-block FFI layer, every trampoline, every opaque-handle call.src/bundler/HTMLScanner.rsdrops from 13unsafeblocks to 3.html_rewriter.rsitself still containsunsafe, and it is worth being precise about what that is, because almost none of it is about lol-html anymore:cell_get) at the handler-to-JS-wrapper boundary. A JS object cannot carry a Rust borrow lifetime, so a handler's&mut Element<'r, 't>has to be parked as a raw pointer for the duration of the JS callback and nulled before the handler returns; that is inherent to exposing lol-html handles to JS at all, and it is now one place instead of being baked into 72 FFI signatures. Every call site obeys its ordering rule: a setter given a rawJSValueruns the value's ToString (arbitrary user JS, which can re-entercell_geton the same wrapper) before the borrow is taken, through one sharedsetter_utf8_arghelper that carries the rule.JsClassrefcount and*mut Responseplumbing, which is independent of which lol-html API is used and is unchanged by this PR.The practical payoff lands in #32988: its streaming change was fighting the C boundary (four review rounds found five real bugs, two of them caused directly by the take-once error buffer and the opaque-handle poisoning model). Rebased on this, that diff sheds the classes of unsafe that made it hard to get right.
Relationship to #32988
#32988 (make HTMLRewriter actually stream) stays open and will be rebased on top of this once it lands. The two are deliberately separate so each is reviewable on its own: this one is the binding migration, and that one is the streaming behavior change.