Skip to content

Fix quadratic hang reporting duplicate-binding parse errors in the transpiler - #31341

Merged
Jarred-Sumner merged 3 commits into
mainfrom
farm/167f0eda/js-parser-error-position-quadratic
May 25, 2026
Merged

Jarred-Sumner merged 3 commits into
mainfrom
farm/167f0eda/js-parser-error-position-quadratic

Conversation

@robobun

@robobun robobun commented May 24, 2026

Copy link
Copy Markdown
Collaborator

Problem

Fuzzing found a hang in the JS parser's error-position tracking: a ~9 KB input full of parse errors keeps Bun.Transpiler.transformSync busy for 20+ seconds, sampled spinning in ErrorPositionState::advance ← LineColumnTracker::error_position — the same line/column computation that previously hung Bun.JSONC.parse, now reached through the transpiler.

The input shape is lines of

try {} catch ([a,a,a,/* ~200 more */,a, `]) {}

where every a after the first is a duplicate catch binding and the backtick starts a template literal that swallows the next line.

Synthetic reproduction (debug build, before this change):

const stmt = "try {} catch ([" + "a,".repeat(210) + "a, `]) {}\n";
new Bun.Transpiler({ loader: "js" }).transformSync(stmt.repeat(168)); // 75 KB → 15.5 s

9 KB → 0.35 s · 19 KB → 1.1 s · 37 KB → 4.0 s · 75 KB → 15.5 s — doubling the input quadruples the time.

Cause

Log computes diagnostic line/columns through a single resumable scan (LineColumnTracker); any diagnostic whose offset is behind the previous one falls back to a full rescan of the source from byte 0.

For this input that fallback is the common case, not the exception:

  1. The parser lexes the whole catch ([...]) binding first. The template literal produces an "Expected identifier" error at the end of the statement, advancing the tracker there.
  2. declare_binding then declares the ~200 as and logs one "has already been declared" error per duplicate — plus a note pointing at the first a. Both locations are behind the tracker, so every duplicate pays two O(offset) rescans ⇒ O(duplicates × file size).

Even without the template literal, the notes alone (always pointing back at the original declaration) make the pattern quadratic.

Fix

Generalize the tracker's single resumable scan into a small pool of four cursors, kept sorted by offset. A diagnostic resumes the furthest-advanced cursor at or below its offset, so the interleaved diagnostic streams the parsers actually produce (lexer high-water mark, duplicate-declaration errors, the notes attached to them) each stay incremental; each cursor only ever moves forward. Offsets below every cursor and offsets inside a multi-byte codepoint keep the existing one-off full-scan fallback, so results remain bit-identical to Source::init_error_position.

No caps or truncation of diagnostics anywhere; diagnostic output is unchanged.

Measurements (debug + ASAN build)

input before after
original 9 KB fuzz repro 0.39 s 0.14 s
75 KB duplicate-catch flood 15.5 s 0.54 s
356 KB duplicate-catch flood > 5 min (extrapolated ~6 min) 2.6 s

Scaling is now linear (2× input ⇒ ~2× time).

Tests

  • src/ast/lib.rs: new differential unit test driving the tracker with the parser's interleaved ordering (lexer error at end of statement, then duplicate/note pairs) against from-scratch init_error_position; existing forward/backwards/strided differential tests still pass.
  • test/bundler/transpiler/transpiler.test.js: new "parse error flood" test spawns a child with a 60 s kill switch (same pattern as the Bun.JSONC.parse pathological test) that transpiles a 356 KB duplicate-catch-binding flood and a plain duplicate-binding flood, expecting the usual AggregateError. Fails without the fix (child killed at 60 s), passes in ~6 s with it.
  • bun bd test test/bundler/transpiler/transpiler.test.js (169 tests), test/js/bun/jsonc/jsonc.test.ts, test/bundler/transpiler/runtime-transpiler.test.ts, test/js/bun/resolve/resolve-error.test.ts, test/js/bun/resolve/build-error.test.ts, test/js/bun/resolve/toml/crash/toml-crash.test.ts — all green locally.

@coderabbitai

coderabbitai Bot commented May 24, 2026 •

Copy link
Copy Markdown
Contributor

Warning

Review limit reached

@robobun, we couldn't start this review because you've used your available PR reviews for now.

Your plan includes 5 reviews of capacity. Refill in 40 minutes and 59 seconds.

Your organization has run out of usage credits. Purchase more in the billing tab.

⌛ How to resolve this issue?

After more review capacity refills, a review can be triggered using the @coderabbitai review command as a PR comment. Alternatively, push new commits to this PR.

We recommend that you space out your commits to avoid hitting the rate limit.

🚦 How do rate limits work?

CodeRabbit enforces hourly rate limits for each developer per organization.

Our paid plans have higher rate limits than trial, open-source, and free plans. In all cases, review capacity refills continuously over time.

Please see our FAQ for further information.

ℹ️ Review info
⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: ASSERTIVE

Plan: Pro

Run ID: 6bdcc746-cc39-4cda-b2ad-95a991e83b5e

📥 Commits

Reviewing files that changed from the base of the PR and between 2fbfcb9 and 2b2e10c.

📒 Files selected for processing (2)
  • src/ast/lib.rs
  • test/bundler/transpiler/transpiler.test.js

Comment @coderabbitai help to get the list of available commands and usage tips.

@robobun

robobun commented May 24, 2026 •

Copy link
Copy Markdown
Collaborator Author
Updated 11:02 PM PT - May 24th, 2026

✅ @robobun, your commit 2b2e10cf3a452d0d3eb73a950c95bf5904cd69af passed in Build #57785! 🎉


🧪   To try this PR locally:

bunx bun-pr 31341

That installs a local version of the PR into your bun-31341 executable, so you can run:

bun-31341 --bun

@robobun

robobun commented May 24, 2026 •

Copy link
Copy Markdown
Collaborator Author

Status: reproduced the fuzz hang locally — the duplicate-catch-binding input makes every "has already been declared" error (and its note) recompute its line/column with a full rescan from byte 0, so error reporting is quadratic in input size (75 KB → 15.5 s in a debug build). With the cursor-pool change the same input takes 0.54 s and scaling is linear; diagnostics are unchanged. The transpiler.test.js "parse error flood" test fails (child killed at its 60 s kill switch) without the src change and passes in ~6 s with it; jsonc, runtime-transpiler, resolve-error, build-error, and toml-crash suites are green locally.

Review feedback addressed: code comments removed; rebased onto main after the transpiler.test.js append conflict with #31347's new describe block.

CI (build 57737): the three red lanes are unrelated to this diff — debian-13-x64-asan failed on a test.failing inline-snapshot flake in test/js/node/net/double-connect.test.ts, alpine-3.23-x64-baseline failed on the RSS assertion in test/js/web/streams/streams-leak.test.ts, and the two darwin test lanes expired waiting for agents. Neither red lane ran the new transpiler test; every lane that did run it passed. Pushed one empty retrigger commit to re-roll.

@claude claude Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I didn't find any issues — the sorted-descending cursor invariant checks out and the differential tests are convincing — but this reworks a data structure in shared parser diagnostic infrastructure, so it's worth a quick human pass on the invariant reasoning.

Extended reasoning...

Overview

This PR fixes a quadratic-time hang in LineColumnTracker::error_position (src/ast/lib.rs) by generalizing the single resumable scan cursor into a fixed pool of four ScanCursors kept sorted by offset descending. A diagnostic resumes the furthest-advanced cursor at or below its offset, so the interleaved diagnostic streams the JS parser produces (lexer high-water mark → duplicate-binding errors → notes pointing back at the original declaration) each stay incremental instead of triggering a full O(file size) rescan per backwards offset. It also adds a differential unit test for the interleaved pattern and an integration test in test/bundler/transpiler/transpiler.test.js that floods the transpiler with ~356 KB of duplicate catch bindings under a 60 s kill switch.

Security risks

None. This is pure line/column bookkeeping for diagnostic messages — no auth, crypto, filesystem, network, or user-controlled-path handling. Worst-case failure mode is an incorrect line/column in an error message, and the differential tests against Source::init_error_position guard against even that.

Level of scrutiny

Moderate. The logic change is small (~40 LOC) and the output is asserted bit-identical to the from-scratch scan by both pre-existing (forward/backward/strided) and new (interleaved) differential tests, with the old full-scan fallback preserved for offsets below every cursor and mid-codepoint offsets. I walked through the "sorted descending" invariant: picking the first cursor with offset <= target and advancing it to target keeps it strictly below the cursors before it (which all had offset > target) and at-or-above the cursors after it (which were ≤ its old offset ≤ target), so order is preserved; reset_for re-initializes all four cursors to offset 0 via Default. ErrorPositionState is already Copy, so the new #[derive(Copy)] on ScanCursor is sound.

Other factors

No CODEOWNERS cover these paths. The bug-hunting system found nothing. The change is well-contained and exceptionally well-documented, but it is a non-mechanical data-structure refactor in diagnostic infrastructure shared across the JS/JSONC/TOML parsers, with an invariant that is correct but not obvious at a glance — that pushes it just past my bar for auto-approval.

@Jarred-Sumner Jarred-Sumner left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Delete all the code comments

@robobun

robobun commented May 24, 2026

Copy link
Copy Markdown
Collaborator Author

Done — removed the comments (e3b90e6).

@robobun
robobun requested a review from Jarred-Sumner May 24, 2026 20:53
Comment thread src/ast/lib.rs Resolved
robobun added 2 commits May 24, 2026 22:06
…iagnostics stay incremental

A parse that reports many diagnostics used to fall back to a full rescan
from byte 0 whenever a diagnostic's offset was behind the previous one.
The JS parser's duplicate-binding errors and their notes always point
behind the lexer's position, so error reporting on inputs full of
duplicate catch bindings was quadratic in input size. Generalize
LineColumnTracker's single resumable scan into a small pool of cursors
(resume the nearest one at or below the queried offset); results stay
identical to Source::init_error_position.
@robobun
robobun force-pushed the farm/167f0eda/js-parser-error-position-quadratic branch from e3b90e6 to f4d8ff9 Compare May 24, 2026 22:08

@claude claude Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

No bugs found. The 4-cursor pool relies on an implicit invariant (picking the first offset <= target in a descending array and advancing it preserves descending order, so no explicit sort is needed) — it checks out and the differential test covers it, but it's subtle enough on core diagnostic infrastructure that a human sign-off is worth having.

Extended reasoning...

Overview

This PR generalizes LineColumnTracker in src/ast/lib.rs from a single resumable scan cursor to a fixed pool of four ScanCursors, so that interleaved diagnostic streams (lexer high-water mark, duplicate-declaration errors, and their back-pointing notes) each stay incremental instead of falling back to O(file-size) rescans. It adds a differential unit test that replays the parser's interleaved offset ordering against Source::init_error_position, plus an integration test in transpiler.test.js that floods the transpiler with ~356 KB of duplicate catch bindings under a 60 s kill switch.

Correctness of the algorithm

I traced the invariant: the cursor array starts at [0,0,0,0]; .iter().position(|c| c.offset <= target) selects the first (i.e. largest, given descending order) cursor at or below target; advancing that cursor to target keeps the array sorted descending because everything before it is > target and everything after it was <= the old value <= target. So the "kept sorted by offset" claim in the PR description holds without an explicit sort step. The mid-codepoint and below-all-cursors cases still fall through to init_error_position, so results are bit-identical to the reference. The per-cursor line_end cache is invalidated on the same condition as before (whenever advance crosses a line break).

Security risks

None. This is error-path-only line/column computation for diagnostics; it does not touch parsing semantics, codegen, auth, or I/O. Worst-case bug impact would be wrong line/column numbers in error messages.

Level of scrutiny

Moderate-to-high. The change is small (~30 LoC of logic) but sits in core parser/diagnostic infrastructure used by every transpile that reports an error, and the self-maintaining sort invariant is non-obvious from the code alone (the PR description says "kept sorted by offset" but there is no sort call). A human should confirm they're comfortable with the design choice of a fixed 4-slot pool vs. alternatives, and with relying on the implicit invariant.

Other factors

  • My earlier doc-comment nit was resolved (author declined per prior review feedback to drop comment changes; thread marked resolved).
  • The two CI failures (streams-leak.test.ts, test-worker-message-port-transfer-terminate.js) are unrelated to the touched files.
  • Test coverage is good: the new differential unit test plus three pre-existing tracker tests pin output equality to init_error_position, and the integration test would catch a perf regression.

@Jarred-Sumner
Jarred-Sumner merged commit 1950a3a into main May 25, 2026
76 of 77 checks passed
@Jarred-Sumner
Jarred-Sumner deleted the farm/167f0eda/js-parser-error-position-quadratic branch May 25, 2026 02:01
springmin pushed a commit to springmin/bun that referenced this pull request May 25, 2026
* oven/main (20 new commits):
  webcore: free Blob's owned content type on drop (oven-sh#31358)
  Support cross-compiling macOS binaries from Linux (oven-sh#31303)
  test: forward keep-alive requests in proxy.test.ts's mock proxy (oven-sh#31352)
  Port Bun.stringWidth to C++ with explicit SIMD (oven-sh#31351)
  Fix quadratic hang reporting duplicate-binding parse errors in the transpiler (oven-sh#31341)
  shell: don't abort when a glob's directory prefix doesn't exist (oven-sh#31367)
  Error instead of crashing on deeply nested statements in the transpiler (oven-sh#31333)
  Fix JSX transform panic when a bare `key` prop precedes `key` with a value (oven-sh#31350)
  Cap ANSI markdown indentation so deeply nested lists render in linear time (oven-sh#31366)
  css: bound selector-list expansion when compiling nesting for older targets (oven-sh#31277)
  node:http2: reassemble HEADERS+CONTINUATION before HPACK decoding (oven-sh#31323)
  Fix `await using` expression printing `using` as `await` (oven-sh#31324)
  Parenthesize `async` when it starts a for-of loop initializer (oven-sh#31326)
  Print Infinity and negative numeric property keys as computed properties (oven-sh#31328)
  css: keep required grouping parens in @container conditions when minifying (oven-sh#31330)
  Fix panic on anonymous export default class with an auto-accessor field (oven-sh#31331)
  node:http2: send GOAWAY frames on stream 0 (oven-sh#31353)
  parser: fix Scope mismatch while visiting panic from decorators on dropped class members (oven-sh#31340)
  webcrypto: reject oversized BufferSource inputs instead of aborting (oven-sh#31356)
  Error instead of crashing on deeply nested TypeScript types in the transpiler (oven-sh#31361)

Resolved conflicts:
  - scripts/build.ts: kept both OHOS and macOS-cross argv entries
  - scripts/build/config.ts: kept both OHOS and macOS-cross config fields
  - scripts/build/deps/webkit.ts: kept OHOS fno-pic exclusion, adopted upstream -flto=thin
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants