Skip to content

Report stack frames at token starts and map the tokens that start a line - #47661

Closed
robobun wants to merge 4 commits into
mainfrom
robobun/4559d9fc/frame-lines-of-multi-line-expressions
Closed

robobun wants to merge 4 commits into
mainfrom
robobun/4559d9fc/frame-lines-of-multi-line-expressions

Conversation

@robobun

@robobun robobun commented Nov 1, 2026 •

Copy link
Copy Markdown
Collaborator

Related to #17153

Draft, in rework. Do not merge this diff. My review of it found regressions and asked for a split into two pull requests.

The text below describes the diff before the split.

Problem

  • With the transpiler on, a stack frame in an expression written over several lines can name another line than with // @bun. In a chain with one link per line, a TypeError points one line too high.
  • JSC reports a name, a spread, new X(args) and a call where the token ends (NodesCodegen.cpp, Parser.cpp). In printed text the next token starts there, so no source map can separate the two.

Fix

Background

  • A divot is the source offset JSC stores per instruction. Bun maps a frame with the last mapping at or before it.
  • Considered a printer-only patch: it fixes 142 of 354 probe rows and breaks 4.

Downsides

  • Frame columns change: a name frame is at the name's start, new X(args) at new. 42 test files and one doc example pinned the old columns.
  • Source maps grow 0.08% (454 of 575,367 mappings, 62 files). A call of a non-name callee costs about 38 more printer instructions.
  • Still different: delete a[b] with ] on its own line, ++ on the line above its operand.
Notes

The rule, measured. Release builds of main (3f3a9ff, WebKit 4d45755fed) and of this branch.

  • Probe of 2,627 shapes, each run with the transpiler and with // @bun: 354 rows name another line on main, 2 here (delete c[kn][kx with the ] on the next line). 0 rows that were right are wrong now. 11 of 11 reported rows and the 6 rows of the second note are equal.
  • test/aot/parallel/errors/stack-positions-of-tokens-that-start-a-line.mjs and its // @bun twin, 37 shapes: 21 name another line on main, 0 here. 3 differ in the column only (a callee in parentheses, an element that is called, an object spread on one line).
  • The program of Error message pointing to wrong line #17153, reduced: main a.ts:7:24, here a.ts:9:9, node 9:12.
  • Text that runs as written (// @bun, eval, node:vm): 123 of 2,627 probe rows change line, all onto node's line. 0 leave node's line.

Other designs. A printer-only patch maps ., ?. and [ at a line break and keeps JSC's positions. It fixes 142 of 354 probe rows, and it moves a ReferenceError that is right today one line late, because the end of a name and the . behind it are one offset. A correction on each stack read (a side byte per mapping, or one helper for every reader) makes every read pay and does not reach the profiler or the debugger. Moving the . and [ positions in JSC too moves 136 more rows of text that runs as written, 25 of them off node's line.

What the printer maps. ., ?. and [ of a member, the end of a callee that is no name, the end of the ?. of an optional call, of, the [ of an array binding, the ... of a property, the backquote of a tagged template. Each only where it stands on another line than the mapping in front of it. The position is read from the source text (FrameToken::find), so no AST node grows. Where the text in front of the anchor is not the expected token (an inlined constant, a comment), nothing is mapped.

Cost. User instructions are from perf_event_open, the minimum of 5 runs. Noise is two runs of one build.

  • Source maps, 62 files (bun build --target=bun --sourcemap=external): 575,367 to 575,821 mappings (+0.079%), 8,576,833 to 8,579,102 bytes (+0.026%). The printed JavaScript is byte for byte the same.
  • Printer, per statement over 100,000 statements of bun build (noise 3): f(a); +3, a.b; +6, a.b.c.d; +22, a[b]; 0, f(); +5, f()(); +40, (() => { f(); })(); +36, f({ ...a }); +12, for (const x of a) f(x); +12. add_source_mapping itself: 368 to 365 instructions per call (gdb stepi).
  • Cold load of the 9 MB typescript.js: 2,119,265,952 to 2,119,571,195 (+0.014%, noise 0.09%).
  • 200,000 reads of an 11-frame stack: new Error().stack -0.006% (noise 0.26%), a TypeError +0.045% (noise 0.03%), Error.prepareStackTrace with call sites -0.27% (noise 0.004%).
  • JSC parse of typescript.js as written, 5 times: -0.01% (noise 0.02%). Bytecode of typescript.js (--bytecode): 10,376,816 to 10,377,424 bytes.
  • Binary (size, release with a local WebKit): text 136,595,353 to 136,601,913 bytes (+6,560).
  • A trimmed hello world of bun build --compile --aot maps 25,961,444 to 25,965,668 bytes (+4,224, Linux x64 in CI). test/bundler/compile-aot-size.test.ts bounds it at 25,880,000, so main is over the bound too.
  • Coverage: 0 of the DA lines of 13 library files differ. A line that holds only ..., the arguments of a call or a template literal gains a DA entry (4 lines of the 37-shape program).
  • The transpiler cache version is 52, so each cached module is transpiled once more.
  • Not in the container: perf, valgrind, bloaty, strace.

The AOT compiler (bun build --compile --aot) reproduces the positions of plain Bun by hand. 159 of 428 programs of test/aot/parallel/errors printed another position before the mirror followed, 0 after. All of test/aot/parallel: 6,796 of 6,810 pass, 12 skip, 1 holds its output in a comment and is pinned again, and module-loaders/raw-behind-a-path-asks-for-the-text-of-the-file fails with erased types on main too.

WebKit. WEBKIT_VERSION is c37c1742a9: the two commits of oven-sh/WebKit#973 on 4d45755fed, the commit main builds against. WebKit's main is three commits ahead of that and changes the schema of the compiler's IR, so a build of #973 on main does not fit this tree yet.

Not part of this change. (c.boom⏎)() (the printer drops the parentheses), x.y⏎+= v, a frame of <parse>, frames that a tail call drops. A frame read by an engine that reports the end of a name (an older Bun that runs a new bundle, Safari) is one line late for a name in front of a line-broken ..


no test proof · iteration 1 · platform-specific test(s) that do not run on this machine, deferring to CI, which covers all platforms: test/js/node/vm/vm.test.ts, test/js/node/module/node-module-module.test.js, test/js/bun/util/inspect.test.js, test/js/bun/test/test-test.test.ts, test/js/bun/sourcemap/source-positions.test.ts, test/js/bun/resolve/import-meta-glob.test.ts, test/js/bun/http/serve.test.ts, test/cli/lint/js-plugins.test.ts, test/cli/hot/hot.test.ts, test/bundler/compile-aot-sourcemap.test.ts, test/bundler/bundler_bytecode_portable.test.ts, test/bundler/bun-build-compile.test.ts, test/aot/workers-from-embedded-files.test.ts, test/aot/cli/sentry-source-map.test.ts, test/aot/cli/obfuscate.test.ts, test/aot/a-getter-under-the-read-of-a-method-has-the-reader-below-it.test.ts

A stack frame of an error in an expression written over several lines named
another line with the transpiler than with `// @bun` as the first line.

JSC reported a name, a spread, `new X(args)` and a call where the token ends.
In the text the transpiler prints, that offset is where the next token starts,
so no source map can tell the two apart. oven-sh/WebKit#973 reports them where
the token starts. WEBKIT_VERSION points at a build of the pinned commit with
that change.

The printer maps the tokens that JSC reports a frame at and that had no
mapping: `.`, `?.`, `[`, the end of a callee that is no name, the end of the
`?.` of an optional call, `of`, the `[` of an array binding, the `...` of a
property and the backquote of a tagged template. It maps one only where it
stands on another line than the mapping in front of it, so text on one line
gets no new mapping. `FrameToken` reads the position of such a token from the
source text.

- `new Error()` printed as `Error()` maps its name to the `new`.
- ErrorStackFrame.cpp reports the divot for every opcode: JSC now reports
  `new` itself.
- The JUnit reporter looked a column up one too far to the right.
- The transpiler cache version is 52: cached maps lack the new mappings.
- The position mirror of the ahead-of-time compiler follows, and two parity
  programs hold the line-broken forms.

Fixes #17153
@github-actions github-actions Bot added the claude label Nov 1, 2026
@robobun

robobun commented Nov 1, 2026 •

Copy link
Copy Markdown
Collaborator Author
Updated 2:52 PM PT - Nov 1st, 2026

❌ @robobun, your commit 1edb16c has 41 failures in Build #136571 (All Failures):


🧪   To try this PR locally:

bunx bun-pr 47661

That installs a local version of the PR into your bun-47661 executable, so you can run:

bun-47661 --bun

@robobun

robobun commented Nov 1, 2026 •

Copy link
Copy Markdown
Collaborator Author

Status

How I reproduced it, on a release build of main (3f3a9ff):

  • I ran each reported file once as it is and once with // @bun as its first line, and compared the line of the frame. The two runs name different lines.
  • test/js/bun/sourcemap/source-positions.test.ts on this branch fails 49 of 331 tests with the src/ of main and passes with this branch.
  • A reduction of the program in Error message pointing to wrong line #17153 reports a.ts:7:24 on main and a.ts:9:9 here. Node reports line 9.

The JavaScriptCore half is oven-sh/WebKit#973.

My review of the diff is complete. It found regressions and asked for a split, so this pull request stays a draft:

CI, build 136571: no failure comes from this change.

  • 40 red test files fail the same way in the builds of the last five merged pull requests (for example build 136407): 39 files under test/aot and test/bundler/compile-aot-size.test.ts.
  • test/js/web/request/request-clone-leak.test.ts is red on the ASAN lane. It measures 64 MB against a bound of < 64. It measures the same 64 MB in those five builds, where it passes on a retry.
  • Every lane of this build ran, macOS x64 too.

Comment thread src/aot/lower/call.rs Resolved
Comment thread src/aot/lower/call/callee.rs Resolved
Comment thread src/aot/lower/call/callee.rs Resolved
A frame at `new X(args)` is at the `new`, and a frame at a name or at a call
of a name is where the name starts. The expectations that held the old columns
follow: snapshots with a caret, columns in stack lines, the places of the trim
report, and the hashes of the bytecode corpus, which move on every platform.
The column of a `node:vm` frame is now the one Node has: one of Node's tests
drops its branch for Bun, and two more hold Node's column in theirs.
Comment thread src/aot/lower/call/callee.rs Resolved
Comment thread src/aot/lower/expression/assignment.rs Resolved
Comment thread src/aot/lower/iteration.rs Resolved
Comment thread src/aot/lower/object/literal.rs Resolved
@robobun
robobun force-pushed the robobun/4559d9fc/frame-lines-of-multi-line-expressions branch from aa67ec1 to 62d375c Compare November 1, 2026 15:52
Comment thread src/aot/lower/pattern/array.rs Resolved
Comment thread src/ast/frame_token.rs Resolved
Comment thread src/ast/frame_token.rs Resolved
Comment thread src/js_printer/lib.rs Resolved
Comment thread src/js_printer/lib.rs Resolved
Comment thread src/jsc/RuntimeTranspilerCache.rs Resolved
Comment thread src/sourcemap/Chunk.rs Resolved
Comment thread src/sourcemap/Chunk.rs Resolved
Comment thread src/sourcemap/Chunk.rs Resolved
Each of the six positions is a `new X(args)`. The frame, and the place that
the trim report names, is at `new` now and was at the name behind it.
@robobun

robobun commented Nov 17, 2026

Copy link
Copy Markdown
Collaborator Author

Closing as stale: this PR conflicts with main, and its last commit is from 2026-11-01.

The branch stays, so the work is not lost. To pick it up again, rebase the branch on main and reopen this PR.

@robobun robobun closed this Nov 17, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants