Skip to content

bundler: run modules that reach a top-level await in the order of ECMA-262 - #47417

Draft
dylan-conway wants to merge 8 commits into
mainfrom
claude/bun-esm-top-level-await-qdpjen
Draft

dylan-conway wants to merge 8 commits into
mainfrom
claude/bun-esm-top-level-await-qdpjen

Conversation

@dylan-conway

@dylan-conway dylan-conway commented Oct 29, 2026 •

Copy link
Copy Markdown
Member

What does this PR do?

Draft, to try the approach. It makes the modules of a bundle that have or reach a top-level await run in the order, and in the promise jobs, that ECMA-262 defines. It covers --format=esm without --splitting. Everything else keeps today's output.

The problem. A bundle prints its modules one after the other, so an await at the top level of one module holds back every module printed after it, related or not. Siblings that overlap when the files run unbundled run in sequence. A program in which a module awaits something that a later sibling provides hangs. Import cycles, errors and import() differ as well. esbuild, Rollup and Rolldown behave the same way (esbuild#253).

The idea. InnerModuleEvaluation, the walk that decides which modules are asynchronous, how they are numbered, who waits for whom and what the cycle roots are, finishes before any promise job runs. For the first evaluation of a graph its result depends on the graph alone. So the linker does the walk and prints the result. Only AsyncModuleExecutionFulfilled / Rejected are left for run time.

  • A module that reaches no top-level await prints as before: flat, scope-hoisted.
  • A module that has or reaches one prints in place as var init_x = __tlaNode(body, hasTLA, [what it waits for]).
  • Only the statements that somebody could see run are in that wrapper. The others print ahead of it, as before: declarations (functions, plain classes, literals, re-exports), and statements that have no side effects and read, where they stand, only bindings that have their final value by then (class A extends B {}, const table = { a, b }, const alias = f). If that leaves nothing, the module has no wrapper, and its importers wait for what it would have waited for. The linker already had what this takes: the parser's can_be_removed_if_unused, LoadEffect, has_been_assigned_to and the resolved imports.
  • A module that an import() inside the bundle reaches can be evaluated later, from any state, so it takes the walk at run time (__tlaModule).
  • The entry point stays unwrapped where it can.
  • If only the entry point has an await, the output has none of this.
// entry.js: import { left } from "./left.js"; import { plain } from "./plain.js"; console.log("entry", left, plain);
// left.js:  import { t } from "./t.js"; export const left = "left-" + t; console.log(left);
// t.js:     console.log("t1"); export const t = await "T"; console.log("t2");
// plain.js: export const plain = "plain"; console.log(plain);

// t.js
var t;
var init_t = __tlaNode(async () => {
  console.log("t1");
  t = await "T";
  console.log("t2");
  __tlaDone(init_t);
}, 1, []);

// left.js
var left;
var init_left = __tlaNode(() => {
  left = "left-" + t;
  console.log(left);
}, 0, [init_t]);

// plain.js
var plain = "plain";
console.log(plain);

// entry.js
await __tlaMain([init_left]);
__tlaResume();
console.log("entry", left, plain);

Unbundled and with this PR: t1 plain t2 left-T entry. Bun 1.4.4: t1 t2 left-T plain entry.

Where to read.

  • src/bundler/linker_context/top-level-await.md: what the specification asks for, what other bundlers do, the design, what no single-file bundle can reproduce, the measurements, the cost, the known gaps, the open decisions.
  • src/bundler/linker_context/tlaPlan.rs: mark_tla_nodes picks the shape of each file (TlaShape) before the wrappers are made, RunsUnnoticed finds the statements that need none. compute_tla_plan works out what each node waits for once the order of the files in the chunk is known.
  • src/runtime.tla.js: the run-time helpers, about 0.9 KB minified (1.2 KB with import()). Only the bundler's runtime module has them, bun:wrap does not.
  • test/bundler/tla-fuzz/: the fuzzer, a transcription of the specification's algorithms, and a prototype linker in JavaScript with the parts that are not in Rust yet.

Other changes in behavior.

  • An await that dead code elimination removes (if (process.env.NODE_ENV === "development") await … in a production build) no longer makes a bundled module asynchronous. The parser reports a top-level await to the bundler only if the visit pass met one outside of functions and of dead code.
  • A lowered top-level using left the exports of a wrapped file as vars inside the wrapper function, so importers threw a ReferenceError. __esm in 1.4.4 has the same bug. More files get a wrapper now, so it is fixed here.
  • export * from "external" in a wrapped entry point was printed inside the wrapper function.
  • In a module that reaches a top-level await, var f = function () {} is undefined for whoever can look at it ahead of that statement (another file of an import cycle, an earlier statement of the file), as it is unbundled. __esm hoists such a value out of the wrapper. A const, which would throw unbundled, is hoisted as before.

What it costs. One function per module that reaches a top-level await and runs code that somebody could see: about 36 bytes minified and 15–18 µs each, measured on 2,000 such modules, linear from 1,000 to 4,000.

All of rxjs 7.8.1 in one bundle, with an await added to a file that 190 of its 228 files reach:

wrappers bytes minified
Bun 1.4.4 (flat) 0 167,225 64,839
a wrapper for every file that reaches the await 189 192,858
none for files that only declare 34 175,462 67,043
none for files that nobody sees run, short lists 21 173,020 66,529
statement by statement (this PR) 20 172,235 66,385

30 statements are left in those 20 wrappers. About 900 of the 1,546 bytes that are left are the helpers.

Known gaps (details in the document):

  • --splitting is unchanged. The prototype has a design for it.
  • entry → dependency with await pays for a wrapper and the helpers, although flat output plus one await 0 would do.
  • Files are marked before tree shaking. If a "sideEffects": false file with an await is dropped, it neither prints nor runs, but its importers keep wrappers that they no longer need.
  • The shape of a file is decided once per build, so with several entry points one entry point's needs can leak into another's chunk as dead weight.
  • import() of a wrapped module that reaches no await still settles a promise job early, synchronous __esm wrappers in a cycle do not mark members that already ran as failed, and __esm hoists the value of var f = function () {}, which another file of a cycle and the statements ahead of it can see. All three predate this PR.
  • __esm(async …) and __promiseAll now look reachable only when a file bundled as CommonJS reaches an await, for which the output was already invalid JavaScript (#46749). They are still here, without a test.
  • What no single-file bundle can do: an importer of the bundle sees it settle one promise job late if the entry point has no await of its own but a dependency has, a throw during the walk is reported one job late, and external imports are hoisted out of their place.

How did you verify your code works?

  • test/bundler/bundler_tla.test.ts: 88 tests. Most programs are bundled as they are, and again behind an import() so that every module takes the run-time walk. The expectations are what Node prints for the unbundled files. 67 of the 88 fail with Bun 1.4.4.

  • The fuzzer compares random module graphs (cycles, throws, timers, modules that wait for each other's side effects, import()) promise job by promise job against Node, Bun and the transcription of the specification. 1,000 graphs per profile, same order / same promise jobs, in percent:

    dag cyc err err2 time sig
    Bun 1.4.4, esbuild, Rollup, Rolldown 49.9 / 22.8 52.2 / 23.7 56.0 / 35.0 49.9 / 34.2 51.2 / 36.8 52.5 / 41.2, 94 hangs
    webpack, Rspack 98.6 / 97.4 98.8 / 96.7 91.1 / 89.6 88.3 / 87.2 95.9 / 94.9 95.6 / 94.5, 23 hangs
    this PR 100 / 100 100 / 100 100 / 100 100 / 100 100 / 100 100 / 100

    Two more profiles, 2,000 graphs each. In inert almost half of the modules only declare things. In pure modules also make objects and classes out of what they import, and their importers log what those hold, which shows a statement that ran too early. Bun 1.4.4: 75.3 / 49.8 and 72.3 / 46.5. This PR: 100 / 100 on both.

    The graphs are far denser in top-level await than real code. The numbers say how much of the semantics a tool implements, not how many real programs break.

  • import() graphs: 98.8 % of 754 (Bun 1.4.4: 58.0 %), and 99.0 % of 835 with the statements of pure (Bun 1.4.4: 64.4 %). All misses but one come from the three __esm gaps above. In that one, the rule that mirrors Bun's loader (what an import() loads does not wait for the module that contains the import()) applies in the bundle and not in the hosts, where import() takes long enough for that module to finish first.

  • test262 language/module-code/top-level-await: 30 of 31 pass when bundled (1.4.4: 29). The remaining one expects a ReferenceError from a temporal dead zone, which scope hoisting loses.

  • bun build from the CLI, Bun.build with minify and sourcemap, --compile and --compile --bytecode --format=esm all print what Node prints for the unbundled files, for the example in the document and for a small app with a class hierarchy, a registry object, a barrel, an import cycle and a logger that queues promise jobs.

  • The rest of test/bundler on a debug build: 105 of 108 files pass. The other three fail on packages that my machine could not install (aws-cdk-lib, a React canary, undici). None of the failing tests bundles a top-level await. cargo clippy on the three changed crates is clean.

Existing tests that changed:

  • default/TopLevelAwaitAllowedImportWithoutSplitting expected 0 1. Node and Bun print 1 0 for the unbundled files, and so does the bundle now.
  • Two tests in bundler_promiseall_deadcode.test.ts and regression/issue/cyclic-imports-async-bundler.test.js pinned __esm(async …) and __promiseAll in the output. They pin the records now. What the programs print is unchanged.
  • dynamic_import_dce/ElideTopLevelAwaitInImportee and edgecase/UsingWithHoistableInitializerKeepsWrapper matched the text of the import() lowering.

Generated by Claude Code

@robobun

robobun commented Oct 29, 2026 •

Copy link
Copy Markdown
Collaborator
Updated 6:50 PM PT - Nov 5th, 2026

❌ @claude, your commit 0a3abbe has 38 failures in Build #137865 (All Failures):


🧪   To try this PR locally:

bunx bun-pr 47417

That installs a local version of the PR into your bun-47417 executable, so you can run:

bun-47417 --bun

Generates random ES module graphs with top-level await (cycles, throws,
sleeps, modules that wait for each other's side effects, import()) and
compares, promise job by promise job, how they evaluate

- unbundled under Node and Bun,
- in a transcription of the algorithms of ECMA-262 (spec-model.mjs),
- bundled by `bun build` and, if they are installed, by esbuild, Rollup,
  Rolldown, webpack and Rspack.

plan-linker.mjs is a prototype linker in JavaScript for trying out ways to
print such a graph into one file. test262.mjs runs the top-level await
tests of a test262 checkout unbundled and bundled.
…A-262

A bundle prints its modules one after the other, so an `await` at the top
level of one module held back every module printed after it, related or
not. Siblings that overlap when the files run unbundled ran in sequence, a
program in which a module awaits something that a later sibling provides
hung, and import cycles, errors and import() differed as well.

InnerModuleEvaluation, the walk that decides which modules are
asynchronous, how they are numbered, who waits for whom and what the cycle
roots are, finishes before any promise job runs. For the first evaluation
of a graph its result only depends on the graph, so the linker does the
walk. For ESM output without code splitting:

- A module that reaches no top-level await is printed as before.
- A module that has or reaches one is printed in place as
  `var init_x = __tlaNode(body, hasTLA, [what it waits for])`.
- A module that an import() in the bundle reaches can be evaluated later,
  from any state, so it takes the walk at run time (`__tlaModule`).
- The entry point stays unwrapped where it can.
- runtime.tla.js has AsyncModuleExecutionFulfilled / Rejected and the rest
  of what is left for run time.

An `await` that dead code elimination removes no longer makes a bundled
module asynchronous.

Also fixes two bugs of wrapped ES modules that more files would have run
into: the exports of a file with a lowered top-level `using` stayed inside
the wrapper function, and `export * from "external"` of a wrapped entry
point was printed inside of it.
What ECMA-262 asks for, what goes wrong when modules are concatenated, what
other bundlers do and how they measure up, the design behind tlaPlan.rs,
what no single-file bundle can reproduce, the measurements, the cost, the
known gaps and the open decisions.
… top-level await

A module that only declares things (functions, classes without `extends`,
literals, re-exports) printed as `__tlaNode(() => {}, 0, [...])` once its
declarations were hoisted. Nobody can tell when such a module runs, so it
now prints as it always did, and its importers wait for what it would have
waited for. A module in an import cycle keeps its node, because the file
that the walk leaves last stands for the whole cycle.

In a bundle of rxjs with an `await` in a file that 190 of the 228 files
reach, this leaves 34 wrappers of 189.

The three flags for the shape of a file become one enum, `TlaShape`.
A module that reaches a top-level await needs a wrapper so that its code
runs at the right moment. It needs none if nobody can tell when it runs.
So far that was a module that only declares things. Now it is also one
whose other statements have no side effects and read, where they stand,
only bindings that have their final value by then: `class A extends B {}`,
`const table = { a, b }`, `const alias = f`. What a function or a method
reads does not count. The marking moves behind the step that matches imports
with exports, so that it can follow an import to the file that declares it.

"Can be removed" is not "can be moved": `extends` throws unless the base is
a constructor and reads its `prototype`, so the base has to be declared as a
class, and calls, `new`, property reads, spreads, computed keys and globals
are not accepted.

Also:
- What a node waits for is printed without what another entry of the list
  waits for already.
- A module that is not wrapped hands what it would have waited for to its
  importers. If that is more than two names and more than one list would get
  a copy, it prints `__tlaNode(() => {}, 0, [...])` behind its statements
  instead, which keeps the output linear in the number of imports.
- A node or a record in an import cycle keeps the values of its `var`s
  inside the wrapper. The other files of the cycle can look at them before it
  has run.
- What the imports of an unwrapped module turn into keeps its source order.
- More forms of an `await` count as certain to be reached, which is what
  lets the entry point stay unwrapped.

In the bundle of rxjs with an `await` that 190 of its 228 files reach, 21
wrappers are left of 189, and the minified bundle is 1,690 bytes larger than
a flat one, 900 of them the run-time.

The fuzzer has two profiles for this: modules make objects and classes out of
what they import, and their importers log what those hold.
Whether nobody can tell when code runs was decided for whole files. One
statement that has to wait (`Model.table = db.name`) kept every statement of
its file in the wrapper, and with them `class Sub extends Model {}` in other
files.

The answer is now kept per top-level statement. In a file that keeps its
wrapper, the statements that nobody sees run print ahead of it, in source
order, and count as having their final value for the files that import them.

A `var` with a value keeps its place behind a statement that stays: that
statement can look at the variable and assign to it. For the same reason a
record no longer hoists the value of a `var`.

In the bundle of rxjs with an `await` that 190 of its 228 files reach, 30
statements in 20 wrappers are left.

The modules of the fuzzer's profile `pure` now also log what they make
themselves.
@dylan-conway
dylan-conway force-pushed the claude/bun-esm-top-level-await-qdpjen branch from c2f2fcc to d4d3975 Compare October 30, 2026 02:09
The change of #47329, so that this branch's CI does not fail on it:
`build-scaling-2.test.ts` has been over its bound of 2.5 on macOS arm64 in
most runs since #47281, on main and on unrelated branches. Two of the three
builds of this branch measured 2.66 and 2.57. With 2.58 on record the bound
is 2.84.

Copy link
Copy Markdown
Member Author

CI on d4d3975 (#135595) has two failures. Neither comes from this PR.

test/aot/build-scaling/build-scaling-2.test.ts, macOS aarch64: "twice as many functions cost 2.57 times the peakBytes; allowed: 2.5".

test/aot/workers-within-address-space-limit.test.ts, Alpine x64: the compiled program aborts with "memory allocation of 8 bytes failed" under its 1 GB limit on the address space, while the main thread exits with 25 Workers spawned.

  • The program is a single file whose only await is in the entry point. For such a file this PR prints the same bundle as before: byte for byte what Bun 1.4.4 prints, with none of the new helpers.
  • The test passed on the two earlier pushes of this PR.
  • I found no fix for it to port. The push of 141649c runs it again.

The four macOS x64 test jobs have not found an agent in any of the three builds.


Generated by Claude Code

Conflicts:
- src/bundler/ParseTask.rs: main's signature of
  `get_runtime_source_comptime`, this branch's comment.
- test/aot/build-scaling: main's files. It has new numbers for the shape whose
  growth in memory this branch had put on record, so that change is gone.

`Binding::to_expr` has a new signature on main. The call that hoists the
`var`s of a lowered `using` out of a wrapper follows it.

dylan-conway commented Nov 6, 2026 •

Copy link
Copy Markdown
Member Author

I merged main (b1305dd) into this branch to resolve a conflict. CI on the merge, 0a3abbe (#137865), is failing under test/aot. The failures come in with main, not from this PR.

So far three files fail:

  • test/aot/parallel/module-loaders/raw-behind-a-path-asks-for-the-text-of-the-file/main.mjs
  • test/aot/what-require-answered-behind-a-mock-of-a-built-in-module-stays-behind-an-import.test.ts
  • test/aot/engine/ir-1.test.ts

main's own build of b1305dd (#137822) fails all three, along with about thirty more files under test/aot and test/bundler/compile-aot-size.test.ts. The earlier builds of main that I checked, back to b9f2ec3, are red or canceled as well. This PR changes nothing under test/aot or in the AOT compiler, and its last build before the merge (#135601) passed.

I found no fix small enough to port.

Update, after the build ended: it fails 38 files. Each of main's last four finished builds (#137822, #137645, #137628, #137481) also fails 38, and every file that fails here fails in them too. Nothing under test/bundler fails except compile-aot-size.test.ts, which is one of the 38.


Generated by Claude Code

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants