Skip to content

Bump WebKit (oven-sh/WebKit#941 preview): the jobs of an import() run under its AsyncLocalStorage store - #47371

Draft
robobun wants to merge 34 commits into
mainfrom
robobun/ef93224c/module-loader-async-context
Draft

robobun wants to merge 34 commits into
mainfrom
robobun/ef93224c/module-loader-async-context

Conversation

@robobun

@robobun robobun commented Oct 29, 2026 •

Copy link
Copy Markdown
Collaborator

Fixes #32693

Draft until oven-sh/WebKit#941 merges: WEBKIT_VERSION is the preview build of its head. Supersedes #37933.

Problem

  • import() under AsyncLocalStorage.run() loses the store. Bun.plugin() hooks of static dependencies print onLoad(b):undefined. So do module bodies. Bun 1.3.13 and Node.js read the store.
  • JSC's module loader jobs install no async context (runInternalMicrotask, JSMicrotask.cpp).

Fix

Background

Downsides

  • In a Bun.ModuleGraph (bun test --isolate=modules) an import() still loses the store.
  • An import() in a beforeExit listener keeps a store that the entry point entered: the listener reads it since Bun 1.4.2.
  • A job with no store: +7 to +23 instructions (+30 or +46 to evaluate). A host load: +15. libJavaScriptCore.a: +9,595 bytes.
Notes

Repro.

import { AsyncLocalStorage } from "node:async_hooks";
import { plugin } from "bun";
const als = new AsyncLocalStorage();
const seen = [];
const S = () => String(als.getStore());
plugin({
  name: "rt",
  setup(build) {
    build.onResolve({ filter: /.*/, namespace: "virt" }, args => (seen.push("onResolve(" + args.path + "):" + S()), { path: args.path, namespace: "virt" }));
    build.onLoad({ filter: /.*/, namespace: "virt" }, args => {
      seen.push("onLoad(" + args.path + "):" + S());
      if (args.path === "a") return { contents: "import b from 'virt:b'; export default 'a' + b", loader: "js" };
      return { contents: "export default 'b'", loader: "js" };
    });
  },
});
await als.run("IMPORT", async () => {
  const name = "virt:a";
  await import(name);
});
console.log(seen.join("\n"));
line Bun 1.3.13 Bun 1.4.2 main this pin Node.js v26.3.0 (module.registerHooks)
onResolve(a) IMPORT IMPORT IMPORT IMPORT IMPORT
onLoad(a) IMPORT IMPORT IMPORT IMPORT IMPORT
onResolve(b) IMPORT IMPORT undefined IMPORT IMPORT
onLoad(b) IMPORT undefined undefined IMPORT IMPORT

The program of #32693 (a module body under store.run("CONTEXT", () => import(...))) prints CONTEXT on Bun 1.2.23 and 1.3.13, undefined on 1.3.14 to 1.4.4, and CONTEXT with this pin and on Node.js. The module-loader rewrite (#29393) is in that range. The range was not bisected.

The pin. autobuild-preview-pr-941-27493dc0 is the preview build of the head of oven-sh/WebKit#941 (27493dc075): the pin of main (a8d825d8a0) with the change. The merged commit replaces the pin before this leaves draft. oven-sh/WebKit#941 has the design, the table of jobs and the measurements. The size is from the release build of a8d825d8a0 and the preview build of that head (bun-webkit-linux-amd64-aot, size -t): text 73,812,430 to 73,822,025 bytes.

What Bun does besides the pin.

  • ModuleGraph.cpp sets VM::asyncContextOfLoadFilter: a load that begins in the context of a graph keeps nothing.
  • GlobalObject::resetAsyncContextAtEventLoop() (ZigGlobalObject.cpp) is the statement that AsyncLocalStorage: drop enterWith() frames at the event-loop boundary, stop allocating per await/then #41190 put where a checkpoint begins: with no script on the stack, the slot gets the event loop's value. The host now also calls it where it begins a load by itself: JSModuleLoader__import (a preload), JSC__JSModuleLoader__loadAndEvaluateModule and JSC__JSModuleLoader__resolveAndLoadAndEvaluateModule (the entry point, a test file, the entry point of a Worker, a --hot reload), Bun__loadBuiltinEntryPoint, NodeModuleModule__callOverriddenRunMain (a Module.runMain that a preload replaced) and BakeLoadModuleByKey. While script is on the stack the call does nothing, so a load that script begins through one of them keeps the caller's context. drainMicrotasks() compiles to the same 108 instructions as on main, and a load that the host begins executes 14 more and a call (counted in the assembly of ZigGlobalObject.cpp at the flags of a release build, without LTO).
  • vm.SourceTextModule#evaluate() says that it evaluates for its caller (JSC::ModuleEvaluationAsyncContextScope, CalledByScript). The modules that wait there for a top-level await continue the call.

Tests.

  • test/js/bun/plugin/plugins.test.ts, block "AsyncLocalStorage in what an import() loads", 44 tests. On main 24 fail (measured at 98cda0d, and again on the head of Upgrade WebKit to 8156ba68efab #48297, which is main at 89452d2 with the upstream sync). The other twenty are controls for what must stay as it is: the cells that a load allocates with and without a store, a load outside any store, require() of an ES module (six tests, one of them inside a graph), what code of a Bun.ModuleGraph loads (three tests) and what the host loads behind a preload (nine tests). On a build of main's src with the engine change 18 fail (measured on main at 98cda0d with 37998c5702): thirteen of the fourteen tests of a preload, node:vm (three tests) and two of a graph. Seventeen fail on this pull request as it was at a60b510, before the host's rule moved to where a load begins. Four fail on main for the host's part alone, which needs no engine change: the hooks for the next preload and a replaced Module.runMain, each in both ways in which Bun reads a preload. The tests cover:
    • onResolve and onLoad hooks and module bodies of static imports
    • an import() from a timer and from a promise reaction
    • Module.runMain() and import.defer()
    • a CommonJS body and what it requires, an object loader's getter, a build.module() factory
    • the then() of a namespace
    • a load that fails, and a top-level await below another import()
    • enterWith() in the hook of a dependency, in the hook of the imported module itself, and in a CommonJS file
    • a CommonJS file whose asynchronous hook requires a module that imports the file
    • a preload that calls enterWith(): the entry point (--preload, --import, --require, a CommonJS entry point, a preload that replaces Module.runMain, a preload whose body runs behind a top-level await), the next preload and the hooks for it, the entry point of a Worker, and two test files under bun test, bun test --isolate and bun test --parallel=2
    • an entry point that waits for a module that a preload began to import
    • a load outside any store: what a module, a CommonJS file or a hook of it enters reaches what the load runs behind it
    • vm.SourceTextModule#evaluate() under run(), under exit() and plain: in a module of a loading import(), in the drain of a require(), in the entry point, behind a second top-level await, and inside graph.run()
    • mock.module() for a linked module with a top-level await below it
    • require() of an ES module: alone, with an import() that the module calls, with an import() in flight (two shapes), and with top-level awaits that the call begins or settles
    • Bun.ModuleGraph: the host's hooks for a load that the host starts, a module of the host that a graph asks for and that outlives the graph, a hook of the host for a load that a graph starts, require() inside a graph
    • a mock.module() factory
  • test/js/node/async_hooks/async-context/async-context-dynamic-import.js: AsyncLocalStorage-tracking.test.ts runs each fixture of that folder on Bun and on Node.js. It fails on main.
  • test/bundler/bundler_compile_prelinked.test.ts, DynamicImportAsyncLocalStorage: an import() of a chunk in a bun build --compile executable, with and without --bytecode and --splitting. The two variants with --splitting fail on main.
  • test/aot/parallel/modules: twenty programs, for bun build --compile --aot. They pass in the driver (test/aot/run-parallel.ts) in each of its ways. At an earlier pin an executable of main and one of the change differed in 71 of the 143 lines that sixteen of them print.
  • Thirteen more programs there are for one fault of the engine change itself, which a review of [JSC] A job of the module loader runs under the async context of the import() that asked for it WebKit#941 found: in an executable, the jobs that the run time runs behind a CommonJS file that ran ahead of the loader read the store of the import() whose job ran the file. Twelve end with what the executable prints, in the order of time. With the engine before the fix (0d50be2a58), eleven of them fail in all four ways. The twelfth holds a rule that the fix keeps (docs/compiler/compatibility.mdx has it). The thirteenth is marked Todo: a store that script holds around the call that hands the graph on. The executable of main prints the same wrong answer there.
  • Ten more programs and a larger test of compiled test runs are for a second fault of the engine change, which a review on my side found: what the run time hands to the loader inside a job that began behind a file ran under the store of that job. Such a job is a job of node:vm, or a call of import() that is made again while the file waits ([JSC] A job of the module loader runs under the async context of the import() that asked for it WebKit#941 has the account).
    • Seven programs end with what the executable prints. With the engine before the fix (0f5a1edd0d) five of them fail. The other two pass before the fix too. One holds the store of the caller of require() with no file that waits. One holds a plugin's hook where the load of an import() under a store goes on inside require() under another store: the hook prints the store of the import().
    • Three programs are marked Todo. Two print every line but that of a mock's factory as bun <file> does: they fail before the fix too. The third is a call that goes on in a reaction of a promise.
    • test/aot/an-import-call-in-a-compiled-test-run-reads-its-store.test.ts has nine projects now. Each expects what bun test prints and runs under bun test and bun test --aot. With the engine before the fix six fail under --aot. One of them has no AsyncLocalStorage: a plugin's hook asks for a mocked module and got the module. Two hold what a call in the context of a Bun.ModuleGraph keeps. Five more projects are known wrong answers of the compiled run, as test.todo with the reason beside each.
    • A file waits in these tests where it calls Bun.build(), which runs the event loop until setup() of a plugin has ended. That wait is to go away (event loop: never re-enter from inside a dispatch callback (de-block expect().resolves and HTMLRewriter.transform, then assert) #33261). Each test checks first that it waited, and says so if not. Of those that fail before the fix, one program and one project need no file that waits.
    • The driver runs each reference at default flags, where plain Bun reads the texts of modules on its own threads. Nine programs hold what plain Bun prints.
    • For seven programs that reference is a race between those threads: plain Bun prints up to six outputs in eight runs. They say what the executable prints.
    • Two programs call import() while a require() loads an ES module. A compiled executable runs that import() after the call, under the store of the import(), as Node.js does (docs/compiler/compatibility.mdx). They say what the executable prints too.
    • require-of-graph-that-awaits-under-store-file-calls-import: the module behind a top-level await of a required graph runs under no store, and not under the store of an import() that a CommonJS file of the graph called.
    • imports-under-stores-files-queue-jobs-stores-are-collected: a collection takes the stores of six calls that are through, three fulfilled and three rejected. Before d6f315cb63 of [JSC] A job of the module loader runs under the async context of the import() that asked for it WebKit#941 the executable kept them.
    • import-from-text-of-a-required-cjs-under-store-reads-its-exports: import() from text of a CommonJS file of the executable calls a getter and the traps of a Proxy under the store of the call, as plain Bun does.
    • import-without-store-is-made-again-while-slot-holds-store: a call with no store that is made again when its text has come reads no store that its caller entered behind the call.
  • Not in this branch: a stored test run for a mock factory below the root of an import() under a store (test/aot/parallel-test-runs). On the branch of AOT: the next base #48120 it passed in six of six runs where the process had one processor, so that the compiler had one thread. With the default threads the debug build of the compiler stopped in its link stage in seven of eight runs, before the program ran ([JSC] A job of the module loader runs under the async context of the import() that asked for it WebKit#941 names the frames). In a debug build it would fail for a reason that is not its subject.
  • test/aot/an-import-call-in-a-compiled-test-run-reads-its-store.test.ts: bun test, bun test --isolate and bun test --aot on one project. A compiled test run reads the text of what an import() names in a later turn, and the modules still read the store of the call. They fail on main. bun test --isolate=modules is a todo there: a test file is a module of a graph.

Other suites. On this head (main at 30261e2, the engine at 27493dc075), in a debug and ASAN build with the CI build of the pin: the 323 tests of plugins.test.ts, 39 programs in the driver in its four ways (four more are marked Todo), the test of compiled test runs and test/aot/ci-lists.test.ts pass. One commit back (the same main, the engine at 0f5a1edd0d) the 323 tests, 32 programs, the test of a compiled test run and test/aot/ci-lists.test.ts passed. One merge back (main at 130ff8a, the engine at a62dd4d93c) the same passed. One commit before that (the same main, the engine at 0d50be2a58) the 44 tests, the twenty programs, the test of a compiled test run and test/aot/ci-lists.test.ts passed. Two merges back (main at 237bba5, the engine at 7aa751c338) the same four passed. Three merges back (main at 5f3f8fe, the engine at 2bc7c74156) the same four passed. Four merges back (main at 6f8424e, the engine at 31ae46ffc3) the same four passed. Five merges back (main at ab9b2eb, the engine at ec0591031c) the same four passed, and so did main's new tests for an import() of a file that throws (#48993): the twelve programs of test/aot/parallel/import-calls-beside-a-require-that-throws in the four ways, and the two stored test runs of import-calls-of-a-file-that-throws. Six merges back (main at 9098897, the engine at 5557a0f6ec) the first four passed, also on the head of #48926 (795f1a9) with the preview build of 58c7da9751. Seven merges back (main at a62cc28, the engine at 4ee8e65bd8) the same four passed. Eight merges back (main at d0f4fdd, the same engine), main's harness for stored test runs (test/aot/test-runs-in-folders-*.test.ts) ran the 443 projects of the 60 folders for an import() under a mock, or with a text that comes in a later turn: 441 passed, and the other two passed at a second run alone. In those two the compiled run had ended before its tests, as the debug build of the compiler does now and then in its link stage (oven-sh/WebKit#941 has the account and the sanitizer's report). Nine merges back (main at 8aa496e, pin 1c4943f623) the first four passed. Ten merges back (main at ccb3611, pin 5b6deeea52), main's harness for stored test runs (test/aot/test-runs-in-folders-*.test.ts) ran the 443 projects of the 60 folders for an import() under a mock, or with a text that comes in a later turn: 442 passed, two of them at a second run alone. In the other one the compiled run ended before its tests in 3 of 23 runs of the harness and passed in the other 20 (oven-sh/WebKit#941 has the account). Eleven merges back (main at 4385062, pin fd9c7641af), with five tests at once and a limit of 60 s, 3,140 tests passed: plugins.test.ts (323), the five files of test/js/node/async_hooks (154), the nine of test/js/node/vm (423), node-module-module.test.js (83), preload-test.test.js (8) and seven files of test/js/bun/module-graph (2,149). bundler_compile_prelinked.test.ts passed 89 of 91: two builds in the first minute of the file ran over the limit, and they passed when they ran again. module-graph-compile.test.ts passed 76 of 76 with a limit of 180 s. The earlier heads passed the same lists in the same way: main at f336318 with pin bb2e5abe7f, main at 447ff87 with pin 102d1739cb, the head of #48297 before it merged, main at 89452d2 with pin bacfec4b27 and main at 98cda0d with pin 37998c5702. On a build of main's src with the engine change no other test of main failed for that reason: the 279 other tests of plugins.test.ts, vm.test.ts, AsyncLocalStorage.test.ts, AsyncLocalStorage-tracking.test.ts, node-module-module.test.js, preload-test.test.js and seven files of test/js/bun/module-graph were run (main at 98cda0d with 37998c5702). At the default of twenty tests at once many tests of these files run out of their 5 s limit in a debug build on this machine, with and without this change. A debug build of main stopped on this machine at BunPlugin.cpp:3562 from #48677 to #49013 (JSC::SuspendExceptionScope with no include of FrameTracers.h), with and without this change: for the heads in between the local build had that include added, and the commits did not.

What changes besides the fix.

  • Bun.ModuleGraph: where the host starts a load under a store (graph.import()), the host's hooks for the static dependencies read that store. A load that begins in the context of a graph keeps no context, so what code of a graph loads runs as before. The context of a graph is in the same slot as the stores, and what runs in it ends when the graph is disposed. A hook of the host and a module of the host's loader must not end with a graph: with a top-level await, such a module would never go on. JSModuleGraph sets that rule as the VM's filter. So inside a graph an import() under a store still loses the store, and a test file under bun test --isolate=modules is a module of a graph.
  • One import() can start a dependency that has a top-level await. A module of another load that waits for that dependency runs in the store of the first import(). The other load can be the host's: an entry point or a test file that waits for a module that a preload began to import reads the store of that import(), and so does what it starts (a server's handlers, later timers). Node.js does the same. Bun 1.3.14 to 1.4.4 read none there.
  • Under an import() with a store, a module enters another store with enterWith(), and the next module of the graph has a top-level await. The modules that wait for it run under the entered store. Node.js does the same.
  • Under a store, a store that the hook of a dependency enters with enterWith() ends with the hook: the hooks behind it and the module bodies read the store of the import(). Node.js and Bun 1.4.4 hand the entered store on to what the load runs next. Bun 1.3.13 handed it to the hooks behind it and not to the bodies. The hooks for the module that import() names run inside the call, so what they enter is what the load keeps.
  • vm.SourceTextModule#evaluate() under a store: the modules that wait for a top-level await read the store of the call, also under run(), behind a second top-level await, and in the entry point. Under exit() they read none. Node.js does the same. On main they read none in each case.
  • The hooks for a preload, and a Module.runMain that a preload replaced, read no store that the preload before them entered. Bun read it where it reads a preload on the thread of the program: on main by default (bun test: Vitest compatibility #44833, is_main in src/runtime/jsc_hooks.rs), and in Bun 1.4.2 to 1.4.4 with BUN_FEATURE_FLAG_DISABLE_ASYNC_TRANSPILER=1 and in a compiled executable whose preloads are embedded.

What does not change.

Differences from Node.js that remain (Node.js v26.3.0, and v26.12.1, which this repository pins, for the rows of a preload and of node:vm). The tests of the block were also run on Bun 1.3.13 and, ported to module.registerHooks, on Node.js.

Older than 1.3.14 (Bun 1.3.13 does the same):

  • A CommonJS file that an ES module imports runs at load time in Bun, in a job of its own. Under a store, a store that the file enters with enterWith() ends with that job. Node.js runs the file with the module bodies, so the importing module sees the store.
  • Two import() calls that load one module at the same time: the module runs in the store of one of them. In 8 runs it was the first call 4 times. Node.js: always the first. The order in which Bun hands transpiled modules to the loader decides (Hand transpiled modules back to the module loader in request order #42051).

Since 1.3.14, and kept (Bun 1.3.13 does what Node.js does):

  • A require() call loads and evaluates every module of the loader that becomes ready while it runs, inside the call: also modules of an import() in flight whose graph it completes, and of an import() that a required module calls. They read the caller's store, and vm.SourceTextModule#evaluate() in such a module hands that store on. Node.js evaluates them later, under the store of their import(). A module of such an import() that runs after the call, behind a top-level await, reads the store of its import(). A bun build --compile --aot executable runs an import() that a required module calls after the call, under the store of that import(), as Node.js does.

New with this change:

  • What the hook of a dependency enters with enterWith() ends with the hook (see "What changes besides the fix").
  • vm.SourceTextModule#evaluate() with nothing current at the call: the modules that wait read what the slot holds when their job runs, for example a store that the awaiting module entered. Node.js reads none there. (Not new: main reads the same.)

The tests of the block that pin one of these outputs say in a comment what Node.js prints.

Review. The design, the rules and the measurements are in oven-sh/WebKit#941, with the account of each review. The last three reviews raised 98 concerns, and 86 are answered. What they changed here:

  • A load that the host begins starts under no store. The first answer was a flag in the loader, and a preload that replaces Module.runMain went past it. The second answer was a reset where a checkpoint ends: it changed what listeners read in the entry point's first turn, Preserve AsyncLocalStorage context in unhandledRejection handlers #31721 has it with its other half, and a deferred task, the printer of a rejection and a --hot reload went past it. The reset is now where the host begins a load.
  • evaluate() of node:vm continues its caller. mock.module() had the same scope for one review, and lost it.
  • The tests for both.

Not taken: a safeguard that lets the engine half merge alone (see "The order" in oven-sh/WebKit#941: the host's reset and its tests need no engine change and can go first). Three debug assertions of the loader fire in programs that nest require() of ES modules, with no AsyncLocalStorage in the program. They fired on a debug build of the pin that main had then too (5a61071f79: JSModuleLoader.cpp lines 869, 1147 and 1148).

Reach. Module bodies have one report (#32693) and one downstream: manten skips three specs on Bun for it. With the skips removed, Bun 1.4.2 gives 117 pass and 3 fail, and the first commit of oven-sh/WebKit#941 gave 120 pass and 0 fail. One of the three specs alone still fails in 5 of 10 runs for the reason in #42051. The hooks, the factories and the CommonJS bodies have no report. They kept the store in Bun 1.3.13 and keep it in Node.js.


no test proof · iteration 10 · platform-specific test(s) that do not run on this machine, deferring to CI, which covers all platforms: test/js/bun/plugin/plugins.test.ts, test/bundler/bundler_compile_prelinked.test.ts

…store

The pin is the preview build of oven-sh/WebKit#941: main's pin 8c6c963e3c plus one commit.

An import() under AsyncLocalStorage.run() lost the store when the call returned. What the load ran later read no
store: Bun.plugin() onResolve and onLoad hooks of static dependencies, module bodies, CommonJS bodies, mock.module()
factories and the then() of a namespace. With this pin a request of the module loader keeps the async context of its
caller, and its jobs run under it where it is set.

Tests:
- test/js/bun/plugin/plugins.test.ts: the hooks, the bodies, Module.runMain(), import.defer(), a load that fails, a
  top-level await below another import(), enterWith() in a hook and in a CommonJS file, Bun.ModuleGraph, require() of
  an ES module, mock.module(), and the cells that a load allocates.
- test/js/node/async_hooks/async-context/async-context-dynamic-import.js: runs on Bun and on Node.js.
- test/bundler/bundler_compile_prelinked.test.ts: an import() of a chunk in a compiled executable.
- test/aot/parallel/modules: seven programs that print the same as plain Bun and as compiled executables.
@robobun

robobun commented Oct 29, 2026 •

Copy link
Copy Markdown
Collaborator Author
Updated 8:35 PM PT - Nov 17th, 2026

✅ @robobun, your commit 8d850ee19df2dbb51aebe7b0ada8756786daec5f passed in Build #142797! 🎉


🧪   To try this PR locally:

bunx bun-pr 47371

That installs a local version of the PR into your bun-47371 executable, so you can run:

bun-47371 --bun

@robobun

robobun commented Oct 29, 2026 •

Copy link
Copy Markdown
Collaborator Author

Status. Reproduced with the script in the Notes of this pull request. On main, onResolve(b) and onLoad(b) print undefined for an import() under AsyncLocalStorage.run(). On this branch they print IMPORT, as in Node.js. The engine change is oven-sh/WebKit#941. This pull request stays a draft until that merges.

The branch has main at 30261e2. The pin is the preview build of the head of oven-sh/WebKit#941 (27493dc075): main's pin a8d825d8a0 with the change.

On a debug and ASAN build of this tree: the 44 new tests of plugins.test.ts pass, 39 programs of test/aot/parallel/modules pass in the driver in its four ways, and test/aot/an-import-call-in-a-compiled-test-run-reads-its-store.test.ts passes. On main at 98cda0d, 24 of the 44 tests fail.

A review of oven-sh/WebKit#941 found that in a compiled executable the module of an import() with no store can read the store of another import() (thread). That is fixed in a62dd4d93c, and main did not have it: the engine change had introduced it. Twelve programs hold it, and a thirteenth is marked Todo. With the engine before the fix, eleven of the twelve fail in all four ways.

A review on my side found a second fault of that kind, and 27493dc075 fixes it: what the run time hands to the loader inside a job that began behind a file ran under the store of that job (a job of node:vm, or a call of import() that is made again while the file waits). Five new programs and six compiled test runs fail with the engine before that fix. One of the test runs has no AsyncLocalStorage: a plugin's hook asked for a mocked module and got the module.

The review threads of oven-sh/WebKit#941 are answered with runs. What they leave is named in that pull request, and this branch holds it as known wrong answers: five compiled test runs that are todo and three programs that are // Todo:.

CI on 8d850ee passed: build 142797 ended with all 188 jobs passed. Three tests passed there on a retry.

…o async context

With oven-sh/WebKit#941 a load keeps the async context of its caller for its jobs. The context of a graph is in
the same slot as the AsyncLocalStorage stores, and what runs in it ends with the graph. The jobs of a load run the
host's hooks and the modules of the realm's own loader, and neither is a graph's. JSModuleGraph sets the loader's
filter for what a load keeps: none where the caller is in the context of a graph, so those jobs run as before.

Tests: a synchronous load, a top-level await and a Bun.ModuleGraph under AsyncLocalStorage stores, in
plugins.test.ts and as programs of test/aot.
Comment thread src/jsc/bindings/ModuleGraph.cpp Resolved
Comment thread src/jsc/bindings/ModuleGraph.cpp Resolved
…b59d8fc79

main pins a7b59e61de now. The preview is that commit plus the commits of oven-sh/WebKit#941.
main pins 4d45755fed now. 1a52269eca is that commit plus the commits of oven-sh/WebKit#941.
… its call in a compiled executable

A program for test/aot/parallel that runs with the transpiler on its own threads. Without the commit of oven-sh/WebKit#941 that keeps the async context in TextThatWaits, five of its ten lines differ from plain Bun.
…f its call in a compiled executable

A program for test/aot/parallel. With oven-sh/WebKit#974 the second import() waits behind the first load and is made again in a job of that load. Without the commit of oven-sh/WebKit#941 that keeps the async context in ImportBehindLoad, three of its seven lines print the store of the first import().
main has the rewritten ahead-of-time compiler (#47942) and pins WebKit 5a61071f79. 328b6c9bb4 is that commit with the commits of oven-sh/WebKit#941 merged in.
…llected, and a compiled test run

The driver runs every reference at default flags now, where plain Bun reads the texts of modules on its own threads.

- Seven programs have a reference that is a race between those threads: they say what the executable prints.
- Two programs call import() while a require() loads an ES module. The rewritten run time runs that import() after the require(), as Node.js does, so they say what the executable prints too.
- New: a require() of a graph that awaits, whose CommonJS file calls import() under a store. The module behind the await runs under no store.
- New: six import() calls under a store each, whose CommonJS files queue a job. A collection takes the stores when the calls are through.
- New: an import() in a compiled test run (bun test --aot) reads the store of its call.
- One program no longer sets BUN_FEATURE_FLAG_DISABLE_ASYNC_TRANSPILER, which the driver refuses.
test/aot/ci-lists.test.ts asks that every test file of test/aot is in test/expected-durations.json, and that a long one is in excludeFiles of test/parallel-allowlist.json.
… and evaluate() continues its caller

The pin is oven-sh/WebKit#941 on the pin of main, with its last commit: the filter of what a load keeps is one
pointer on the VM, and a host function that evaluates a module for its caller says so.

- GlobalObject::resetAsyncContextAtEventLoop() is the statement that drainMicrotasks() has where a checkpoint begins.
  The host now also calls it where it begins a load with no script on the stack: a preload, the entry point, a test
  file, the built-in entry point, a Module.runMain that a preload replaced, a module of the bundler's server. What the
  script that ran last left in the slot with enterWith() is not the async context of that load. With the pin, the load
  would keep it for all of its jobs.
- ModuleGraph.cpp sets VM::asyncContextOfLoadFilter.
- vm.SourceTextModule#evaluate() continues its caller: the modules that wait there for a top-level await read what
  the caller reads.

Tests: a preload that calls enterWith() (the entry point, a CommonJS entry point, a replaced Module.runMain, the next
preload and its hooks, the entry point of a Worker, test files under three modes of bun test), an entry point that
waits for a module that a preload began to import, node:vm, mock.module() for a linked module, a load outside any
store, and three AOT programs.
The fixture gave the Worker the pathname of a file URL as its preload. On Windows that is /C:/..., which does not
resolve, so the test of the Worker's preload failed there.
main has #48120, which pins WebKit at ee59053d32. WEBKIT_VERSION is now 37998c5702: oven-sh/WebKit#941 with that
commit merged in.

Conflicts: the pin, the end of plugins.test.ts (main's new tests, then the block of this branch), and the two lists of
CI's runner, which take main's and get the test of this branch again (test/aot/ci-lists.test.ts).
main has #48436, which pins WebKit at 873ab3a426. WEBKIT_VERSION is now bacfec4b27: oven-sh/WebKit#941 with that
commit merged in.

Conflicts: the pin, and one list of CI's runner, which takes main's and gets the test of this branch again
(test/aot/ci-lists.test.ts).
main has #48478, which pins WebKit at 264cf6db41. WEBKIT_VERSION is now 102d1739cb: oven-sh/WebKit#941 with that
commit merged in.

Conflict: the pin.
main has #48297, the upgrade to the upstream sync, which pins WebKit at 7f020d2818. WEBKIT_VERSION is now bb2e5abe7f:
the head of oven-sh/WebKit#941, which is that commit with the change.

Conflict: the pin.
main has #48617, which pins WebKit at 7101dff5ab. WEBKIT_VERSION is now fd9c7641af: the head of oven-sh/WebKit#941,
which is that commit with the change.

Conflicts: the pin, and the two lists of CI's runner, which take main's and get the test of this branch again
(test/aot/ci-lists.test.ts).
main has #48677, which pins WebKit at 6fc709c298. WEBKIT_VERSION is now 5b6deeea52: the head of oven-sh/WebKit#941,
which is that commit with the change.

Conflicts: the pin, and one list of CI's runner, which takes main's and gets the test of this branch again
(test/aot/ci-lists.test.ts).
main has #48714, which pins the preview build of oven-sh/WebKit#1042. That pull request is merged as b553971ea8.
WEBKIT_VERSION is now the preview build of the head of oven-sh/WebKit#941 (1c4943f623), which is b553971ea8 with the
change.

Conflicts: the pin, and one list of CI's runner, which takes main's and gets the test of this branch again
(test/aot/ci-lists.test.ts).
main has #48750: its pin is b553971ea8 itself now, which is oven-sh/WebKit#1042 as merged. WEBKIT_VERSION stays the
preview build of the head of oven-sh/WebKit#941 (1c4943f623), which is b553971ea8 with the change.

Conflict: the pin.
main has #48771: its pin is de043228c2, which is oven-sh/WebKit#1045 as merged. WEBKIT_VERSION is now the preview
build of the head of oven-sh/WebKit#941 (4ee8e65bd8), which is de043228c2 with the change.

Conflicts: the pin, and the end of test/parallel-allowlist.json (main's list, then the one test file of this change).
main still pins WebKit at de043228c2, so WEBKIT_VERSION stays the preview build of 4ee8e65bd8 (de043228c2 with the
change of oven-sh/WebKit#941).

Conflict: the imports of test/js/bun/plugin/plugins.test.ts. main's #48873 takes `autoInstallTempDir` from the harness
there, and this branch takes `mock` from bun:test. Both stay.
main has #48815: its pin is c495a7e0e4, which is oven-sh/WebKit#1047 as merged. WEBKIT_VERSION is now the preview
build of the head of oven-sh/WebKit#941 (5557a0f6ec), which is c495a7e0e4 with the change.

Conflict: the pin.
main has #48926 and #48993: its pin is b00aff56c9, which is oven-sh/WebKit#1059 as merged. WEBKIT_VERSION is now the
preview build of the head of oven-sh/WebKit#941 (ec0591031c), which is b00aff56c9 with the change.

Conflict: the pin.
main has #49004: its pin is ed67c459b7, which is oven-sh/WebKit#1061 as merged. WEBKIT_VERSION is now the
preview build of the head of oven-sh/WebKit#941 (31ae46ffc3), which is ed67c459b7 with the change.

Conflicts: the pin, and test/parallel-allowlist.json, where main's list ends with six new files of test/aot.
The file of this branch stays the last entry.
main has #49037: its pin is 9e983dfc1d, which is oven-sh/WebKit#1063 as merged. WEBKIT_VERSION is now the
preview build of the head of oven-sh/WebKit#941 (2bc7c74156), which is 9e983dfc1d with the change.

Conflict: the pin.
main has #49081: its pin is 3bc9421149, which is oven-sh/WebKit#1064 as merged. WEBKIT_VERSION is now the
preview build of the head of oven-sh/WebKit#941 (7aa751c338), which is 3bc9421149 with the change.

Conflict: the pin.
main has #49108: its pin is ae8978d033, which is oven-sh/WebKit#1067 as merged. WEBKIT_VERSION is now the
preview build of the head of oven-sh/WebKit#941 (0d50be2a58), which is ae8978d033 with the change.

Conflicts: the pin, and test/parallel-allowlist.json, where main and this branch each add entries at the end
of the list. All three entries stay.
robobun added a commit to oven-sh/WebKit that referenced this pull request Nov 17, 2026
…er read no async context of that job

In a compiled executable a CommonJS file of an imported graph runs ahead of the loader. Behind it,
Modules::transferGraphToLoader() runs the jobs that were queued in front of the step of that file,
inside the job that ran the file. On this branch a job of the loader enters the async context of
its load, so those jobs ran inside that context too, and a job of a load that keeps no context
enters nothing: the module of an import() with no store read the store of another import()
(#941 (comment)). main does not have that,
because no job of the loader enters anything there.

Now those jobs read what the job found in the slot where it entered, and what they leave there is
what the job puts back at its end, as if they had run in front of it:

- ModuleLoaderJobAsyncContextScope is the scope of a job of the loader: of the twelve job functions
  in JSMicrotask.cpp. With ENABLE(AOT) the VM has the scopes of the jobs that entered, the innermost
  first (VM::m_asyncContextScopeOfModuleLoaderJob, and each scope has the one around it). Without
  ENABLE(AOT) the class adds nothing to ModuleLoaderAsyncContextScope.
- Where the files of a load begin to run ahead, Modules notes the innermost job that the steps in
  front of the files are a part of too (EagerLoad::jobAroundStepsInFront). The job of a load that
  keeps a context runs the files, and those steps are no part of it. A graph that is handed to the
  loader begins files again for the same load inside the transfer, and takes the note of that load.
- AOT::AsyncContextOutsideModuleLoaderJobsScope, around runStepsOfLoaderInFrontOf(), puts into the
  slot what the outermost job inside of the noted one found there, and takes what the steps leave
  as what that job puts back. runStepsOfLoaderInFrontOf() takes it by reference, so the call does
  not compile without it.

The scope of a job that enters nothing writes nothing for this. The registry of evaluations
(VM::m_moduleEvaluationAsyncContextScope) needs no change for the steps: a module with a top-level
await begins in executeAsync() only, under a job function that has a scope of its own.

An earlier commit of this branch put the context from before the files back at that call
(asyncContextBeforeFiles), and merge 516122f dropped it. That one took what the slot held where
the files began: the store of the load, where a job of the load runs the files. So it did not
close this. It is what a load with no job around it needs (see below).

Tests:
- testaot module-programs/imports-in-flight-where-file-of-import-under-async-context-calls-import.
  Without the change its fourth line is `runs: third.js, under the context of the import`.
- testaot module-programs/steps-inside-one-job-of-loader, job-of-loader-that-enters-nothing,
  steps-inside-jobs-of-loader-that-nest and steps-inside-one-job-of-loader-twice hold the chain and
  the scope on a bare VM.
- Bun's test/aot has thirteen programs for it (oven-sh/bun#47371).

Not in this commit:
- A store that script holds around the call that hands the graph on, where no job of the loader
  entered: the steps in front read it. Bun's main has that too, and one program of
  oven-sh/bun#47371 marks it as a known wrong answer.
- A call of import() that is made again where its text has come (startImportWhoseTextHasCome(),
  makeCallAgain()) enters its context with a plain AsyncContextSwapScope, and is no job here. Where
  it hands a required graph to the loader (transferSynchronousLoad()), the steps of that graph read
  the store of the call. That is this branch's own, and two more review threads are about the same
  scope (r4585961473, r4585961488): it gets a change of its own.
…he loader, and add programs for it

WEBKIT_VERSION is now the preview build of a62dd4d93c, the head of oven-sh/WebKit#941: main's pin
ae8978d033 with that change. Its last commit answers a review of it: in a compiled executable, the
jobs that the run time runs behind a CommonJS file that ran ahead of the loader read no store of the
import() whose job ran the file.

Thirteen programs under test/aot/parallel/modules hold it. Twelve end with what the executable
prints, in the order of time: the executable runs the file ahead of the loader, so the order is not
that of `bun <file>`, and a module between two lines of the file is what each program is for. On the
preview build of the head before that commit (0d50be2a58), eleven of the twelve fail in all four ways
of the driver. The twelfth holds a rule that the fix keeps: a load with no store that a drain under a
store takes reads that store. docs/compiler/compatibility.mdx now says so.

The thirteenth is marked Todo. A store that script holds around the call that hands the graph on is
no job of the loader, so the modules of other calls still read it. The executable of main prints the
same wrong answer.
robobun added a commit to oven-sh/WebKit that referenced this pull request Nov 18, 2026
…e of the jobs that began behind the file

In a compiled executable and in a compiled test run a CommonJS file of an imported graph runs ahead
of the loader. Where something in the file makes the loader go on (a call of import(), a drain),
Modules hands the graph of the file to the loader: the load of the file goes on
(innerModuleLoading()), and the steps in front of the file run (runStepsOfLoaderInFrontOf(), and
for a required graph transferSynchronousLoad()). The last commit made the steps in front of the
file run outside of the job that ran the file. This one is about a job of the loader that began
behind the file, inside it, and entered an async context: what Modules hands over there ran
under that context. Two kinds of job do that.

A job of node:vm. The file evaluates a module of node:vm under a store and runs the queue of its
context, and the module calls import(). The load of the file went on inside that job: a hook of the
host for the next module of a load with no store read the store of the job. Nothing has to wait
for this one.

A call of import() that Modules makes again where the text of its module has come
(startImportWhoseTextHasCome(), makeCallAgain()). It goes on under the async context that it was
made under, because the slot is how Modules hands that context to the load and to what continues
the call. It entered that context with a plain AsyncContextSwapScope, so the VM did not have it
among the jobs of the loader. A file can wait for the event loop (today: Bun.build() runs it while
setup() of a plugin runs, and that is to go away, oven-sh/bun#33261), and the call is then made
again inside the file:
- the steps of a required graph in front of the file read the store of the call, and what they
  left in the slot ended with the call;
- the load of the file went on under the store of the call, as above;
- the async context says more than stores. While the factory of a mock runs, it says whose factory
  runs, and an import() of the mocked module from there gets the module itself. So in a compiled
  test run with no AsyncLocalStorage in it, a hook of the host that asked for a mocked module got
  the module, and the module ran.
The executables and the compiled test runs of Bun's main show none of this: no job of the loader
enters a context there.

Now:

- A call that is made again is a job of the loader, and has one scope:
  ModuleEvaluationAsyncContextScope with CallMadeAgain. It always enters the context of its call,
  and the VM has it. The comment at the class keeps the two halves of "always" apart: for a call
  with no context it is the rule, and in the drain of a synchronous load it is there for the slot
  alone.
- EagerLoad::jobAroundFiles is the innermost job where the files began to run ahead. In
  transferGraphToLoader() the load goes on outside of the jobs that began later
  (AsyncContextOutsideModuleLoaderJobsScope with that job), and that scope ends before the scope of
  the steps in front begins: two scopes of one job come one after the other.
- transferSynchronousLoad() has the scope around loadModule() and the steps, and sets the registry
  of evaluations aside for them, as a synchronous load does for itself. A module that began to
  evaluate among the steps (mock.module() evaluates a linked one) handed the context of the call to
  the modules that wait for it. runSynchronousStepsInFrontOf() takes the scope by reference, as
  runStepsOfLoaderInFrontOf() does, so the call does not compile without it.
- fulfillQueuedFetchAfterJobs() registers its last job under no async context. That job makes a
  call again, and a reaction runs under what was current where it was registered: the call would
  find the context of its own caller in the slot, and the steps in front of a file would read it.

The part about a call that is made again is a guard for the state in which a file waits, and it
is small. It does not make the slot a sound carrier of the context of a call. What is left, all of
it this branch's own unless it names main:

- What a call that is made again runs at once reads the context of the call: the factory of a mock
  below it. Under `bun test` a job of the load runs that factory, and such a job enters nothing in
  the drain of require() and for a load with no context. So the factory reads the store of the
  call inside require() where `bun test` prints the store of the caller of require(). And it reads
  none for a call with no context where `bun test` prints what the slot holds: inside require()
  under a store, and in the turn in which a module entered a store and laid a mock anew (nothing
  waits there, and there is no require()). main's compiled run prints what `bun test` prints in
  these, unless the mock stood before the call.
  (#941 (comment))
- A call that is made again can wait behind the factory of a mock. It then goes on in a reaction of
  a promise (importBehindFactories()), which runs under the context of the call and is no job of
  the loader. Where that reaction runs inside a file of a required graph, the steps in front of
  the file read the store of the call: `outside held` where `bun test` and main print undefined.
  A drain of the jobs in the file is enough for it (bun:jsc), nothing has to wait.
  importByKeyBehindFactories(), resumeDelayedImport() and didFetchFactoryChainImport() are
  reactions of the same kind.
- A call in the context of a module graph keeps what the filter of the host leaves, which is
  nothing. Where its mocks wait for its text, the factory of a mock below it reads neither the
  graph nor a store, as under `bun test`, and main's compiled run shows it both. Where the factory
  runs inside the call of import() itself, this branch shows it both, as main does.
  (#941 (comment))
- A transfer that begins inside another transfer of the same load takes the jobs as it finds
  them. The one program that reaches that does not end on main either.

The change that closes the first two hands the context of a call to the load and to those
reactions by value, so that a call that is made again enters as a job of the loader does. That
changes how aot/runtime carries a context, and it is the owner's to want.

Three answers of a compiled test run to "the module or its mock" change with this commit. One is
the fault above: it is again what `bun test` and main say. One becomes what `bun test` says where
main says the module, and only in a run that tracked async contexts before the call. In the third
`bun test` says the module and main's compiled run says the mock: the last commit said the module
through the fault above, and this commit says the mock again.

Tests:
- testaot module-programs/call-made-again-under-store-is-job-of-loader,
  call-made-again-without-store-is-job-of-loader,
  call-made-again-inside-synchronous-load-is-job-of-loader,
  steps-of-synchronous-load-inside-job-of-loader-and-call-made-again and
  load-goes-on-and-steps-run-inside-call-made-again hold the scopes on a bare VM.
- Bun's test/aot has programs and compiled test runs for each fault above, and rows that are
  marked as known wrong answers for what is left (oven-sh/bun#47371). One program and one compiled
  test run that fail without this commit need no file that waits.
… to the loader inside a later job, and add tests for it

The engine change made a job of the loader enter the store of its load. In a compiled executable and in a
compiled test run, a CommonJS file of an imported graph runs ahead of the loader, and the run time hands
the graph of the file to the loader where the file makes the loader go on. Where a job that entered a
store began inside the file (a job of node:vm, or a call of import() that is made again where its text
has come), what was handed over ran under that store: a plugin's hook for the next module, a package of
a required graph, and a mocked module that a hook asks for got the module in place of its mock. Bun's
main has none of that. The pinned commit fixes it.

Programs (test/aot/parallel/modules), each fails with the engine before the fix unless it says so:
- import-under-store-is-made-again-while-file-of-required-module-waits and
  import-without-store-is-made-again-while-file-of-required-module-waits: the steps of a required graph.
- factory-of-mock-in-required-module-runs-where-import-under-store-is-made-again: a factory that runs
  where the loader is asked for the required graph again.
- import-under-store-is-made-again-while-file-of-module-that-job-of-import-requires-waits: the steps read
  the store of the caller of require(), which is not what the queue had.
  file-of-module-that-job-of-import-under-store-requires-calls-import holds the same with no file that
  waits, and passes before the fix too.
- load-of-file-without-store-goes-on-inside-job-of-vm-module-under-store: no file waits.
- load-of-file-of-import-under-store-goes-on-inside-require-under-another-store: the load of an import()
  under a store goes on inside require() under another store, and a plugin's hook reads the store of
  the import(). It passes before the fix too. It holds what the scopes of the loader do while the run
  time has the queue of a synchronous load set aside
  (oven-sh/WebKit#941 (comment)).
- import-under-store-is-made-again-where-factory-of-mock-calls-import and its twin without a store print
  the right lines but for the factory of a mock, which is a known wrong answer (// Todo).
- import-under-store-waits-behind-factory-and-goes-on-inside-file-of-required-module is a known wrong
  answer as a whole (// Todo): the call goes on in a reaction of a promise.
- The head of import-under-store-is-made-again-while-file-of-import-under-store-waits said that the
  loader runs the other jobs where the call is made again. It runs them where the file calls import().

Compiled test runs (test/aot/an-import-call-in-a-compiled-test-run-reads-its-store.test.ts): each project
expects what bun test prints, and runs under bun test and bun test --aot.
- Six fail under bun test --aot with the engine before the fix: a call made again inside require(), the
  same with a module that awaits among the steps, the same with a factory that calls require(), the load
  of a file that waits, the load of a file inside a job of node:vm (no file waits), and a mocked module
  that a hook asks for (no AsyncLocalStorage).
- Two hold what a call in the context of a Bun.ModuleGraph keeps.
- Five are known wrong answers of the compiled run, as test.todo with the reason beside each.

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

AsyncLocalStorage store is lost during dynamic import() module evaluation

2 participants