Repository navigation
Conversation
…store The pin is the preview build of oven-sh/WebKit#941: main's pin 8c6c963e3c plus one commit. An import() under AsyncLocalStorage.run() lost the store when the call returned. What the load ran later read no store: Bun.plugin() onResolve and onLoad hooks of static dependencies, module bodies, CommonJS bodies, mock.module() factories and the then() of a namespace. With this pin a request of the module loader keeps the async context of its caller, and its jobs run under it where it is set. Tests: - test/js/bun/plugin/plugins.test.ts: the hooks, the bodies, Module.runMain(), import.defer(), a load that fails, a top-level await below another import(), enterWith() in a hook and in a CommonJS file, Bun.ModuleGraph, require() of an ES module, mock.module(), and the cells that a load allocates. - test/js/node/async_hooks/async-context/async-context-dynamic-import.js: runs on Bun and on Node.js. - test/bundler/bundler_compile_prelinked.test.ts: an import() of a chunk in a compiled executable. - test/aot/parallel/modules: seven programs that print the same as plain Bun and as compiled executables.
|
Updated 8:35 PM PT - Nov 17th, 2026
✅ @robobun, your commit 8d850ee19df2dbb51aebe7b0ada8756786daec5f passed in 🧪 To try this PR locally: bunx bun-pr 47371That installs a local version of the PR into your bun-47371 --bun |
|
Status. Reproduced with the script in the Notes of this pull request. On main, The branch has main at 30261e2. The pin is the preview build of the head of oven-sh/WebKit#941 ( On a debug and ASAN build of this tree: the 44 new tests of A review of oven-sh/WebKit#941 found that in a compiled executable the module of an A review on my side found a second fault of that kind, and 27493dc075 fixes it: what the run time hands to the loader inside a job that began behind a file ran under the store of that job (a job of The review threads of oven-sh/WebKit#941 are answered with runs. What they leave is named in that pull request, and this branch holds it as known wrong answers: five compiled test runs that are todo and three programs that are CI on 8d850ee passed: build 142797 ended with all 188 jobs passed. Three tests passed there on a retry. |
…o async context With oven-sh/WebKit#941 a load keeps the async context of its caller for its jobs. The context of a graph is in the same slot as the AsyncLocalStorage stores, and what runs in it ends with the graph. The jobs of a load run the host's hooks and the modules of the realm's own loader, and neither is a graph's. JSModuleGraph sets the loader's filter for what a load keeps: none where the caller is in the context of a graph, so those jobs run as before. Tests: a synchronous load, a top-level await and a Bun.ModuleGraph under AsyncLocalStorage stores, in plugins.test.ts and as programs of test/aot.
…b59d8fc79 main pins a7b59e61de now. The preview is that commit plus the commits of oven-sh/WebKit#941.
main pins 4d45755fed now. 1a52269eca is that commit plus the commits of oven-sh/WebKit#941.
… its call in a compiled executable A program for test/aot/parallel that runs with the transpiler on its own threads. Without the commit of oven-sh/WebKit#941 that keeps the async context in TextThatWaits, five of its ten lines differ from plain Bun.
…f its call in a compiled executable A program for test/aot/parallel. With oven-sh/WebKit#974 the second import() waits behind the first load and is made again in a job of that load. Without the commit of oven-sh/WebKit#941 that keeps the async context in ImportBehindLoad, three of its seven lines print the store of the first import().
main has the rewritten ahead-of-time compiler (#47942) and pins WebKit 5a61071f79. 328b6c9bb4 is that commit with the commits of oven-sh/WebKit#941 merged in.
…llected, and a compiled test run The driver runs every reference at default flags now, where plain Bun reads the texts of modules on its own threads. - Seven programs have a reference that is a race between those threads: they say what the executable prints. - Two programs call import() while a require() loads an ES module. The rewritten run time runs that import() after the require(), as Node.js does, so they say what the executable prints too. - New: a require() of a graph that awaits, whose CommonJS file calls import() under a store. The module behind the await runs under no store. - New: six import() calls under a store each, whose CommonJS files queue a job. A collection takes the stores when the calls are through. - New: an import() in a compiled test run (bun test --aot) reads the store of its call. - One program no longer sets BUN_FEATURE_FLAG_DISABLE_ASYNC_TRANSPILER, which the driver refuses.
test/aot/ci-lists.test.ts asks that every test file of test/aot is in test/expected-durations.json, and that a long one is in excludeFiles of test/parallel-allowlist.json.
… and evaluate() continues its caller The pin is oven-sh/WebKit#941 on the pin of main, with its last commit: the filter of what a load keeps is one pointer on the VM, and a host function that evaluates a module for its caller says so. - GlobalObject::resetAsyncContextAtEventLoop() is the statement that drainMicrotasks() has where a checkpoint begins. The host now also calls it where it begins a load with no script on the stack: a preload, the entry point, a test file, the built-in entry point, a Module.runMain that a preload replaced, a module of the bundler's server. What the script that ran last left in the slot with enterWith() is not the async context of that load. With the pin, the load would keep it for all of its jobs. - ModuleGraph.cpp sets VM::asyncContextOfLoadFilter. - vm.SourceTextModule#evaluate() continues its caller: the modules that wait there for a top-level await read what the caller reads. Tests: a preload that calls enterWith() (the entry point, a CommonJS entry point, a replaced Module.runMain, the next preload and its hooks, the entry point of a Worker, test files under three modes of bun test), an entry point that waits for a module that a preload began to import, node:vm, mock.module() for a linked module, a load outside any store, and three AOT programs.
The fixture gave the Worker the pathname of a file URL as its preload. On Windows that is /C:/..., which does not resolve, so the test of the Worker's preload failed there.
main has #48120, which pins WebKit at ee59053d32. WEBKIT_VERSION is now 37998c5702: oven-sh/WebKit#941 with that commit merged in. Conflicts: the pin, the end of plugins.test.ts (main's new tests, then the block of this branch), and the two lists of CI's runner, which take main's and get the test of this branch again (test/aot/ci-lists.test.ts).
main has #48436, which pins WebKit at 873ab3a426. WEBKIT_VERSION is now bacfec4b27: oven-sh/WebKit#941 with that commit merged in. Conflicts: the pin, and one list of CI's runner, which takes main's and gets the test of this branch again (test/aot/ci-lists.test.ts).
main has #48478, which pins WebKit at 264cf6db41. WEBKIT_VERSION is now 102d1739cb: oven-sh/WebKit#941 with that commit merged in. Conflict: the pin.
main has #48297, the upgrade to the upstream sync, which pins WebKit at 7f020d2818. WEBKIT_VERSION is now bb2e5abe7f: the head of oven-sh/WebKit#941, which is that commit with the change. Conflict: the pin.
main has #48617, which pins WebKit at 7101dff5ab. WEBKIT_VERSION is now fd9c7641af: the head of oven-sh/WebKit#941, which is that commit with the change. Conflicts: the pin, and the two lists of CI's runner, which take main's and get the test of this branch again (test/aot/ci-lists.test.ts).
main has #48677, which pins WebKit at 6fc709c298. WEBKIT_VERSION is now 5b6deeea52: the head of oven-sh/WebKit#941, which is that commit with the change. Conflicts: the pin, and one list of CI's runner, which takes main's and gets the test of this branch again (test/aot/ci-lists.test.ts).
main has #48714, which pins the preview build of oven-sh/WebKit#1042. That pull request is merged as b553971ea8. WEBKIT_VERSION is now the preview build of the head of oven-sh/WebKit#941 (1c4943f623), which is b553971ea8 with the change. Conflicts: the pin, and one list of CI's runner, which takes main's and gets the test of this branch again (test/aot/ci-lists.test.ts).
main has #48750: its pin is b553971ea8 itself now, which is oven-sh/WebKit#1042 as merged. WEBKIT_VERSION stays the preview build of the head of oven-sh/WebKit#941 (1c4943f623), which is b553971ea8 with the change. Conflict: the pin.
main has #48771: its pin is de043228c2, which is oven-sh/WebKit#1045 as merged. WEBKIT_VERSION is now the preview build of the head of oven-sh/WebKit#941 (4ee8e65bd8), which is de043228c2 with the change. Conflicts: the pin, and the end of test/parallel-allowlist.json (main's list, then the one test file of this change).
main still pins WebKit at de043228c2, so WEBKIT_VERSION stays the preview build of 4ee8e65bd8 (de043228c2 with the change of oven-sh/WebKit#941). Conflict: the imports of test/js/bun/plugin/plugins.test.ts. main's #48873 takes `autoInstallTempDir` from the harness there, and this branch takes `mock` from bun:test. Both stay.
main has #48815: its pin is c495a7e0e4, which is oven-sh/WebKit#1047 as merged. WEBKIT_VERSION is now the preview build of the head of oven-sh/WebKit#941 (5557a0f6ec), which is c495a7e0e4 with the change. Conflict: the pin.
main has #48926 and #48993: its pin is b00aff56c9, which is oven-sh/WebKit#1059 as merged. WEBKIT_VERSION is now the preview build of the head of oven-sh/WebKit#941 (ec0591031c), which is b00aff56c9 with the change. Conflict: the pin.
main has #49004: its pin is ed67c459b7, which is oven-sh/WebKit#1061 as merged. WEBKIT_VERSION is now the preview build of the head of oven-sh/WebKit#941 (31ae46ffc3), which is ed67c459b7 with the change. Conflicts: the pin, and test/parallel-allowlist.json, where main's list ends with six new files of test/aot. The file of this branch stays the last entry.
main has #49037: its pin is 9e983dfc1d, which is oven-sh/WebKit#1063 as merged. WEBKIT_VERSION is now the preview build of the head of oven-sh/WebKit#941 (2bc7c74156), which is 9e983dfc1d with the change. Conflict: the pin.
main has #49081: its pin is 3bc9421149, which is oven-sh/WebKit#1064 as merged. WEBKIT_VERSION is now the preview build of the head of oven-sh/WebKit#941 (7aa751c338), which is 3bc9421149 with the change. Conflict: the pin.
main has #49108: its pin is ae8978d033, which is oven-sh/WebKit#1067 as merged. WEBKIT_VERSION is now the preview build of the head of oven-sh/WebKit#941 (0d50be2a58), which is ae8978d033 with the change. Conflicts: the pin, and test/parallel-allowlist.json, where main and this branch each add entries at the end of the list. All three entries stay.
…er read no async context of that job In a compiled executable a CommonJS file of an imported graph runs ahead of the loader. Behind it, Modules::transferGraphToLoader() runs the jobs that were queued in front of the step of that file, inside the job that ran the file. On this branch a job of the loader enters the async context of its load, so those jobs ran inside that context too, and a job of a load that keeps no context enters nothing: the module of an import() with no store read the store of another import() (#941 (comment)). main does not have that, because no job of the loader enters anything there. Now those jobs read what the job found in the slot where it entered, and what they leave there is what the job puts back at its end, as if they had run in front of it: - ModuleLoaderJobAsyncContextScope is the scope of a job of the loader: of the twelve job functions in JSMicrotask.cpp. With ENABLE(AOT) the VM has the scopes of the jobs that entered, the innermost first (VM::m_asyncContextScopeOfModuleLoaderJob, and each scope has the one around it). Without ENABLE(AOT) the class adds nothing to ModuleLoaderAsyncContextScope. - Where the files of a load begin to run ahead, Modules notes the innermost job that the steps in front of the files are a part of too (EagerLoad::jobAroundStepsInFront). The job of a load that keeps a context runs the files, and those steps are no part of it. A graph that is handed to the loader begins files again for the same load inside the transfer, and takes the note of that load. - AOT::AsyncContextOutsideModuleLoaderJobsScope, around runStepsOfLoaderInFrontOf(), puts into the slot what the outermost job inside of the noted one found there, and takes what the steps leave as what that job puts back. runStepsOfLoaderInFrontOf() takes it by reference, so the call does not compile without it. The scope of a job that enters nothing writes nothing for this. The registry of evaluations (VM::m_moduleEvaluationAsyncContextScope) needs no change for the steps: a module with a top-level await begins in executeAsync() only, under a job function that has a scope of its own. An earlier commit of this branch put the context from before the files back at that call (asyncContextBeforeFiles), and merge 516122f dropped it. That one took what the slot held where the files began: the store of the load, where a job of the load runs the files. So it did not close this. It is what a load with no job around it needs (see below). Tests: - testaot module-programs/imports-in-flight-where-file-of-import-under-async-context-calls-import. Without the change its fourth line is `runs: third.js, under the context of the import`. - testaot module-programs/steps-inside-one-job-of-loader, job-of-loader-that-enters-nothing, steps-inside-jobs-of-loader-that-nest and steps-inside-one-job-of-loader-twice hold the chain and the scope on a bare VM. - Bun's test/aot has thirteen programs for it (oven-sh/bun#47371). Not in this commit: - A store that script holds around the call that hands the graph on, where no job of the loader entered: the steps in front read it. Bun's main has that too, and one program of oven-sh/bun#47371 marks it as a known wrong answer. - A call of import() that is made again where its text has come (startImportWhoseTextHasCome(), makeCallAgain()) enters its context with a plain AsyncContextSwapScope, and is no job here. Where it hands a required graph to the loader (transferSynchronousLoad()), the steps of that graph read the store of the call. That is this branch's own, and two more review threads are about the same scope (r4585961473, r4585961488): it gets a change of its own.
…he loader, and add programs for it WEBKIT_VERSION is now the preview build of a62dd4d93c, the head of oven-sh/WebKit#941: main's pin ae8978d033 with that change. Its last commit answers a review of it: in a compiled executable, the jobs that the run time runs behind a CommonJS file that ran ahead of the loader read no store of the import() whose job ran the file. Thirteen programs under test/aot/parallel/modules hold it. Twelve end with what the executable prints, in the order of time: the executable runs the file ahead of the loader, so the order is not that of `bun <file>`, and a module between two lines of the file is what each program is for. On the preview build of the head before that commit (0d50be2a58), eleven of the twelve fail in all four ways of the driver. The twelfth holds a rule that the fix keeps: a load with no store that a drain under a store takes reads that store. docs/compiler/compatibility.mdx now says so. The thirteenth is marked Todo. A store that script holds around the call that hands the graph on is no job of the loader, so the modules of other calls still read it. The executable of main prints the same wrong answer.
…e of the jobs that began behind the file In a compiled executable and in a compiled test run a CommonJS file of an imported graph runs ahead of the loader. Where something in the file makes the loader go on (a call of import(), a drain), Modules hands the graph of the file to the loader: the load of the file goes on (innerModuleLoading()), and the steps in front of the file run (runStepsOfLoaderInFrontOf(), and for a required graph transferSynchronousLoad()). The last commit made the steps in front of the file run outside of the job that ran the file. This one is about a job of the loader that began behind the file, inside it, and entered an async context: what Modules hands over there ran under that context. Two kinds of job do that. A job of node:vm. The file evaluates a module of node:vm under a store and runs the queue of its context, and the module calls import(). The load of the file went on inside that job: a hook of the host for the next module of a load with no store read the store of the job. Nothing has to wait for this one. A call of import() that Modules makes again where the text of its module has come (startImportWhoseTextHasCome(), makeCallAgain()). It goes on under the async context that it was made under, because the slot is how Modules hands that context to the load and to what continues the call. It entered that context with a plain AsyncContextSwapScope, so the VM did not have it among the jobs of the loader. A file can wait for the event loop (today: Bun.build() runs it while setup() of a plugin runs, and that is to go away, oven-sh/bun#33261), and the call is then made again inside the file: - the steps of a required graph in front of the file read the store of the call, and what they left in the slot ended with the call; - the load of the file went on under the store of the call, as above; - the async context says more than stores. While the factory of a mock runs, it says whose factory runs, and an import() of the mocked module from there gets the module itself. So in a compiled test run with no AsyncLocalStorage in it, a hook of the host that asked for a mocked module got the module, and the module ran. The executables and the compiled test runs of Bun's main show none of this: no job of the loader enters a context there. Now: - A call that is made again is a job of the loader, and has one scope: ModuleEvaluationAsyncContextScope with CallMadeAgain. It always enters the context of its call, and the VM has it. The comment at the class keeps the two halves of "always" apart: for a call with no context it is the rule, and in the drain of a synchronous load it is there for the slot alone. - EagerLoad::jobAroundFiles is the innermost job where the files began to run ahead. In transferGraphToLoader() the load goes on outside of the jobs that began later (AsyncContextOutsideModuleLoaderJobsScope with that job), and that scope ends before the scope of the steps in front begins: two scopes of one job come one after the other. - transferSynchronousLoad() has the scope around loadModule() and the steps, and sets the registry of evaluations aside for them, as a synchronous load does for itself. A module that began to evaluate among the steps (mock.module() evaluates a linked one) handed the context of the call to the modules that wait for it. runSynchronousStepsInFrontOf() takes the scope by reference, as runStepsOfLoaderInFrontOf() does, so the call does not compile without it. - fulfillQueuedFetchAfterJobs() registers its last job under no async context. That job makes a call again, and a reaction runs under what was current where it was registered: the call would find the context of its own caller in the slot, and the steps in front of a file would read it. The part about a call that is made again is a guard for the state in which a file waits, and it is small. It does not make the slot a sound carrier of the context of a call. What is left, all of it this branch's own unless it names main: - What a call that is made again runs at once reads the context of the call: the factory of a mock below it. Under `bun test` a job of the load runs that factory, and such a job enters nothing in the drain of require() and for a load with no context. So the factory reads the store of the call inside require() where `bun test` prints the store of the caller of require(). And it reads none for a call with no context where `bun test` prints what the slot holds: inside require() under a store, and in the turn in which a module entered a store and laid a mock anew (nothing waits there, and there is no require()). main's compiled run prints what `bun test` prints in these, unless the mock stood before the call. (#941 (comment)) - A call that is made again can wait behind the factory of a mock. It then goes on in a reaction of a promise (importBehindFactories()), which runs under the context of the call and is no job of the loader. Where that reaction runs inside a file of a required graph, the steps in front of the file read the store of the call: `outside held` where `bun test` and main print undefined. A drain of the jobs in the file is enough for it (bun:jsc), nothing has to wait. importByKeyBehindFactories(), resumeDelayedImport() and didFetchFactoryChainImport() are reactions of the same kind. - A call in the context of a module graph keeps what the filter of the host leaves, which is nothing. Where its mocks wait for its text, the factory of a mock below it reads neither the graph nor a store, as under `bun test`, and main's compiled run shows it both. Where the factory runs inside the call of import() itself, this branch shows it both, as main does. (#941 (comment)) - A transfer that begins inside another transfer of the same load takes the jobs as it finds them. The one program that reaches that does not end on main either. The change that closes the first two hands the context of a call to the load and to those reactions by value, so that a call that is made again enters as a job of the loader does. That changes how aot/runtime carries a context, and it is the owner's to want. Three answers of a compiled test run to "the module or its mock" change with this commit. One is the fault above: it is again what `bun test` and main say. One becomes what `bun test` says where main says the module, and only in a run that tracked async contexts before the call. In the third `bun test` says the module and main's compiled run says the mock: the last commit said the module through the fault above, and this commit says the mock again. Tests: - testaot module-programs/call-made-again-under-store-is-job-of-loader, call-made-again-without-store-is-job-of-loader, call-made-again-inside-synchronous-load-is-job-of-loader, steps-of-synchronous-load-inside-job-of-loader-and-call-made-again and load-goes-on-and-steps-run-inside-call-made-again hold the scopes on a bare VM. - Bun's test/aot has programs and compiled test runs for each fault above, and rows that are marked as known wrong answers for what is left (oven-sh/bun#47371). One program and one compiled test run that fail without this commit need no file that waits.
… to the loader inside a later job, and add tests for it The engine change made a job of the loader enter the store of its load. In a compiled executable and in a compiled test run, a CommonJS file of an imported graph runs ahead of the loader, and the run time hands the graph of the file to the loader where the file makes the loader go on. Where a job that entered a store began inside the file (a job of node:vm, or a call of import() that is made again where its text has come), what was handed over ran under that store: a plugin's hook for the next module, a package of a required graph, and a mocked module that a hook asks for got the module in place of its mock. Bun's main has none of that. The pinned commit fixes it. Programs (test/aot/parallel/modules), each fails with the engine before the fix unless it says so: - import-under-store-is-made-again-while-file-of-required-module-waits and import-without-store-is-made-again-while-file-of-required-module-waits: the steps of a required graph. - factory-of-mock-in-required-module-runs-where-import-under-store-is-made-again: a factory that runs where the loader is asked for the required graph again. - import-under-store-is-made-again-while-file-of-module-that-job-of-import-requires-waits: the steps read the store of the caller of require(), which is not what the queue had. file-of-module-that-job-of-import-under-store-requires-calls-import holds the same with no file that waits, and passes before the fix too. - load-of-file-without-store-goes-on-inside-job-of-vm-module-under-store: no file waits. - load-of-file-of-import-under-store-goes-on-inside-require-under-another-store: the load of an import() under a store goes on inside require() under another store, and a plugin's hook reads the store of the import(). It passes before the fix too. It holds what the scopes of the loader do while the run time has the queue of a synchronous load set aside (oven-sh/WebKit#941 (comment)). - import-under-store-is-made-again-where-factory-of-mock-calls-import and its twin without a store print the right lines but for the factory of a mock, which is a known wrong answer (// Todo). - import-under-store-waits-behind-factory-and-goes-on-inside-file-of-required-module is a known wrong answer as a whole (// Todo): the call goes on in a reaction of a promise. - The head of import-under-store-is-made-again-while-file-of-import-under-store-waits said that the loader runs the other jobs where the call is made again. It runs them where the file calls import(). Compiled test runs (test/aot/an-import-call-in-a-compiled-test-run-reads-its-store.test.ts): each project expects what bun test prints, and runs under bun test and bun test --aot. - Six fail under bun test --aot with the engine before the fix: a call made again inside require(), the same with a module that awaits among the steps, the same with a factory that calls require(), the load of a file that waits, the load of a file inside a job of node:vm (no file waits), and a mocked module that a hook asks for (no AsyncLocalStorage). - Two hold what a call in the context of a Bun.ModuleGraph keeps. - Five are known wrong answers of the compiled run, as test.todo with the reason beside each.
Fixes #32693
Draft until oven-sh/WebKit#941 merges:
WEBKIT_VERSIONis the preview build of its head. Supersedes #37933.Problem
import()underAsyncLocalStorage.run()loses the store.Bun.plugin()hooks of static dependencies printonLoad(b):undefined. So do module bodies. Bun 1.3.13 and Node.js read the store.runInternalMicrotask,JSMicrotask.cpp).Fix
resetAsyncContextAtEventLoop(): the host resets the slot where it begins a load (entry point, preload, test file).plugins.test.ts(44 new, 24 fail on main), 43test/aotprograms. Self-reviewed: 261 concerns raised, 238 addressed (23 open: Notes).Background
AsyncLocalStoragestores. A job captures it when scheduled.enterWith()in a module body leaves its store in the slot until the next checkpoint (AsyncLocalStorage: drop enterWith() frames at the event-loop boundary, stop allocating per await/then #41190).Module.runMaingoes past it) and a reset where a checkpoint ends (it changes what listeners read: Preserve AsyncLocalStorage context in unhandledRejection handlers #31721).Downsides
Bun.ModuleGraph(bun test --isolate=modules) animport()still loses the store.import()in abeforeExitlistener keeps a store that the entry point entered: the listener reads it since Bun 1.4.2.libJavaScriptCore.a: +9,595 bytes.Notes
Repro.
module.registerHooks)onResolve(a)onLoad(a)onResolve(b)onLoad(b)The program of #32693 (a module body under
store.run("CONTEXT", () => import(...))) printsCONTEXTon Bun 1.2.23 and 1.3.13,undefinedon 1.3.14 to 1.4.4, andCONTEXTwith this pin and on Node.js. The module-loader rewrite (#29393) is in that range. The range was not bisected.The pin.
autobuild-preview-pr-941-27493dc0is the preview build of the head of oven-sh/WebKit#941 (27493dc075): the pin of main (a8d825d8a0) with the change. The merged commit replaces the pin before this leaves draft. oven-sh/WebKit#941 has the design, the table of jobs and the measurements. The size is from the release build ofa8d825d8a0and the preview build of that head (bun-webkit-linux-amd64-aot,size -t): text 73,812,430 to 73,822,025 bytes.What Bun does besides the pin.
ModuleGraph.cppsetsVM::asyncContextOfLoadFilter: a load that begins in the context of a graph keeps nothing.GlobalObject::resetAsyncContextAtEventLoop()(ZigGlobalObject.cpp) is the statement that AsyncLocalStorage: drop enterWith() frames at the event-loop boundary, stop allocating per await/then #41190 put where a checkpoint begins: with no script on the stack, the slot gets the event loop's value. The host now also calls it where it begins a load by itself:JSModuleLoader__import(a preload),JSC__JSModuleLoader__loadAndEvaluateModuleandJSC__JSModuleLoader__resolveAndLoadAndEvaluateModule(the entry point, a test file, the entry point of a Worker, a--hotreload),Bun__loadBuiltinEntryPoint,NodeModuleModule__callOverriddenRunMain(aModule.runMainthat a preload replaced) andBakeLoadModuleByKey. While script is on the stack the call does nothing, so a load that script begins through one of them keeps the caller's context.drainMicrotasks()compiles to the same 108 instructions as on main, and a load that the host begins executes 14 more and a call (counted in the assembly ofZigGlobalObject.cppat the flags of a release build, without LTO).vm.SourceTextModule#evaluate()says that it evaluates for its caller (JSC::ModuleEvaluationAsyncContextScope,CalledByScript). The modules that wait there for a top-level await continue the call.Tests.
test/js/bun/plugin/plugins.test.ts, block "AsyncLocalStorage in what an import() loads", 44 tests. On main 24 fail (measured at 98cda0d, and again on the head of Upgrade WebKit to 8156ba68efab #48297, which is main at 89452d2 with the upstream sync). The other twenty are controls for what must stay as it is: the cells that a load allocates with and without a store, a load outside any store,require()of an ES module (six tests, one of them inside a graph), what code of aBun.ModuleGraphloads (three tests) and what the host loads behind a preload (nine tests). On a build of main'ssrcwith the engine change 18 fail (measured on main at 98cda0d with 37998c5702): thirteen of the fourteen tests of a preload,node:vm(three tests) and two of a graph. Seventeen fail on this pull request as it was at a60b510, before the host's rule moved to where a load begins. Four fail on main for the host's part alone, which needs no engine change: the hooks for the next preload and a replacedModule.runMain, each in both ways in which Bun reads a preload. The tests cover:onResolveandonLoadhooks and module bodies of static importsimport()from a timer and from a promise reactionModule.runMain()andimport.defer()build.module()factorythen()of a namespaceimport()enterWith()in the hook of a dependency, in the hook of the imported module itself, and in a CommonJS fileenterWith(): the entry point (--preload,--import,--require, a CommonJS entry point, a preload that replacesModule.runMain, a preload whose body runs behind a top-level await), the next preload and the hooks for it, the entry point of a Worker, and two test files underbun test,bun test --isolateandbun test --parallel=2vm.SourceTextModule#evaluate()underrun(), underexit()and plain: in a module of a loadingimport(), in the drain of arequire(), in the entry point, behind a second top-level await, and insidegraph.run()mock.module()for a linked module with a top-level await below itrequire()of an ES module: alone, with animport()that the module calls, with animport()in flight (two shapes), and with top-level awaits that the call begins or settlesBun.ModuleGraph: the host's hooks for a load that the host starts, a module of the host that a graph asks for and that outlives the graph, a hook of the host for a load that a graph starts,require()inside a graphmock.module()factorytest/js/node/async_hooks/async-context/async-context-dynamic-import.js:AsyncLocalStorage-tracking.test.tsruns each fixture of that folder on Bun and on Node.js. It fails on main.test/bundler/bundler_compile_prelinked.test.ts,DynamicImportAsyncLocalStorage: animport()of a chunk in abun build --compileexecutable, with and without--bytecodeand--splitting. The two variants with--splittingfail on main.test/aot/parallel/modules: twenty programs, forbun build --compile --aot. They pass in the driver (test/aot/run-parallel.ts) in each of its ways. At an earlier pin an executable of main and one of the change differed in 71 of the 143 lines that sixteen of them print.import()whose job ran the file. Twelve end with what the executable prints, in the order of time. With the engine before the fix (0d50be2a58), eleven of them fail in all four ways. The twelfth holds a rule that the fix keeps (docs/compiler/compatibility.mdxhas it). The thirteenth is marked Todo: a store that script holds around the call that hands the graph on. The executable of main prints the same wrong answer there.node:vm, or a call ofimport()that is made again while the file waits ([JSC] A job of the module loader runs under the async context of the import() that asked for it WebKit#941 has the account).require()with no file that waits. One holds a plugin's hook where the load of animport()under a store goes on insiderequire()under another store: the hook prints the store of theimport().bun <file>does: they fail before the fix too. The third is a call that goes on in a reaction of a promise.test/aot/an-import-call-in-a-compiled-test-run-reads-its-store.test.tshas nine projects now. Each expects whatbun testprints and runs underbun testandbun test --aot. With the engine before the fix six fail under--aot. One of them has noAsyncLocalStorage: a plugin's hook asks for a mocked module and got the module. Two hold what a call in the context of aBun.ModuleGraphkeeps. Five more projects are known wrong answers of the compiled run, astest.todowith the reason beside each.Bun.build(), which runs the event loop untilsetup()of a plugin has ended. That wait is to go away (event loop: never re-enter from inside a dispatch callback (de-block expect().resolves and HTMLRewriter.transform, then assert) #33261). Each test checks first that it waited, and says so if not. Of those that fail before the fix, one program and one project need no file that waits.import()while arequire()loads an ES module. A compiled executable runs thatimport()after the call, under the store of theimport(), as Node.js does (docs/compiler/compatibility.mdx). They say what the executable prints too.require-of-graph-that-awaits-under-store-file-calls-import: the module behind a top-level await of a required graph runs under no store, and not under the store of animport()that a CommonJS file of the graph called.imports-under-stores-files-queue-jobs-stores-are-collected: a collection takes the stores of six calls that are through, three fulfilled and three rejected. Before d6f315cb63 of [JSC] A job of the module loader runs under the async context of the import() that asked for it WebKit#941 the executable kept them.import-from-text-of-a-required-cjs-under-store-reads-its-exports:import()from text of a CommonJS file of the executable calls a getter and the traps of a Proxy under the store of the call, as plain Bun does.import-without-store-is-made-again-while-slot-holds-store: a call with no store that is made again when its text has come reads no store that its caller entered behind the call.import()under a store (test/aot/parallel-test-runs). On the branch of AOT: the next base #48120 it passed in six of six runs where the process had one processor, so that the compiler had one thread. With the default threads the debug build of the compiler stopped in its link stage in seven of eight runs, before the program ran ([JSC] A job of the module loader runs under the async context of the import() that asked for it WebKit#941 names the frames). In a debug build it would fail for a reason that is not its subject.test/aot/an-import-call-in-a-compiled-test-run-reads-its-store.test.ts:bun test,bun test --isolateandbun test --aoton one project. A compiled test run reads the text of what animport()names in a later turn, and the modules still read the store of the call. They fail on main.bun test --isolate=modulesis a todo there: a test file is a module of a graph.Other suites. On this head (main at 30261e2, the engine at 27493dc075), in a debug and ASAN build with the CI build of the pin: the 323 tests of
plugins.test.ts, 39 programs in the driver in its four ways (four more are marked Todo), the test of compiled test runs andtest/aot/ci-lists.test.tspass. One commit back (the same main, the engine at 0f5a1edd0d) the 323 tests, 32 programs, the test of a compiled test run andtest/aot/ci-lists.test.tspassed. One merge back (main at 130ff8a, the engine at a62dd4d93c) the same passed. One commit before that (the same main, the engine at 0d50be2a58) the 44 tests, the twenty programs, the test of a compiled test run andtest/aot/ci-lists.test.tspassed. Two merges back (main at 237bba5, the engine at 7aa751c338) the same four passed. Three merges back (main at 5f3f8fe, the engine at 2bc7c74156) the same four passed. Four merges back (main at 6f8424e, the engine at 31ae46ffc3) the same four passed. Five merges back (main at ab9b2eb, the engine at ec0591031c) the same four passed, and so did main's new tests for animport()of a file that throws (#48993): the twelve programs oftest/aot/parallel/import-calls-beside-a-require-that-throwsin the four ways, and the two stored test runs ofimport-calls-of-a-file-that-throws. Six merges back (main at 9098897, the engine at 5557a0f6ec) the first four passed, also on the head of #48926 (795f1a9) with the preview build of 58c7da9751. Seven merges back (main at a62cc28, the engine at 4ee8e65bd8) the same four passed. Eight merges back (main at d0f4fdd, the same engine), main's harness for stored test runs (test/aot/test-runs-in-folders-*.test.ts) ran the 443 projects of the 60 folders for animport()under a mock, or with a text that comes in a later turn: 441 passed, and the other two passed at a second run alone. In those two the compiled run had ended before its tests, as the debug build of the compiler does now and then in its link stage (oven-sh/WebKit#941 has the account and the sanitizer's report). Nine merges back (main at 8aa496e, pin 1c4943f623) the first four passed. Ten merges back (main at ccb3611, pin 5b6deeea52), main's harness for stored test runs (test/aot/test-runs-in-folders-*.test.ts) ran the 443 projects of the 60 folders for animport()under a mock, or with a text that comes in a later turn: 442 passed, two of them at a second run alone. In the other one the compiled run ended before its tests in 3 of 23 runs of the harness and passed in the other 20 (oven-sh/WebKit#941 has the account). Eleven merges back (main at 4385062, pin fd9c7641af), with five tests at once and a limit of 60 s, 3,140 tests passed:plugins.test.ts(323), the five files oftest/js/node/async_hooks(154), the nine oftest/js/node/vm(423),node-module-module.test.js(83),preload-test.test.js(8) and seven files oftest/js/bun/module-graph(2,149).bundler_compile_prelinked.test.tspassed 89 of 91: two builds in the first minute of the file ran over the limit, and they passed when they ran again.module-graph-compile.test.tspassed 76 of 76 with a limit of 180 s. The earlier heads passed the same lists in the same way: main at f336318 with pin bb2e5abe7f, main at 447ff87 with pin 102d1739cb, the head of #48297 before it merged, main at 89452d2 with pin bacfec4b27 and main at 98cda0d with pin 37998c5702. On a build of main'ssrcwith the engine change no other test of main failed for that reason: the 279 other tests ofplugins.test.ts,vm.test.ts,AsyncLocalStorage.test.ts,AsyncLocalStorage-tracking.test.ts,node-module-module.test.js,preload-test.test.jsand seven files oftest/js/bun/module-graphwere run (main at 98cda0d with 37998c5702). At the default of twenty tests at once many tests of these files run out of their 5 s limit in a debug build on this machine, with and without this change. A debug build of main stopped on this machine atBunPlugin.cpp:3562from #48677 to #49013 (JSC::SuspendExceptionScopewith no include ofFrameTracers.h), with and without this change: for the heads in between the local build had that include added, and the commits did not.What changes besides the fix.
Bun.ModuleGraph: where the host starts a load under a store (graph.import()), the host's hooks for the static dependencies read that store. A load that begins in the context of a graph keeps no context, so what code of a graph loads runs as before. The context of a graph is in the same slot as the stores, and what runs in it ends when the graph is disposed. A hook of the host and a module of the host's loader must not end with a graph: with a top-level await, such a module would never go on.JSModuleGraphsets that rule as the VM's filter. So inside a graph animport()under a store still loses the store, and a test file underbun test --isolate=modulesis a module of a graph.import()can start a dependency that has a top-level await. A module of another load that waits for that dependency runs in the store of the firstimport(). The other load can be the host's: an entry point or a test file that waits for a module that a preload began to import reads the store of thatimport(), and so does what it starts (a server's handlers, later timers). Node.js does the same. Bun 1.3.14 to 1.4.4 read none there.import()with a store, a module enters another store withenterWith(), and the next module of the graph has a top-level await. The modules that wait for it run under the entered store. Node.js does the same.enterWith()ends with the hook: the hooks behind it and the module bodies read the store of theimport(). Node.js and Bun 1.4.4 hand the entered store on to what the load runs next. Bun 1.3.13 handed it to the hooks behind it and not to the bodies. The hooks for the module thatimport()names run inside the call, so what they enter is what the load keeps.vm.SourceTextModule#evaluate()under a store: the modules that wait for a top-level await read the store of the call, also underrun(), behind a second top-level await, and in the entry point. Underexit()they read none. Node.js does the same. On main they read none in each case.Module.runMainthat a preload replaced, read no store that the preload before them entered. Bun read it where it reads a preload on the thread of the program: on main by default (bun test: Vitest compatibility #44833,is_maininsrc/runtime/jsc_hooks.rs), and in Bun 1.4.2 to 1.4.4 withBUN_FEATURE_FLAG_DISABLE_ASYNC_TRANSPILER=1and in a compiled executable whose preloads are embedded.What does not change.
unhandledRejectionlisteners see no store in Bun, for a failedimport()as for any rejection (AsyncLocalStorage context is not preserved whenunhandledRejectionis emitted #40223, AsyncLocalStorage store is not available inside process.on('unhandledRejection') listeners (Node restores it) #44650). Open work: Preserve AsyncLocalStorage context in unhandledRejection handlers #31721, [JSC] An unhandled rejection is reported in the async context of what scheduled the job that rejects WebKit#729.enterWith()outside anyrun()reaches the jobs of the same turn of the event loop, and not a job behind a timer (AsyncLocalStorage: drop enterWith() frames at the event-loop boundary, stop allocating per await/then #41190).enterWith()gives its store neither to the entry point, nor to the next preload, nor to a test file, as in a file run of every release of Bun (1.0.36 to 1.4.4).node --requiregives the store of a CommonJS preload to the entry point, andnode --importgives none. Bun waits for every preload asnode --importdoes. Nobody has asked for the answer ofnode --require. It is a maintainer's choice, and it would be made inload_preloads(src/runtime/jsc_hooks.rs).enterWith(): abeforeExitlistener, anunhandledRejectionlistener for a rejection of that turn, the first signal or message listener. Since Bun 1.4.2 it reads that store, and it does here. Animport()in such a callback keeps the store, as anyimport()keeps what its caller reads: that part is new. Preserve AsyncLocalStorage context in unhandledRejection handlers #31721 resets the slot where a checkpoint ends, together with the capture that gives anunhandledRejectionlistener what Node.js gives it. An earlier shape of this pull request had that reset alone. A review of it measured 34 readings of such listeners that change, 12 of them away from Node.js, so it is not here.mock.module(): where it evaluates a linked module early, the job of the loader that runs the calling module says what the modules behind a top-level await get, as before. An earlier shape continued the caller there, and handed a pending mock factory's frame to modules of another load.require()call runs, it runs in the context of its caller. That coversrequire()of an ES module and of any other file that goes through the module loader.Bun.build()plugin hooks: bundler: run Bun.build plugin hooks in the AsyncLocalStorage stores of the Bun.build() call #41562.Differences from Node.js that remain (Node.js v26.3.0, and v26.12.1, which this repository pins, for the rows of a preload and of
node:vm). The tests of the block were also run on Bun 1.3.13 and, ported tomodule.registerHooks, on Node.js.Older than 1.3.14 (Bun 1.3.13 does the same):
enterWith()ends with that job. Node.js runs the file with the module bodies, so the importing module sees the store.import()calls that load one module at the same time: the module runs in the store of one of them. In 8 runs it was the first call 4 times. Node.js: always the first. The order in which Bun hands transpiled modules to the loader decides (Hand transpiled modules back to the module loader in request order #42051).Since 1.3.14, and kept (Bun 1.3.13 does what Node.js does):
require()call loads and evaluates every module of the loader that becomes ready while it runs, inside the call: also modules of animport()in flight whose graph it completes, and of animport()that a required module calls. They read the caller's store, andvm.SourceTextModule#evaluate()in such a module hands that store on. Node.js evaluates them later, under the store of theirimport(). A module of such animport()that runs after the call, behind a top-level await, reads the store of itsimport(). Abun build --compile --aotexecutable runs animport()that a required module calls after the call, under the store of thatimport(), as Node.js does.New with this change:
enterWith()ends with the hook (see "What changes besides the fix").vm.SourceTextModule#evaluate()with nothing current at the call: the modules that wait read what the slot holds when their job runs, for example a store that the awaiting module entered. Node.js reads none there. (Not new: main reads the same.)The tests of the block that pin one of these outputs say in a comment what Node.js prints.
Review. The design, the rules and the measurements are in oven-sh/WebKit#941, with the account of each review. The last three reviews raised 98 concerns, and 86 are answered. What they changed here:
Module.runMainwent past it. The second answer was a reset where a checkpoint ends: it changed what listeners read in the entry point's first turn, Preserve AsyncLocalStorage context in unhandledRejection handlers #31721 has it with its other half, and a deferred task, the printer of a rejection and a--hotreload went past it. The reset is now where the host begins a load.evaluate()ofnode:vmcontinues its caller.mock.module()had the same scope for one review, and lost it.Not taken: a safeguard that lets the engine half merge alone (see "The order" in oven-sh/WebKit#941: the host's reset and its tests need no engine change and can go first). Three debug assertions of the loader fire in programs that nest
require()of ES modules, with noAsyncLocalStoragein the program. They fired on a debug build of the pin that main had then too (5a61071f79:JSModuleLoader.cpplines 869, 1147 and 1148).Reach. Module bodies have one report (#32693) and one downstream: manten skips three specs on Bun for it. With the skips removed, Bun 1.4.2 gives 117 pass and 3 fail, and the first commit of oven-sh/WebKit#941 gave 120 pass and 0 fail. One of the three specs alone still fails in 5 of 10 runs for the reason in #42051. The hooks, the factories and the CommonJS bodies have no report. They kept the store in Bun 1.3.13 and keep it in Node.js.
no test proof · iteration 10 · platform-specific test(s) that do not run on this machine, deferring to CI, which covers all platforms: test/js/bun/plugin/plugins.test.ts, test/bundler/bundler_compile_prelinked.test.ts