Repository navigation
Conversation
|
Updated 3:04 AM PT - Oct 22nd, 2026
❌ @robobun, your commit 264782d has 1 failures in
🧪 To try this PR locally: bunx bun-pr 46004That installs a local version of the PR into your bun-46004 --bun |
|
Status Reproduced on Bun 1.4.0, 1.4.3 and
With this PR both programs give Node's result. PR: #46004. It is a draft because it pins the preview build of oven-sh/WebKit#832. |
|
The engine change that oven-sh/WebKit#281 was for is now oven-sh/WebKit#845. oven-sh/WebKit#281 is closed. When |
|
#46129 fixes the crash that the Notes of this PR list under "Not fixed": a second
|
ae752b5 to
7038705
Compare
7038705 to
0597933
Compare
|
The comment check flagged 39 groups of comment lines in |
module.evaluate() called Evaluate() once for each dependency and then for the module. A legal import cycle failed with a TDZ ReferenceError, and a module that threw after a top-level await ended the process with an unhandled rejection that the program could not catch. evaluate() now calls JSC's Evaluate() once, and JSC walks the graph. A vm.SyntheticModule has a JSModuleRecord without code, and the moduleLoaderEvaluate hook runs its evaluateCallback where the walk reaches the record. A module reads its status and its error from its record. A vm run that a timeout or a SIGINT cuts short gives its error to the node:vm modules that it stopped. The checkpoint of an afterEvaluate context runs once, after the walk, and runs only the jobs of that context. The WebKit version is the preview build of oven-sh/WebKit#832: a job that finishes a module from inside a module body does not strand its importers and does not start them early.
0597933 to
264782d
Compare
|
Closing as stale: this PR conflicts with main, and its last commit is from 2026-10-22. The branch stays, so the work is not lost. To pick it up again, rebase the branch on main and reopen this PR. |
Problem
vm.SourceTextModuleimport cycle fails:ReferenceError: Cannot access 'x' before initialization.Vitest's vm pools hit it (arktype, configstore). A caught throw after a top-levelawaitexits with code 1.NodeVMModule::evaluateDependencies(src/jsc/bindings/NodeVMModule.cpp:262) calls Evaluate() once per dependency (node:vm: pass the Node v26.3.0 vm test suite (+25 tests, ~65% → 97%) #32018), and nothing handles those promises.Fix
evaluate()calls JSC's Evaluate() once, as Node calls V8's. Status and error come from the module record. A timeout or SIGINT gives its error to the modules it stopped.vm.SyntheticModulegets aJSModuleRecordwithout code. ThemoduleLoaderEvaluatehook runs itsevaluateCallbackin the walk.test/js/node/vm/vm-module-graph-evaluation.test.ts(Node v26.3.0 passes it) andvm-module-graph-evaluation-bun.test.ts. Alsotest/js/node/vm/,test/js/bun/resolve/, Node'stest-vm-*.Background
Downsides
script.runInThisContext()(3,557 to 3,569), +13 per ES module body, +564 per evaluatedvm.SyntheticModule. Bytes: +96 for that module, +704 of.text.node:vmchanges and three questions for a maintainer. Longest import chain: 15,960 to 10,630 modules.Errorkeeps its graph alive until [JSC] A module error does not keep the module record of its Evaluate() alive WebKit#840 (15 KB to 1,115 KB in one test). Not fixed: a module parked at anawaitwhen its cycle fails ([JSC] An evaluation error does not end a module body that is suspended at a top-level await WebKit#845).Notes
Questions for a maintainer.
vm.SyntheticModulewhoseevaluateCallbackthrew, and is evaluated later, rejects with that error and does not run. That is the rule of ECMA-262 for a module that failed, and the rule ofSyntheticModuleRecordin ec68522d83 ([JSC] FFI: compile C modules lowered to BIR with B3 WebKit#653). Node v26.3.0 andmainrun the importer with the exports that the callback set, and Node then reports a rejected promise of V8 that no program can reach. Keep the rule: yes or no?evaluate()stopped it. This PR keeps that for the module of the call, and gives the same error to every othernode:vmmodule that the termination stopped: a module on the stack of the walk, a module that a job of the checkpoint was running, a module of another graph that the same checkpoint resumed, the checkpoint of a Script, a deferred module that a Script reads first. The alternative for those is Node's state (below). Keep the extension: yes or no?Landing order. The JavaScriptCore half is oven-sh/WebKit#832 (+168 -5 in
CyclicModuleRecord.cppandAbstractModuleRecord.cpp, underUSE(BUN_JSC_ADDITIONS), with aJSTests/stressfile of 20 graphs).WEBKIT_VERSIONis its preview build, which cannot merge. When #832 has merged, the pin and the five plain ES module tests of this PR can land first as their own WebKit bump: they fix animport()that never settles and a segfault that need nonode:vm. This PR then rebases without the pin. oven-sh/WebKit#840 (a module error does not keep the module record alive) belongs in the same bump. #41654 (oven-sh/WebKit#670) and #44833 (oven-sh/WebKit#781) change the same line. If oven-sh/WebKit#781 merges first, the bump also takes its change.The regression. No user reported it. A fuzz run found it, and these two programs show it (Node needs
--experimental-vm-modules):Node v26.3.0, Bun 1.3.14 and this PR print
a sees 1. Bun 1.4.0, 1.4.3 andmainprintReferenceError: Cannot access 'x' before initialization.: they runabeforeb.Node and this PR print
caught: from band exit with code 0. Bun 1.4.0, 1.4.3 andmainprint the same line, thenerror: from b, and exit with code 1. Bun 1.3.14 prints nothing: it lost the error.Who it reaches. Vitest's vm pools make a
vm.SourceTextModulefor each file of a dependency. A test file that imports a package with a two-file import cycle, with Vitest 4.1.9 and--pool=vmForks: Bun 1.3.14, Node and this PR pass. Bun 1.4.0, 1.4.3 andmainfail withReferenceError: Cannot access 'helper' before initialization.(--pool=vmThreadsis the same on 1.4.x. Bun 1.3.14 does not start a worker of that pool.) A loader that does what those pools do was run over the ES module entry of 1,245 packages: 8 fail on 1.4.0, 1.4.3 andmain, and none on 1.3.14, Node and this PR. They are arktype, @ark/schema, @prisma/cli-engine, atomically, configstore, stubborn-fs, @solid-primitives/storage and alchemy. The errors includeTypeError: undefined is not an object (evaluating 'this.resolvedConfig.name')andCannot access 'Handlers' before initialization.A build with assertions could not run a Vitest vm pool until #46270 (ASSERTION FAILED: More attributes nodes than requests). With it the Vitest test of this PR also passes on a debug build, under the ASAN lane's environment.Considered.
a.evaluate()that nobody awaits), and the order stays wrong.main's WebKit (+13 -1 inNodeVMModule.cpp): the same mark, and the importer's record is set to EVALUATING by hand while its dependencies run as roots of their own, so a dependency that imports it back runs first.vm-module-graph-evaluation.test.ts(204 tests,main: 109 fail)vm-module-graph-evaluation-bun.test.ts(72 tests,main: 46 fail)main: 184, 184 and 161)The rows are release builds on ed68082, with the two test files as they were then (204 and 72 tests). 36, 36 and 40 of the patch's random graphs differ only by the text "errored mod", which one more line fixes. With the patch each dependency is still a root of its own Evaluate():
evaluate()of a member of a cycle fulfils while the cycle still waits at a top-levelawait, a member of a failed cycle that ran stays "evaluated", and status and error stay copies. The patch is not run on a debug build or on Node'stest-vm-*files yet. This PR deletes the lines that the patch changes, so the patch is not in its way.SyntheticModuleRecordfor theevaluateCallback([JSC] SyntheticModuleRecord runs the host's evaluation steps WebKit#796, closed, and ec68522d83 in [JSC] FFI: compile C modules lowered to BIR with B3 WebKit#653):SyntheticModuleRecordhas no status and no error thatnode:vmcould read, so the module would keep a copy of both. AJSModuleRecordwithout code has them.Random graphs.
test/js/node/vm/vm-module-graph-random-fixture.jsmakes a module graph from a seed (imports, cycles, top-level awaits, throws before and after the await,SyntheticModules), evaluates some of its modules, and prints the order, the statuses, the errors, eachevaluate()result and the unhandled rejections.scripts/vm-module-graph-random.tscompares a binary with thenodeon PATH. For seeds 0 to 11,999 in each kind of context (main, a context, anafterEvaluatecontext):mainIn 81 of the 110, Node ran the rest of a body that waited at a top-level
awaitwhen its cycle failed (the first bullet of "Not fixed"). In 41, a module imports aSyntheticModulewhoseevaluateCallbackthrew (question 1). 12 have both. Node aborts on a V8CHECKfor 39 of the 36,000 runs, where Bun gives a result. The generator has limits: a module never reads a binding, never callsevaluate()from its code, and the program handles every promise. The two test files have those shapes by hand.Changes that a
node:vmprogram can see. Each has a test. Where Node is not named, the new result is Node's.evaluate()of a graph with a module that throws after a top-levelawaitreports nothing that the program handled (the two programs above).await, its own or one of a module it imports, is "evaluated". A secondevaluate()gives the same pending promise. Onmainit was "evaluating" and the second call rejected withERR_VM_MODULE_STATUS.evaluate()of aSourceTextModulethat failed as the root of anevaluate()gives the promise of that call, so a later call that the program drops is not reported. Any other module with an error gives a new rejection for each call, as onmain. The promises are of the realm of the module. Onmainsome were of the main realm.afterEvaluatecontext, the jobs that the bodies queue run in one checkpoint after the walk, and a top-levelawaitcan finish inside thatevaluate().maindid a checkpoint after each dependency. That checkpoint runs only the jobs of the context. Onmainit could also run theprocess.nextTickqueue and the jobs of the main queue.evaluate({ timeout })can reject withERR_SCRIPT_EXECUTION_TIMEOUTwhile its module is "evaluated": the deadline passed in a plain job of the checkpoint, or after the last body. The nextevaluate()fulfils. Onmainthe module became "errored".import deferin a vm module is lazy, as the proposal says. The module runs at the first read of its namespace.mainran it with the importer. Node v26.3.0 does not parseimport defer.evaluateCallbackruns in theAsyncLocalStoragecontext of theevaluate()call.mainran it in the context of theSyntheticModuleconstructor call.link()that fails in the instantiation, the dependencies that linked are "linked" and can be evaluated.mainleft every module "unlinked".link()that ends after an importer linked the module resolves and changes nothing.mainrejected it withERR_VM_MODULE_STATUS.ERR_VM_MODULE_LINK_FAILUREends with "errored module". It ended with "errored mod".RangeErrorwhile the modules behind the limit stay "linked". Onmainthe limit is about 15,960, a longer chain gives aRangeErrorthat marks every module, and 30,000 modules end in a segfault. The walk is now JSC's, the one that plain ES modules use.Differences from Node that stay. Each has a test in
vm-module-graph-evaluation-bun.test.ts.null, andevaluate()of the module the call was on never settles. A module that a job resumed after a top-levelawaitstays "evaluated", and itsevaluate()never settles. When a job started the body, Node aborts on a V8CHECK(Check failed: !is_null().).evaluate()never settles. The first is a module that waits at a top-levelawaitwhen the termination drops the job that was to resume it: JSC clears the queue and does not say which jobs it dropped. The module of the call gets the error (question 2), a module that it imports does not. The second is a module of the main context, or of a context without its own queue, that a job of the main queue was running inside the run:node:vmstarts no checkpoint of that queue, so it does not see one that is stopped.evaluate()does not settle.awaitthat evaluates a root that imports its own module: the root runs at once. JSC takes a module on the stack of an outer walk as a dependency that is done (depInOuterSCC, whichrequire()of an ES module needs). In Node the root waits for the module.maindoes the same as this PR.evaluate()of a member of a cycle that ran as a dependency, after the cycle failed, rejects with the error of the cycle. Node aborts on a V8CHECK. So does anevaluateCallbackthat evaluates a module that imports its ownSyntheticModule, which Bun runs.initializeImportMetais called before the body of its module, as onmain. Node calls it when the code first evaluatesimport.meta.Not fixed.
awaitwhen a later member of its cycle throws, or is stopped, does not run the rest of its body. JSC gives up the code of a module that gets an error. Plain ES modules do the same. [JSC] An evaluation error does not end a module body that is suspended at a top-level await WebKit#845 is the open engine change for that (it follows JSModuleRecord::evaluate: resume suspended TLA body when a cycle sibling throws WebKit#281). WhenWEBKIT_VERSIONhas it, the tests ofvm-module-graph-evaluation-bun.test.tsthat pin this result change.import()that a job starts inside the walk, for a module on the stack of that walk, never settles. It does not onmaineither. [JSC] A job that finishes a module from inside a module body does not strand or start its importers early WebKit#832 does not cover it. It is a separate bug.main: a laterimport()of it rejects with the string "JavaScript execution terminated.". Onlynode:vmmodules get the error of the run. Anode:vmmodule that throws that very value as a dependency can get the error of a later run that is cut short. Both are separate bugs.import.metaof a module without a context has Bun's properties (url,require,resolve,env), where Node gives an empty object. node:vm: give SourceTextModule an empty import.meta by default #33342 is the open PR for that.awaitdoes not run in theAsyncLocalStoragecontext ofevaluate(), as onmain. JSC: evaluate dynamically imported modules under the importer's async context WebKit#274 (AsyncLocalStorage: evaluate dynamically imported modules under the importer's store (WebKit pin bump) #37933) has that engine change.evaluate()pipes the promise of anafterEvaluatecontext into one of the caller's realm. JSC: route promise jobs to the handler/then realm's microtask queue WebKit#277 is the open engine change.How a run that was cut short finds the modules.
NodeVMRunTermination::finish()of the run that the termination belongs to callsNodeVMModule::errorModulesCutShortwith the error that it throws. JSC keeps no list of the records that a termination stopped, and the walk that a deferred namespace starts has no caller in Bun. So that function looks at the module records in the heap. A record that a walk had on its stack has JSC's termination value as its error, no cycle root, and a promise that is pending or absent. A record that a job was running is found by its state, in a context whose checkpoint the termination unwound. Only a run that was cut short pays for that, and only in a VM that made anode:vmmodule.What Bun keeps after this PR. Bun calls
asyncExecutionRejected,asyncExecutionFulfilledandsetEvaluationErrorof JSC for the modules of a run that was cut short (NodeVMModule.cpp), which no code of Bun did before. The record of aSyntheticModuleis aJSModuleRecordwithout code, set to LINKED by hand. The two new test files take about 70 to 100 s on a debug build.Measured costs. Release builds on linux-x64:
mainat ed68082 against thesrc/of this PR on that commit, before the rebase. An instruction count is the user-space instructions of the main thread (perf_event_open), as the slope between N and 2N operations, withBUN_JSC_useConcurrentJIT=0,BUN_JSC_useConcurrentGC=0,BUN_JSC_numberOfGCMarkers=1,BUN_GC_TIMER_DISABLE=1and no ASLR. Two runs agree to 0.1 percent. A line with "gdb" is a single-step count.Zig::GlobalObject::moduleLoaderEvaluatetoJSModuleRecord::evaluate(the hook asks whethernode:vmmade the module), and 218 to 221 ininnerModuleEvaluationbefore the body of a leaf module (the WebKit change).script.runInThisContext(): 3,557 to 3,569 instructions for each run.script.runInContext(): 3,682 to 3,694. Of the 12, 9 are inNodeVMRunTermination: its constructor,finish()and the code after it (gdb). Its size on the stack: 72 to 176 bytes.new vm.SyntheticModule(["a"], fn): 2,594 to 2,522. Withevaluate(): 6,549 to 7,113. Heap for each evaluated module: 834 to 930 bytes.new vm.SourceTextModule("")withlink(): 18,376 to 18,162. Withevaluate(): 22,688 to 22,385.SyntheticModules, and 519,790 to 502,712 forSourceTextModules.evaluate()of a root with 1,000 dependencies makes: 1,001 to 1 (2,002 to 2 in anafterEvaluatecontext).main, and the same with this PR in a VM that never made a vm module. In a VM that made one: about 320,000, plus about 75 for each module record in the heap (322,000, 426,000, 471,000 and 1,066,000 for 1, 1,001, 3,001 and 10,001 records).Errorof a graph that failed: 15 KB to 1,115 KB of heap for each kept error, for a graph of 22 modules with 100 KB each. A context whoseSyntheticModuleroot threw a hostError: 0 of 20 to 20 of 20 stay alive while the errors are kept. With [JSC] A module error does not keep the module record of its Evaluate() alive WebKit#840: 18 KB, and 0 of 20.sizeof:NodeVMModule104 to 80,NodeVMSourceTextModule168 to 136,NodeVMSyntheticModule128 to 112,JSVMClientData6,248 to 6,256.NodeVMScriptFetcher,NodeVMGlobalObjectandZig::GlobalObjectare unchanged..text+704 bytes,.bun_builtins-577 bytes..rodata,.dataand.bssare unchanged. The functions of the WebKit change are 282 bytes smaller in total: a heap replaces a sort.Tests.
vm-module-graph-evaluation.test.tsusesnode:test, and Node runs the identical file: 203 pass. Bun: 204 pass (one more test starts Node on the file). With thesrc/ofmainand this PR's WebKit, on a debug build: 109 of 204 fail.vm-module-graph-evaluation-bun.test.ts: 73 tests. A debug build skips 4 of them: two long chains, and the two Vitest runs, which need 30 to 40 s each there. With thesrc/ofmain: 38 fail. The code that a timeout must stop runs in child processes (vm-module-graph-cut-short-fixture.js), and every scenario tries again until its deadline passed where the scenario needs it. The file is out of the parallel batch of the test runner (test/parallel-allowlist.json): its children need a processor. It also has the plain ES module tests for the WebKit change, which hang or segfault onmain.sourcetextmodule-link-gc.test.tsno longer accepts the unhandled rejection of this bug. With thesrc/ofmain: 1 of 3 fails.todoon Windows: Vitest's vm pools do not find Vitest's own packages there (Cannot find package '@vitest/spy', import.meta.resolve(specifier, parent) fails for bare package names #40931). The random-graph tests have the digests of 12 seeds for each kind of context.test/js/third_party/vitest/vitest.test.ts), the twocollectContinuouslyrows (30 s) and Node's run of the whole file (30 s).test/js/node/vm/(vm.test.ts: 337 pass),test/js/bun/resolve/(421 pass) and Node'stest-vm-*files (98 of 98).test/js/node/vm/: all pass but two files that need more than the default 5 s there, and do not use the code of this PR to need it:script-leak.test.ts(a loop of 35 s) and fourcollectContinuouslytests ofvm-script-fetcher-leak.test.ts.test/js/bun/resolve/: 407 pass. Node'stest-vm-*files: 98 of 98 pass.test/js/bun/module-graph/: 2,150 pass and 51 fail, inmodule-graph-workers.test.ts(48, file descriptors of a Worker that was terminated),module-graph-isolation.test.ts(2) andmodule-graph-compile.test.ts(1, a hook that times out). Those three files pass on a release build of this PR and ofmain.BUN_JSC_validateExceptionChecks=1.vm-module-graph-evaluation.test.tsin the batch form of the test runner (--parallel=7on 8 processors, the environment of the ASAN lane, 54 files): 204 pass in 2 of 2 rounds, one of them beside 8 busy loops.Self-review. Four of the 37 concerns are open.
evaluate()from their code: not built. The two test files have those shapes by hand.Other PRs. The rebase onto f2de590 keeps what #46270 added to
NodeVMSourceTextModule.cpp: after a failedLink(),replaceUnlinkedRecords()gives the modules that did not link new records, which fixes the crash of a secondlink()that these Notes listed as not fixed. Here it walks the modules ofm_linkedModules, which this PR has in the place ofm_resolveCache. #46129 is an open PR for the same crash. This PR replaces #40939, which can close when this one merges. Earlier attempts that this PR does again: #33346, #33562, #33851 and the static half of #33853 (all closed). #45551 (draft) makes the same change toNodeVMGlobalObject::drainOwnMicrotasksand edits the same two hook functions inZigGlobalObject.cpp: the one that lands second must keep themoduleRecordOfNodeVMbranch.no test proof · iteration 2 · platform-specific test(s) that do not run on this machine, deferring to CI, which covers all platforms: test/js/node/vm/vm-module-graph-evaluation-bun.test.ts, test/js/node/vm/sourcetextmodule-link-gc.test.ts