Repository navigation
Conversation
|
No actionable comments were generated in the recent review. Walkthrough
ChangesGlob callback flow
Suggested reviewers: Merge Risk:
|
|
Updated 4:02 PM PT - Nov 13th, 2026
❌ @robobun, your commit 3e169b5 has 1 failures in
🧪 To try this PR locally: bunx bun-pr 48788That installs a local version of the PR into your bun-48788 --bun |
|
Status: reproduced on main ( const fs = require("node:fs");
process.nextTick(() => {
const log = [];
fs.glob("*.cjs", { cwd: __dirname }, (err, res) => {
log.push("cb");
process.nextTick(() => log.push("tick"));
queueMicrotask(() => log.push("job"));
setImmediate(() => console.log(log.join(" ")));
});
});
The second problem, which the review of this PR found, in a file named const fs = require("node:fs");
Array.from = () => ["replaced"];
console.log(fs.globSync("*.cjs", { cwd: __dirname }).join());
CI: two Buildkite runs of the same tree. No test of this PR fails on either run, and each of the 188 jobs passed on at least one of them.
|
The record that internal/fs/glob hands out is now { globAll, globSync, Glob }. The inspector test finds that record in a heap snapshot by the names of its keys, so it names globAll in place of glob.
There was a problem hiding this comment.
Nothing blocking. The comments below are optional suggestions. There is no need to push a fix for them before merging.
Beyond the inline note on globSync, I also checked that nothing still imports the removed glob export of internal/fs/glob (fs.promises.glob calls Glob#glob() directly), that settleCallback keeps its rm/rmdir/Dir.read callers, and that guardCallback returns the callback's value, so a promise returned from the callback flows into then() as the description says. I did not run the AOT header-hash test (test/internal/second-texts-of-modules.test.ts) or the new fs.test.ts cases; those were checked by reading only.
Extended reasoning...
The diff changes how fs.glob invokes its callback in src/js/node/fs.ts and its AOT twin, adds Glob#globAll in src/js/internal/fs/glob.ts, and adds ordering/throw/returned-promise tests; it touches no security-sensitive surface. A human look is still worthwhile because it changes user-observable callback timing and where a callback throw is reported, a tradeoff the description itself acknowledges.
Additional findings (outside the current diff — GitHub can't attach inline comments there):
-
🟣
src/js/node/fs.ts— Pre-existing, sync twin left out: a program that replaces Array.from still changes what fs.globSync returns, while fs.glob is now shielded from a replaced Array.fromAsync. src/js/node/fs.ts:1259 wraps an array that is already an array in the user-replaceable Array.from, and Glob#globSync at src/js/internal/fs/glob.ts:454 builds its result with it too. Fix: build the globSync result without user-overridable machinery at every site, e.g. collect with $arrayPush in Glob#globSync as globAll does and return that array directly from fs.globSync, then refresh the AOT header hash. Same pattern at 3 sites (src/js/node/fs.ts:1259, src/js/ahead-of-time/node/fs.ts:1346, src/js/internal/fs/glob.ts:454).Why this was flagged
A program assigns Array.from (a polyfill, a test spy, or the replacement the AOT suite's deeds fixture style performs) before calling fs.globSync. fs.globSync at src/js/node/fs.ts:1259 and src/js/ahead-of-time/node/fs.ts:1346 calls Array.from on the array that Glob#globSync already returned, and Glob#globSync at src/js/internal/fs/glob.ts:454 calls Array.from over its ResultSet with the withFileTypes map function. The replaced function decides what the caller receives: a throwing replacement makes fs.globSync throw, a wrong one returns wrong matches. The base branch behaves the same for globSync; this PR removes the equivalent exposure only from the async path (Glob#globAll at src/js/internal/fs/glob.ts:764 uses $arrayPush and the new test 'the program replaced Array.fromAsync before node:fs loaded' guards it), so the sync twin is the one site of the class left unfixed. No safeguard applies: the ensureCallback guard only wraps the async callback, and nothing captures Array.from at module load.
Verification: pre-existing — the base already reaches fs.globSync through the same route; this PR fixes only the async twin. Triggering condition: a program assigns
Array.frombefore callingfs.globSync. src/js/node/fs.ts:1259 re-wraps withArray.froman arrayGlob#globSync(src/js/internal/fs/glob.ts:454) built withArray.from. A replacement decides whatfs.globSyncreturns (or throws).
Each call of fs.globSync called the public Array.from two times. Glob#globSync built the result with it, from a Set and with a bound mapper. fs.globSync copied that array with it. A program that replaces Array.from, Function.prototype.bind or the iterator of Set decided the result. Node uses its own copies and a SafeSet. Glob#globSync now pushes the matches from the private $forEach of the set, and fs.globSync returns that array. Both texts of node:fs change.
…lob-callback-promise-job
|
The
Main ( |
There was a problem hiding this comment.
Nothing blocking. The comments below are optional suggestions. There is no need to push a fix for them before merging.
Beyond the inline finding, I checked the second-text sync for the latest push: the SHA-256 recorded in the src/js/ahead-of-time/node/fs.ts header matches the current src/js/node/fs.ts (computed locally), and the glob/globSync edits in both texts are identical. I also confirmed the removed glob export of internal/fs/glob has no remaining callers (fs.promises.glob uses Glob#glob() directly), and that settleCallback/validateFunction keep their other users in node:fs, so nothing is left dead. I did not run the test suite or the AOT compile.
Extended reasoning...
The change rewires fs.glob's callback to be the promise reaction (guarded by ensureCallback) and makes Glob#globSync/globAll gather results with $forEach/$arrayPush instead of Array.from/Array.fromAsync, in both the plain and ahead-of-time texts of node:fs, with tests for ordering, throw handling and replaced built-ins. It touches no security-sensitive surface. The one surviving finding is a pre-existing null-options validation gap posted inline; the second-text hash check and dead-code audit named in the body are the new items ruled out in this run.
Problem
process.nextTickqueued by thefs.globcallback runs before its microtask:cb tick job. Node v26.3.0 printscb job tick.glob(src/js/node/fs.ts:1254) sends the callback throughprocess.nextTick. Node's callback is the promise reaction.fs.globSyncbuilds its result with 2Array.fromcalls (src/js/internal/fs/glob.ts:454,src/js/node/fs.ts:1258), which a program can replace. No user reported either problem.Fix
globwraps the callback withensureCallbackand gives$thenNode's two handlers, in both texts ofnode:fs.Glob#globAll()is new, as on Node's main branch.Glob#globSync()gathers with$forEach, andfs.globSyncreturns that array.settleCallbackusers keep the hop, as in Node. Self-reviewed twice: 36 concerns raised, 34 addressed, 2 rejected (Notes).fs.test.ts(8 of 10 new tests fail on main),glob.test.ts(3 of 3), AOT runs, Node'stest-fs-glob*.mjs.Background
process.nextTickticks, then microtasks. A promise reaction is a microtask, so its microtasks run first.bun build --compile --aotcompiles the second text,src/js/ahead-of-time/node/fs.ts.$thenand$forEachare the engine's private copies of built-ins: no program can replace them.then()'s promise, and by default Bun tells nouncaughtExceptionlistener (A default-mode unhandled rejection does not reach anuncaughtExceptionlistener #44019).Downsides
fs.globSyncruns 1.4% more instructions over 100 matches. A kept result of 1,004 matches holds 3,200 bytes more. Binary text: +349 bytes.Notes
Repro
process.nextTickcb job tickcb job tickcb job tickcb tick jobd0f4fdd65fb)cb tick jobcb tick jobcb job tickcb job tickNo issue reports this. It came out of a comparison with Node in the work on #48713, which lists it as separate work. It rests on the Node compatibility rule that callback timing is contract, as #43728 did. Only the callback form changes:
util.promisify(fs.glob)andfs.promises.globprint the same on Node, on main and with this PR.The second column of Bun 1.3.14 is another defect: with no tick queue yet, the first tick that a job queues runs before the jobs that wait. #33088 and #43366 are open for it. It does not reach
fs.globafter this PR, becausenode:fsreadsprocess.nextTickwhen it loads and that read creates the queue. A new test holds that.What the order decides
Node v26.3.0 and this PR print
saw close. Bun 1.4.4 and main never do: the tick runs before the job, so the listener is late.Node's code (
lib/fs.js#L3207-L3214)Node's main branch has
new Glob(pattern, options).globAll()in place of theArrayFromAsync(...)line.History
fs.glob,fs.globSync, andfs.promises.glob#16676 addedfs.globwith.then(result => callback(null, result)).catch(callback). A throw of the callback called it again with the thrown error.process.nextTickhop, so that the throw is an uncaught exception. The order changed with it, in 1.4.0.settleCallbackand keptglobon it.Order against Node
One script prints 12 lines: both overloads, a list of patterns,
withFileTypes, a walk that fails, nested calls, calls from a timer, an immediate, a job and an fs callback, and a call with no pattern. It runs as CommonJS, as ESM with the default import, as ESM with named imports, and as abun build --compile --aotexecutable. Node v26.3.0 and the Node nightly print the same 12 lines.The other users of
settleCallbackThe repro with each API in place of
fs.glob:globcb job tickcb tick jobcb job tickcp,rm, recursiverm,opendir,Dir.read,Dir.read(buffered),Dir.closecb tick jobcb tick jobcb tick jobThe 43 rows of the existing order table in
fs.test.tspass without a change.A callback that throws
The callback queues a tick and a job, then throws. The line lists what runs, in order.
uncaughtExceptioncb job tick uncaughtException(unhandledRejection)cb uncaughtException(uncaughtException) tick jobcb uncaughtException(uncaughtException) job tickuncaughtExceptionandunhandledRejectioncb job tick unhandledRejectioncb uncaughtException(uncaughtException) tick jobcb uncaughtException(uncaughtException) job tickcb job tick, exit code 1cb tick job, exit code 1cb job tick, exit code 1Without the guard, on a debug build of this branch: the two throw tests fail with exit code 1 and no call of the listener, and
test-fs-glob-throw.mjsfails withMismatched <anonymous> function calls. Expected exactly 1, actual 0. Bun prints an unhandled rejection and sets the exit code. It callsuncaughtExceptionlisteners only with--unhandled-rejections=throworstrict. #44019 asked for the default to change and is closed as not planned.What the report of the throw shows:
fs.globfs.globfs.statthrow new Error(),throw "a string",throw anErrorCreatedEarlierbun testnames the throw line, for a failedexpectand the same three throwsDebugger.pausedevents withsetPauseOnExceptionsuncaughtA job of its own for the callback (
queueMicrotask(() => callback(...))) needs no guard. It was not used. The callback then runs one job later than in Node. And in Bun anuncaughtExceptionlistener reads noAsyncLocalStoragestore for a throw fromqueueMicrotask. For a throw of thefs.globcallback it reads the store of the caller, on Node, on main and with this PR. The throw tests hold that.A callback that returns a promise
Node's first handler returns what the callback returns, so the promise of
then()takes the state of a promise that the callback returns. This PR does the same. Main ignores the return value.unhandledRejectionlistenerArray.fromAsyncMain calls
Array.fromAsyncwhenfs.globruns, so a program that replaced it changesfs.glob. Node v26.3.0 takes it from its primordials and Node's main branch does not use it. WithGlob#globAll()this PR does not use it either:fs.globcalls back withArray.fromAsyncreplaced by a function that throws, before or afternode:fsloads.fs.globSyncThe automatic review of this PR found the sync form of the
Array.fromAsyncpoint. It marked the finding as older than this PR and as optional. No user reported it. On main eachfs.globSynccall does this:Glob#globSync(src/js/internal/fs/glob.ts:454) builds the result withArray.from(this.#results, mapper).#resultsis an instance of a subclass ofSet. WithwithFileTypesthe mapper comes fromstatForFileTypes.bind(...).fs.globSync(src/js/node/fs.ts:1258, and line 1345 of the second text) copies that array with a secondArray.from. The copy is left from feat(node/fs): addfs.glob,fs.globSync, andfs.promises.glob#16676, whereglobSyncwas a generator. Since fs: port Node.js v26.3.0 fs tests and fix the gaps they surface — cp error semantics, watcher event delivery, watch ignore+AbortSignal, FileHandle pull/writer, glob port, opendir/Dir, mkdtempDisposable, rmdir-recursive end-of-life, mock.fn (+119 tests) #31830Glob#globSyncreturns an array.Node v26.3.0 has
return new Glob(pattern, options).globSync(), and itsGlob#globSyncgathers from aSafeSetwith the primordialArrayFrom. Node's main branch has a native walk (nodejs/node#65392) and builds theDirents in an index loop.The table shows what
fs.globSync("*.txt", { cwd })returns overa.txtandb.txtwhile the program has one built-in replaced. Each cell gives the result as strings, then the names withwithFileTypes.Array.from, by a function that returns["replaced"]replaced, thenreplacedFunction.prototype.bindreplacedtwiceSet.prototype[Symbol.iterator], for a set that is not a plainSetreplaced, thennullArray.prototype[Symbol.iterator], for the array of the matchesreplaced, then the matchesNow
Glob#globSyncpushes each match from the private$forEachof the set, with a named function and no bound one. WithwithFileTypesan index loop then puts eachDirentin the place of its path.fs.globSyncreturns that array in both texts.Costs for each
fs.globSynccall, on the same builds as the table of costs below:Array.fromFunction.prototype.bind, withwithFileTypesBUN_JSC_useGC=0,heapStats)withFileTypesBUN_JSC_useJIT=0, no patternBUN_JSC_useJIT=0, 100 matcheswithFileTypesObjectcells that main allocates in the first calls are the iterator results ofArray.fromover the set. The optimizing JIT removes them on main too. After 3,000 calls the difference is the one array of the copy, and the bound function withwithFileTypes.$forEachcalls a function for each match, which costs about 300 instructions for each match in the interpreter. Afor...ofover the set costs less than main'sArray.from(2,248,240 to 2,218,525 instructions on the earlier base). But it readsSet.prototype[Symbol.iterator], and the third row of the first table then printsreplaced.Array.frommade a copy of exactly the length. The bytes are the heap size for each of 300 kept results, after a full collection.Limits of this part:
glob.tsstill calls public built-ins, for exampleArray.prototype.push,Set.prototype.addandfor...ofover aMap. Node v26's text uses primordials there. Only the step that gathers the result changed.node/fs.ts, its second text,internal/fs/glob.ts) readsArray.fromorArray.fromAsyncnow. Main has 5 such lines there. See "Not in this PR" for the 21 lines elsewhere insrc/js.Costs, on release builds of main (
d0f4fdd65fb, merged into this branch) and of this PR, Linux x64fs.glob([], cb)call (N = 20,000,bun:jscheapStats)TickObjectthatasync_hookssees per 100 completions, returning and throwingrequire("node:fs")size bun: textsize bun: data, bssbunon diskBUN_JSC_useJIT=0The instruction count is the slope between 200 and 1,200 chained
fs.glob([], cb)completions, two runs of each.perfandvalgrindare not installed here. The count comes fromqemu-x86_64 -one-insn-per-tb -d exec,nochain, with the lines of the main thread counted. The two runs of one build differ by 24 instructions per completion on main and by 3 with this PR. Thefs.globSynccounts above are the slope between 40 and 120 calls, one run of each. With no pattern the slope is between 200 and 1,200 calls.Clauses of the fix, and what fails without each
Each line is one edit to a build of this branch.
src/: 8 of the 10 Bun tests of the new describe fail. The 3 new tests ofglob.test.tsfail too. The 2 that pass are the throw tests.validateFunctionin place ofensureCallback: the 2 throw tests fail, andtest-fs-glob-throw.mjsfails withMismatched <anonymous> function calls. Expected exactly 1, actual 0.Array.fromAsync(this.glob())as the body ofglobAll(): theArray.fromAsynctest fails witherror: the replaced Array.fromAsync ran.test/aot/run-parallel.tsfails the new program. The executable printsfulfilled: file.txt, tick, microtaskwhere plain Bun printsfulfilled: file.txt, microtask, tick.for...ofover the set in place of$forEach: thefs.globSyncscript offs.test.tsfails. It printsreplaced.Also different after this PR
fs.globwith a callback that is not a function names the argument"cb", as Node does. Main names it"callback".process.nextTickthat holds every tick, put in place beforenode:fsloads, no longer holds thefs.globcallback (table below).async_hookssees noTickObjectfor anfs.globcompletion.fs.globSyncmakes no call of the publicArray.fromorFunction.prototype.bind, and it returns the array thatGlob#globSyncbuilt.process.nextTickglobcprm, recursiverm,opendir,Dir.readDir.closeNot in this PR
FileHandlestreams have the same order difference. node:fs: call a FileHandle stream back from a microtask, through a job arm of the binding #48713 changes them.dns.lookupof a name,dns.lookupService,dns.resolve4,dns.reverseand thebind()callback ofnode:dgramprintcb job tick. Node printscb tick jobthere. That is the opposite direction, in other modules, and Bun 1.3.14 prints the same.fs.glob(pattern, null, callback)andfs.globSync(pattern, null)walk as with no options. Node throwsERR_INVALID_ARG_TYPE. The cause isoptions ?? kEmptyObjectatsrc/js/node/fs.ts:1254and:1259and atsrc/js/ahead-of-time/node/fs.ts:1341and:1346:nullbecomes{}before theGlobconstructor checks it. A fix changes what a program gets that passesnulltoday.src/jsoutside the glob path still readArray.fromorArray.fromAsync. Main has 26, with 5 in the glob path. The command isgrep -rnE --include='*.ts' 'Array\.from(Async)?\b' src/js | grep -vE '^[^:]+:[0-9]+:\s*(//|\*|/\*)'. The 21 are 13Array.fromcalls, 5$Array.fromcalls, 1Array.fromAsynccall (internal/streams/iter/pull.ts:944) and 2 copies taken at load. They are in the console,Buffer, sql, the inspector,node:test,node:http2,node:http, trace events, streams, the repl andutil.inspect.src/js.cwdis omitted #48715 changes the same statement ofGlob#globSync(it picks another stat function whencwdis omitted). The PR that lands second calls the picked function in the index loop.Self-review
A review of the first form of this change raised 16 concerns. That form kept a copy of
Array.fromAsynctaken whennode:fsloaded, and caught a throw in each handler. The main finding: a program that replacedArray.fromAsyncbeforenode:fsloaded madefs.globthrow aTypeError, where main and Node call back. All 16 are addressed in the code, the tests or this text. Two asks inside them are not done:bun test --aottest, with a stand-in forArray.fromAsyncin a preload. No text ofnode:fscallsArray.fromAsyncnow,fs.test.tsholds the first text, and the new deed of the census test compares the two texts.A second review read the
fs.globSynccommit and raised 20 concerns. 18 are addressed:glob.test.tsreplace a built-in by assignment and put it back infinally. The file gets no new import.Function.prototype.bindhas a test, and one script replacesArray.fromand theSetiterator beforenode:fsloads. That script also runs on Node.Setiterator is closed with$forEach. The first form of the commit had afor...ofthere.2 are rejected. Both ask to keep
fs.globSyncout of this PR: one by a reply alone, one by a later PR. The review rules count a sync and an async form as one concern. A separate PR changes the same hash line of the second text, so the two conflict.Verification
All runs are on this branch with main (
d0f4fdd65fb) merged in. "Main" is a build of the same tree with main's threesrc/jsfiles.bun bd test test/js/node/fs/fs.test.ts -t "except for fs.glob": with main'ssrc/, 8 of the 10 Bun tests fail. With this PR all 10 pass. The 2 that pass both ways are the throw tests. They fail withoutensureCallback.28.0.0.test/js/node/fs/glob.test.ts: 37 pass on the ASAN debug build. The 3 new tests fail with main'ssrc/. One names the callback argument. Two replaceArray.fromandFunction.prototype.bind: main makes 4 calls and 1 call, this PR makes 0.fs.test.tson release builds: 649 pass and 1 fail with this PR. With main the 8 tests above fail too.keeps a non-throwing callback in the same place in the event loop. This PR does not touch it, and it fails on main in the same runs. It racesfs.statagainstsetImmediate. Before the merge of main its script printed the expected order in 77 of 120 runs with this PR. With main it did so in 80 of 120.fs.test.tson the ASAN debug build with this PR: 645 pass, 5 fail. Four arereaddir(path, {recursive: true}) ... x 100tests. Two reach their 10 s limit, and the two behind them fail after that. The fifth isfs/promises > rmdir > removes a dir. It reached its 5 s limit while two other jobs ran on the machine, and it passes alone. None of the five callsfs.glob.test/aot/run-parallel.tswith the new program: pass in all four builds on the release build. Node v26.3.0 and the Node nightly print the same six lines. On the ASAN debug build three of the four builds passed, and the fourth ended with theSEGVbelow.test/aot/the-library-calls-the-same-built-ins-compiled-as-plain.test.tswith the two new deeds: 10 pass, on the release build and on the ASAN debug build. The deeds pass on main too. They hold the two texts ofglob()and ofglobSync()to each other.test/aot/under-the-inspector-the-console-changes-what-is-public-and-no-record-of-an-internal-file.test.tsfinds the record thatinternal/fs/globexports in a heap snapshot, by the names of its keys. It now namesglobAllin place ofglob. With main'ssrc/it finds no record and fails. With this PR it passes on the release build.test/aot/embedded-assets.test.ts, whose executables callfs.globSync: 149 pass on the release build.test/aot/parallel/under-proof-checks-a-unit-names-nothing-out-of-its-reach/a-record-of-one-member-holds-a-function-under-a-name-that-the-glob-library-reads.cjspasses in its three builds.SEGVinWTF::removeIterator, underJSC::AOT::Typed::LinkFlow. For the program of the inspector test (--compile-inspector) it does so in 4 of 6 builds with this PR, and for the new program under--aot-harden=proof-checksin 1 of 6. Before the merge of main the counts were 8 of 10 with this PR. With main'ssrc/they were 6 of 10. WebKit compilesWTF::removeIteratoronly withoutNDEBUG, so a release build has no such call.test-fs-glob.mjs448 pass, andtest-fs-glob-throw.mjspasses. All 337test-fs-*files oftest/js/node/test/parallelpass, as on main.test/aot/other-suites/run.ts node test/js/node/test --only test-fs-globon the release build:same 2,DIFFERS 0.test/js/node/fs/dir.test.ts,cp.test.ts,test/internal/second-texts-of-modules.test.ts,test/internal/export-lists-of-both-texts-of-modules.test.ts,test/internal/arms-of-modules-for-compiled-programs.test.ts: pass.bun lint: no problems in 257 files.bun src/codegen/check-types-of-modules.ts: 3 errors, all for files ofvendor/WebKit, which this checkout does not hold. Main prints the same 3.no test proof · iteration 4 · platform-specific test(s) that do not run on this machine, deferring to CI, which covers all platforms: test/js/node/fs/glob.test.ts, test/js/node/fs/fs.test.ts