Repository navigation
AOT: strings that packages evaluate, modules in bundles, class factories, mocks in import cycles - #48677
Conversation
…ies, mocks in import cycles On 4385062. 1,933 files, +84,461 -1,451: test 1,755 files, +68,238 -757; src 175 files, +16,203 -684; docs 2 files, +19 -9; scripts 1 file, +1 -1. Pins the preview build of oven-sh/WebKit#1040. [skip size check], as in 607519a. Executables and builds - A mixin or a class factory is copied for each call in module code, also in TypeScript. Constructors that are kept in variables, fields or namespaces are classes. Enums are data. - The modules inside the output of webpack and Browserify are compiled as files, with their requests. Names that go through the helpers of Babel, esbuild, Rollup and swc are read directly. - What a package joins from its own files and hands to new Function() or eval() is compiled, and so is a WebAssembly file that glue code names beside itself. - Stores into a path's variables are followed in order. Some files that the program only writes are no longer warned about (TS100098). - A route that is the Bun.file() of an embedded file is served. A data file that is imported under two types and required by a path made at run time has the form that `bun run` gives it in more programs. - Names of packages and paths of data files are read when the program first needs them. `bun test --aot` - Partial mocks in import cycles, imports behind awaited imports, packages that resetModules() keeps and hoisted mocks over packages are as in `bun test`. Fewer notices where a run is as in `bun test`. Plain Bun too - concat() keeps the elements of a species constructor's result where the argument has holes. The bound of the compiler's bytes goes up: 29,515,694 from 26,919,547, of it aot/walk 1.22 MB, codegen 0.52, facts 0.37, templates 0.11, the rest 0.38 MB. Known issues: see the pull request.
|
Updated 4:05 AM PT - Nov 12th, 2026
@Jarred-Sumner, your commit 0b35f90 is building: |
There was a problem hiding this comment.
Beyond the inline findings, I also checked the recursion in calls_only_published_helpers (src/aot/known_helpers/resolve.rs:317) for the same unbounded-cycle shape as the class-heritage one — helper matching in src/aot/known_helpers/find.rs:292-316 marks a function InProgress while verifying its references and rejects anything that re-enters, so recognized helpers cannot form a cycle there. The is_plain_javascript regex-literal concern (src/sema/parser/parser/mod.rs:599) and the .wasm mention embedding path (src/aot/compile/bind_stage.rs) were also examined and not confirmed. No build or test run was performed as part of this review; the engine-test and from-source .ir fixtures were read, not executed.
Extended reasoning...
The change spans 1933 files and roughly 84k added lines, almost all under src/aot, src/sema, src/standalone_graph and their test fixtures, plus a WebKit dependency pin, three JSC binding files and node_fs. The security-relevant surface is the compiled-program trailer parser in src/resolver/compiled_program.rs and src/standalone_graph/compiled_program.rs, which reads untrusted executable payloads; one inline finding already covers a panic there. Six confirmed findings, including a build-breaking preview-tag pin and a user-reachable abort, plus the sheer size rule out approval; this note only records what else was examined so a human reviewer need not retrace it.
Findings marked 🟡 are optional suggestions and need no follow-up push.
…t follow [skip size check] 99 files, +1,469 -1,082: test 65 files, +1,092 -716; src 31 files, +349 -358; scripts 3 files, +28 -8. Pins the preview build of oven-sh/WebKit#1040 at 1a7e70d8. - Wrong on main: a function that calls itself last behind `!`, `satisfies` or a JSDoc cast, beside such a call without one, lost what the first stored. The loop has the variables of the calls that jump, and only these jump. - Wrong on main: inside `with (..)`, a test of a name (`typeof x`, `!x`, `if (x)`) was decided by the variable outside. What the build folds asks whether an object may hold the name. - The functions of a literal are read once for all their `this`, so a build's work grows with them. - `Object.assign()` of literals is compiled as on main: the first commit's stores by name go. - A constant of a bundle that `__export()` names stays in a scope, so the build no longer stops. The text of a function has no parenthesis of an operand that is left out; a statement without a semicolon had one on main too. - Classes that extend each other in a ring no longer hang the build. The class of a worker and a method are found through chains of any length. - A damaged list of package names in an executable ends the program with the message of a damaged one. A folder that macOS clones is noted file by file. - Where the front end looks for `undefined` beside a namespace, it asks the binder. The other new places that compare names as text are listed in test/aot/compiler/names-compared-as-text.ts, with their reasons. - Lints: bundles of stored programs name their modules by paths. Four lines of the front end share a method or have other names. scripts/verify-baseline-static lists five more functions of the engine, which ask the processor first. - Tests: twenty files of expected compiler output, and others, follow what the first commit changed (constants, enums as data, narrowed values, names of the global object that nothing stores). In test/aot/perf, rows whose twins got faster go or have another bound, and two of them are gaps. The stored program what-a-suspended-function-holds.mjs goes: what it printed depended on the time of a collection. - New stored programs: what is above, and what the engine's fifth and sixth commits fix.
There was a problem hiding this comment.
Code review found no new issues
No new issues were found in this update; 2 findings from earlier reviews are still open above.
Still open from earlier reviews (2):
- Unresolved: 1 minor or pre-existing, 1 blocking on lines changed since (possibly already fixed).
If you have decided not to act on one of these findings, resolve its thread (a reply alone leaves it open) and the next review stops counting it. To review this commit again now, use Re-run on its "Claude Code Review" check.
…ze check] 65 files, +2,109 -106: test 60 files, +2,093 -100; src 4 files, +15 -5; scripts 1 file, +1 -1. Pins the preview build of oven-sh/WebKit#1040 at 8c0d09b7. - The helpers that step the matches of `matchAll()` come where a text names it, and no longer count as called by its native code: a program that lets `String.prototype` go kept thousands of members for them. - Tests of captured variables and of a full address range use values that are not constants, and objects that have no room for later names. Rows of test/aot/runtime-use-prototypes.test.ts ask for names read from the prototype of a record. - The row of test/aot/build-scaling on as many names as classes is judged by its instructions, which still grow; its counts and its memory no longer do. A stored program on few objects with other names and a test of elements under constants go with the rules they were for. - The row of test/aot/plain-bun.test.ts on `sort()` goes: it failed or not by the moment at which the JIT compiles. - The programs of test/aot/from-source run in 48 test files instead of 32, each with fewer of them. - New stored programs: elements that a loop carries past a `continue`, stores in front of an append, stores of a loop that is written out, narrow empty slots, and parents that a helper writes.
There was a problem hiding this comment.
Beyond the inline findings, I also checked the deletion of test/aot/parallel/async/what-a-suspended-function-holds.mjs: the property it covered (locals, parameters and this of a suspended async function or generator staying alive) is still asserted by the untouched test/aot/parallel/async/suspended-async-keeps-alive and suspended-generator-keeps-alive programs, so it is not a lost safety net. This review ran no build or test commands; the hunt was also cut off at its finding budget rather than running dry.
Extended reasoning...
The change is roughly 2,000 files touching the AOT compiler (src/aot, src/sema), the compiled-program runtime (standalone_graph, node_fs, server_body) and a large body of AOT tests; the one candidate ruled out here was the deleted suspended-function test, which has pre-existing sibling coverage.
Findings marked 🟡 are optional suggestions and need no follow-up push.
Still open from earlier reviews (2):
- Unresolved: 1 minor or pre-existing, 1 blocking on lines changed since (possibly already fixed).
If you have decided not to act on one of these findings, resolve its thread (a reply alone leaves it open) and the next review stops counting it. To review this commit again now, use Re-run on its "Claude Code Review" check.
| // Like a file on disk. | ||
| let Some(file) = argument.as_class_ref::<Blob>().filter(|it| is_asset(it)) else { | ||
| return Ok(None); | ||
| }; | ||
| let file = AnyBlob::Blob(file.dupe()); | ||
| let route = StaticRoute::init_from_any_blob_with_headers(global, file, None, 200)?; |
There was a problem hiding this comment.
🟡 (optional) A compiled program that passes an embedded file to routes and later rewrites that file on disk keeps serving the build-time bytes, while bun run serves the new ones. The new branch at src/runtime/server/server_body.rs:891-897 turns an is_asset Blob into a StaticRoute, which snapshots the embedded bytes once at Bun.serve() time. Fix: serve an embedded-file route from the current contents per request (as FileRoute does for a disk file), falling back to the embedded bytes only while the disk has no copy; or refuse the write, so that compiled and plain runs agree. The same staleness already exists for new Response(Bun.file(embedded)) on the base, so that form is pre-existing.
Why this was flagged
Trigger: a program built with --compile --aot and --asset ./page.html registers routes: { "/page": Bun.file(path.join(import.meta.dir, "page.html")) }, is run where the source folder is not on disk, and later writes that same path (allowed once the folder exists on disk, see refuse_open_for_write at src/runtime/node/node_fs.rs:570-584). Bun.file() of the embedded path yields the template Bytes blob (src/runtime/webcore/Blob.rs:3201-3207), so the new code at src/runtime/server/server_body.rs:892-897 calls is_asset and builds a StaticRoute from a dupe of the embedded store. After the write, find_at_source_path (src/standalone_graph/compiled_program.rs:2316-2319) makes Bun.file() and fs.readFileSync return the disk copy, but the route still answers with the embedded bytes. Under bun run the same route is a FileRoute that opens the path on every request (src/runtime/server/FileRoute.rs:263-301), so clients see the new contents. On the base branch the compiled program threw the "'routes' expects a Record<...>" error at startup instead.
Verification: Bun.file(path) in a compiled program returns the embedded template Bytes blob (src/runtime/webcore/Blob.rs:3201-3208), so the new branch at src/runtime/server/server_body.rs:892-897 wraps it in a StaticRoute, which serves self.blob.slice() forever, unlike the FileRoute bun run builds (FileRoute.rs:191-201). On the base branch this route shape threw "'routes' expects a Record<...>".
| @@ -25,26 +25,6 @@ async function plain(program: string, env: Record<string, string> = {}) { | |||
| return { stdout, exitCode }; | |||
| } | |||
|
|
|||
There was a problem hiding this comment.
🟡 nit (optional): maintainers lose the only check that tracked the JSC JIT bug where Array.prototype.sort skips the write-back of sorted items after a mutating comparator. The test.failing block at test/aot/plain-bun.test.ts:28-45 on the base is deleted, and the program's comparisons helper (test/aot/parallel/arrays/sort-write-back-after-mutation/program.js:5) is changed from calling sort with a counting comparator to a fixed lower bound, so nothing now runs that program under plain bun run with a JIT-sensitive comparator. Fix: keep a check of the property, either by converting the block to test if the WebKit bump fixed the bug, or by keeping test.failing with the original trigger and a comment naming why it changed.
Why this was flagged
The base branch has test.failing("Array.prototype.sort writes every sorted item back although the comparator has changed the array in its last comparison", ...) at test/aot/plain-bun.test.ts:28-45, which runs test/aot/parallel/arrays/sort-write-back-after-mutation under plain bun eleven times (once with BUN_JSC_useConcurrentJIT=0) and compares with the stored output; its comment says "Once the bug is fixed this test fails: change test.failing to test." The diff removes the whole block, and no other test in the suite spawns that program under bun run. The same diff rewrites comparisons in program.js:5 from mk().sort((x, y) => (n++, cmp(x, y))) to Math.max(1, mk().filter(x => x !== undefined).length - 1), so the counting comparator path is gone too. After merging, a regression or fix of the JIT write-back bug is observed by nothing; on the base it flipped this test. No reason or replacement is visible in the checkout.
Verification: Commit d4fcdfe's message says "The row of test/aot/plain-bun.test.ts on sort() goes: it failed or not by the moment at which the JIT compiles", and offers no replacement. The diff removes the whole test.failing block (base lines 28-45). test/aot/run-parallel.ts:1320 if (stored !== undefined) return undefined; skips the plain-Bun reference run, so nothing else runs that program under plain Bun.
| pub(crate) fn is_asset(blob: &Blob) -> bool { | ||
| let graph = Graph::get_ref().filter(|it| it.compiled_program.is_some()); | ||
| let (Some(graph), Some(store)) = (graph, blob.store()) else { | ||
| return false; | ||
| }; | ||
| let path = store | ||
| .get_path() | ||
| .filter(|it| is_bun_standalone_file_path(it)); | ||
| let is_whole = blob.offset.get() == 0 && blob.size.get() >= store.size(); | ||
| let file = path.and_then(|it| graph.find_data(it)); | ||
| is_whole | ||
| && file.is_some_and(|it| is_view_of(store, it) && !graph.is_file_for_file_system_alone(it)) |
There was a problem hiding this comment.
🟣 pre-existing, not blocking: pre-existing: a compiled program that serves a file the compiler embedded on its own (no --asset) still fails at startup with the generic routes error, while bun run serves it with 200. is_asset at src/runtime/api/standalone_graph_jsc.rs:97 rejects every embedded file whose record has is_asset false, so AnyRoute::from_js at src/runtime/server/server_body.rs:892 returns None and ServerConfig.rs:845 throws "'routes' expects a Record<string, Response | HTMLBundle | ...". Fix: accept any whole embedded-file blob as a static route (drop the is_file_for_file_system_alone gate, or map such blobs to a FileRoute), or at least throw a message naming the path and --asset; the new test only covers the --asset case.
A small fix can ride a push you are already making; otherwise a short reply is enough.
Why this was flagged
A program writes routes: { '/d': Bun.file(join(import.meta.dir, 'd.txt')) } and is built with bun build --compile --aot without --asset ./d.txt. The compiler records ./d.txt as a file it embeds for the file system, with is_asset false in its record. At run time Bun.file(path) returns the Bytes-backed template blob, so FileRoute::from_js at src/runtime/server/FileRoute.rs:191-201 skips it and StaticRoute::from_js only takes a Response. The new fallback at src/runtime/server/server_body.rs:892 calls is_asset, which at src/runtime/api/standalone_graph_jsc.rs:97 requires !graph.is_file_for_file_system_alone(it); is_file_for_file_system_alone at src/standalone_graph/compiled_program.rs:1590-1593 is true for every embedded file whose record is not an asset. So from_js returns None and src/runtime/server/ServerConfig.rs:844-846 throws the generic routes message at Bun.serve. Under bun run the same value is a disk file and FileRoute serves it with 200. The base branch threw the same error for every embedded blob; this change opens the door only for --asset files and leaves the compiler-found ones failing as before.
Verification: pre-existing. Trigger: a program compiled with --aot whose routes holds Bun.file(join(import.meta.dir, "d.txt")) and that was built without --asset ./d.txt. The new fallback at src/runtime/server/server_body.rs:892 requires is_asset(blob), and is_asset ends with && !graph.is_file_for_file_system_alone(it). So from_js returns Ok(None) and ServerConfig.rs:844-846 throws.
| fn note_copied(path: &[u8]) { | ||
| note_opened_for_write(path, true); | ||
| #[cfg(target_os = "macos")] | ||
| if let Some(graph) = compiled_program_graph() { | ||
| for file in graph.find_files_on_disk_below(path) { | ||
| note_opened_for_write(&file, true); | ||
| } | ||
| } | ||
| } |
There was a problem hiding this comment.
🟣 pre-existing, not blocking: pre-existing: on Linux a compiled program that holds an embedded file open and then fs.cpSync()s a folder over it keeps reading the old embedded bytes from that descriptor, while fs.readFileSync of the same path returns the new bytes. note_copied at src/runtime/node/node_fs.rs:524 enumerates the files below the copied folder only under cfg(target_os = "macos"), and the sync recursive copy (cp_sync_one_directory, copy_single_file_sync) notes nothing per file, so move_descriptors_to_disk only ever sees the folder path, which lookup_file cannot find. Fix: move the descriptors of every file the sync recursive copy writes on every platform, e.g. drop the macOS gate on the enumeration in note_copied or note each file inside cp_sync_one_directory, so cpSync matches fs.cp.
A small fix can ride a push you are already making; otherwise a short reply is enough.
Why this was flagged
Trigger: a compiled executable on Linux (or any non-macOS unix) whose script opens an embedded file with fs.openSync and then calls fs.cpSync(srcDir, embeddedDir, { recursive: true }) where embeddedDir holds that file. NodeFS::cp at src/runtime/node/node_fs.rs:9131-9139 runs cp_sync_inner, whose per-file path copy_single_file_sync (node_fs.rs:9507) and cp_sync_one_directory (node_fs.rs:9291) call no note_opened_for_write. The only note is note_copied(args.dest) at 9139, which calls note_opened_for_write(folder, true) and then enumerates files below the folder only inside #[cfg(target_os = "macos")] at node_fs.rs:524. move_descriptors_to at src/standalone_graph/compiled_program.rs:1660-1667 does key_under_own_root then lookup_file on the folder key, finds no file, and returns without reopening any descriptor. So the fd registered by add_descriptor keeps reading the embedded copy while a fresh read of the path finds the written file on disk. The base behaved the same for this sync path (it noted only args.dest).
Verification: pre-existing. note_copied (522-530) calls note_opened_for_write(folder) and only enumerates files below the folder under #[cfg(target_os = "macos")] (524-528). note_opened_for_write → graph.move_descriptors_to_disk (515) → move_descriptors_to (compiled_program.rs:1658-1681), which does key.and_then(|it| self.lookup_file(&it)) (1663), so a folder key yields None and the function returns before touching DESCRIPTORS.
…s [skip size check] 95 files, +511 -108: test 82 files, +391 -70; src 12 files, +119 -37; scripts 1 file, +1 -1. Pins the preview build of oven-sh/WebKit#1040 at 6f7dec74. - A read of an embedded file's open descriptor (`node:fs`, `Bun.file()`) no longer falls into the moment at which another thread's first write of that file moves the descriptor to the disk: it could mix old and new bytes. What a thread of the pool has written is seen by the next call of `node:fs`. - In plain Bun too these reads ask one word of memory whether the descriptor is recorded; outside an executable it is zero, and they read as before. - Two stored programs ask for the room of arrays that a loop pushes onto by turns. A test of the engine goes with what it was for. - A stored program on a data file that is required ahead of its import holds its output: what `bun run` prints there hangs on its threads. - `require("url")` is an object where a build follows strings that a package evaluates. One question for a regular file serves the compiler and the bundler. - The programs of a-few-of-every-sweep run in eight test files instead of four, those of test/aot/from-source in 64 instead of 48, and a program on readable streams makes its calls in two turns instead of three: builds with AddressSanitizer took too long. - A file that holds a webpack or browserify bundle beside a function of its own with a direct `eval()` no longer stops the build: a request in the bundle is the request of the module that the build has noted around it.
…ace, code from strings (#1040) The JavaScriptCore side of a round of work on Bun's ahead-of-time compiler (`bun build --compile --aot` and `bun test --aot`, both experimental). Bun's side is oven-sh/bun#48677. This follows #1036. 847 files, +74,933 -6,494: the rest of aot/ 619 files, +63,912 -5,928; aot/tests 181 files, +7,585 -253; outside aot/ 47 files, +3,436 -313. Nothing in heap/, WTF or bmalloc. Builds that stop on main, fixed - A loop over a typed array under `y - (y & ~7)`, where `y` is 2 ** 32 or more. arm64: more than 128 MB of compiled regular expressions. The remainder of a product of BigInts, `(a * b) % c`, in two arms of a loop. - With `--aot-harden=proof-checks`: a built-in constructor passed as a callback (`list.map(Date)`); `a[0] - 1 + a[2] * x` in both arms of a branch, then `a[2] = 7`; `if (a[i] < 0) a[i] = x; else a[i] = x` in a loop that then adds the next element and multiplies the stored one. Wrong on main, fixed - Accessors that a program defines on a prototype one after the other, behind the first instance, each answer by their own definition. Plain Bun too - `concat()` keeps the elements of a species constructor's result where the argument has holes. Objects and classes - An object or array that each iteration of a loop allocates and that doesn't escape is carried through the loop as its fields, private fields included, and allocated only on the exits that need it. - int32 fields take four bytes. Short arrays, empty literals and some records take smaller cells. - A class that a function creates (a mixin, `Class.create()`, also in TypeScript) has code and a structure for each call in module code. Subclasses of Array, typed arrays, Map and Set are allocated from templates. - Static fields are read as variables, also through subclasses. Accessors from `Object.defineProperty()`, defaults on prototypes and assigned prototypes keep their slots. - A Proxy trap written next to the Proxy is called directly; without a trap, the target's slot is read. `Reflect.set()` runs in a stub. Loops and numbers - Loops over typed arrays and arrays of numbers use 128-bit vectors. An element stored by one iteration reaches the next in a register. Arrays that only some iterations read are checked before the loop too. - `push()`, `pop()`, `shift()` and `splice()` of arrays in fields, in variables and behind methods are inlined. An array filled by a loop reserves its capacity. - A loop over arrays of unknown kinds gets a version for the kinds it meets. Module variables that hold typed arrays cache the address and length. Remainders, products modulo `n`, shifts and carries stay integers. A name that no object of a loop has is answered inline after the first miss. Strings and regular expressions - A string that a loop consumes piece by piece is carried as a position in its base string. A loop over `split()` or `matchAll()` takes one piece at a time. - A sticky pattern, or one held by a constant, a field or a private field, is matched in place, and only the groups that are read are created. One created at run time is attempted only where the character before it and its leading lookaheads let a match begin. Calls - Callbacks of array methods run in one loop also where the method is in a function that the loop calls. `sort()` computes keys once where the comparator only reads them. - A chain of `then()` calls keeps one object until something needs a promise of it. Generators are stepped inside more consumers. Recursive functions check arrays and numbers once, at their entry. Code from strings - Strings that a program evaluates at run time (`eval()`, `new Function()`, `node:vm`) are interpreted in an executable. The interpreter caches properties, globals and the names of a context, and calls getters and common built-ins directly. Build, checks and tests - The new forms cost build time. Fewer queries per call, literal, edge and register, and tables that are filled when they are first needed, pay for part of it. - More generated programs for fuzzing, and injected faults that the checks of a build must refuse. Tests of vectors and of the assembler's new forms.
There was a problem hiding this comment.
Findings marked 🟡 are optional suggestions and need no follow-up push.
Still open from earlier reviews (5):
- Unresolved: 4 minor or pre-existing, 1 blocking on lines changed since (possibly already fixed).
If you have decided not to act on one of these findings, resolve its thread (a reply alone leaves it open) and the next review stops counting it. To review this commit again now, use Re-run on its "Claude Code Review" check.
…ed loosely [skip size check] 11 files, +200 -19: test 6 files, +140 -14; src 4 files, +59 -4; scripts 1 file, +1 -1. Pins oven-sh/WebKit#1040 as merged, 6fc709c298409fe433a60f6aff328a45a8974532. - test/cli/hot/hot.test.ts writes texts of one length, so a load never reads the rest of a longer one; test/cli/run/no-orphans.test.ts waits for what a parent does a moment later. Both failed at times. - A build followed `==` and `!=` of a variable of the environment as it follows `===`: a file that a package chooses by such a test of a number, a boolean or null could be missing, and the read failed with ENOENT. Values of unlike kinds go both ways now; `null` and `undefined` are equal. - A loop of test/aot/embedded-assets.test.ts ends with an error behind 30 seconds. - The build of a JavaScript file of 4 MiB or more, in which a variable holds objects of more than 1,001 unrelated classes, did not end. Where a part of a file meets that limit, the file waits for the last round and is compiled whole.
3 files, +134 -4: test 1 file, +113 -0; src 2 files, +21 -4. - The build joined the operands of `+` and `+=` as text where it follows how a package chooses a file, also where both are numbers, booleans or null: the file could be missing, and the read failed with ENOENT. They are added as JavaScript does now. - What the build finds by following a file's statements is added to what it finds by its scan and takes nothing away, so nothing is left out that was embedded without it. - test/aot/embedded-assets.test.ts has a test with a package that chooses its files in many ways.
…ize check] 2 files, +15 -23: src 1 file, +14 -22; test 1 file, +1 -1. - What the build finds by following a file's statements is added to what its scan finds, so a build tells of one more read of path-read-before-overwrite/main.mjs (TS100098), as on main, and the rows of the source tests name the paths that the scan finds too. Two rows go out, of loops that join a path: the scan finds hundreds of paths there.
| fn find_function_passed(inputs: &ScanInputs<'_, '_>, callee: ExprId) -> Option<(Holder, Atom)> { | ||
| let &ScanInputs { hir, bound, .. } = inputs; | ||
| if !matches!(hir[callee].kind, ExprKind::Ident(_)) { | ||
| return None; | ||
| } | ||
| let (function, place, path) = find_parameter_key_path(hir, bound, callee)?; | ||
| let parameter = hir[function].params.iter().nth(place)?; | ||
| if !path.is_empty() { | ||
| return None; | ||
| } | ||
| let no_names = |_: Atom| Vec::new(); | ||
| let passed = find_arguments_where_written(hir, bound, function, &no_names); | ||
| let passed = passed.and_then(|it| hir.ids(it).nth(place)); | ||
| let mut functions = hir[parameter].default.some().into_iter().chain(passed); | ||
| functions.find_map(|it| find_function_named(inputs, hir.skip_type_assertions(it))) |
There was a problem hiding this comment.
🔴 A compiled program can fail to find an embedded file when the writer of a Bun.file is a defaulted parameter, (text, write = Bun.write) => write(Bun.file(p), text), and a caller passes a function that reads the file instead. find_function_passed at src/sema/import_sites/changed_files.rs:185-199 answers from the parameter default alone whenever the function is not an IIFE, so are_all_uses_writes at changed_files.rs:777 takes the file object as written only and call_sites.rs:1162 drops it from the reads to embed. Fix: when deciding is_written_alone, accept a parameter's value only where no call can pass another one (the IIFE arguments), and treat a default-only answer as unknown; keeping the default-based answer for reads is harmless since it only embeds more.
Why this was flagged
Trigger: a file compiled with bun build --compile holds export const save = (text, write = Bun.write) => write(Bun.file(path.join(import.meta.dir, "state.json")), text); and somewhere save("x", file => file.text()) or a test seam passes another writer. scan_calls at src/sema/import_sites/call_sites.rs:1158-1162 keeps a Bun.file path only if !is_written_alone(call). are_all_uses_writes at src/sema/import_sites/changed_files.rs:766-787 sees the Call write(...) with the file at place 0 and asks find_function(callee). find_function (changed_files.rs:178-182) reaches find_function_passed. There find_arguments_where_written (parameters.rs:243-263) returns None because the arrow is not immediately called, so functions holds only the default Bun.write, which resolves to (Holder::Bun, write) and line 777 returns true for place 0. The path is therefore not embedded. At run time the reading callback calls file.text() on a path the executable does not hold and gets ENOENT, while on the base branch every Bun.file path that was not otherwise used was embedded.
Verification: find_function_passed at src/sema/import_sites/changed_files.rs:185-199 calls find_arguments_where_written, which returns Some only for an IIFE, so lines 198-199 resolve the default Bun.write with no look at any caller of save. The is_exported/is_every_mention guard only sits in the None arm (changed_files.rs:779-783), so it is bypassed. On the base, call_sites.rs kept the path as a read and embedded it.
| let functions = functions.filter_map(|it| { | ||
| let end = end_of_body(hir, it)?; | ||
| Some((hir[it].start, end, Owner::Function(it))) | ||
| }); |
There was a problem hiding this comment.
🔴 A compiled program can throw "OmittedFromExecutable" where bun run evaluates the string, when a parameter default of an empty-bodied function changes the eval text. collect_identifiers at src/aot/load/known_evaluations/settled.rs:81 skips every function whose block body has no statements, because end_of_body returns None for it. Identifiers in that function's parameter defaults are then owned by the enclosing function, so a store there never enters stored_elsewhere and the Settler keeps the old value and settles the eval. Fix: give every function a range covering its parameter list even when its body is empty (end at the last parameter or the closing brace), so stores and reads in defaults count as another owner's, as they do for non-empty bodies.
Why this was flagged
Trigger: a sloppy or strict function such as function f(env) { let code = "a"; function g(_ = (code = "b")) {} if (env.DEBUG) g(); return eval(code); }, compiled with code generation from strings allowed, where the known-evaluations evaluator records "a" but not "b". end_of_body at src/aot/load/known_evaluations.rs:85-93 returns None for a block body with no statements (line 90), so the filter_map at settled.rs:80-83 gives g no range and assign_owners at settled.rs:88-107 assigns the code identifier in g's default to Owner::Function(f). The check at settled.rs:127-139 then skips it, so code is neither in stored_elsewhere nor captured. read at settled.rs:388-410 answers "a", and settles() at settled.rs:276-281 succeeds. is_settled_eval is then true at src/aot/lower/scope/eval.rs:65-67, and emit_settled_eval_end (eval.rs:219-240) throws RuntimeErrorKind::OmittedFromExecutable when the runtime text "b" matches no known evaluation. On the base branch settled_evals does not exist and emit_direct_eval performs a real direct eval, which returns the value of "b".
Verification: src/aot/load/known_evaluations/settled.rs:80-83: end_of_body returns None for FnBody::Block with no statements, so an empty-bodied g is dropped from ranges. Lines 134-136: if owner == Owner::Function(declaring) { continue; } runs before lines 137-139, so the store to code is never recorded in stored_elsewhere. eval.rs:239 emits RuntimeErrorKind::OmittedFromExecutable.
| Value::Number(at as f64), | ||
| Value::String(Rc::clone(text)), | ||
| ]; | ||
| let value = self.call_function(*function, *around, &args); | ||
| let value = self.call_function(*function, *around, &args, Value::Unknown); | ||
| self.text_of(&value) | ||
| } | ||
| Some(value) => { |
There was a problem hiding this comment.
🟡 nit (optional): the build-time evaluator splices a function's source code into an eval text when replace() is given a function on a non-ASCII string, instead of giving up. In replace at src/aot/load/known_evaluations/evaluator.rs:2001 the Value::Function arm is guarded by text.is_ascii(), so a function replacement on non-ASCII text falls to the Some(value) arm, whose text_of returns the function's source. Fix: answer Value::Unknown for a function replacement whenever the function arm does not apply, so no candidate text is built from a function's source. The bogus text is parsed, consumes text_allowance, and never matches at run time.
Why this was flagged
A file compiled with code generation from strings holds something like const t = "function NAME() { return 'héllo' }"; eval("(" + t.replace("NAME", () => "add") + ")"), with a non-ASCII character in the template. In replace at src/aot/load/known_evaluations/evaluator.rs:1998-2016 the arm Some(Value::Function(function, around)) if text.is_ascii() at line 2001 is skipped because text is not ASCII, and the next arm Some(value) at line 2010 calls self.text_of(value). text_of at evaluator.rs:1392-1397 maps Value::Function to source_of, the function's source text, so the replacement becomes () => "add". On the base branch the same arm order exists in this file, but the evaluator did not read files or run nested texts, so fewer non-ASCII templates reached it. The wrong text is appended and parsed by append_texts (known_evaluations.rs:231), costing the per-file text allowance (evaluator.rs:1993 pays_for_text), and it can never equal the string the program computes at run time, so no compiled function is used for that eval. No safeguard rejects it before parsing: the only filter is that the appended text parse as JavaScript.
Verification: In src/aot/load/known_evaluations/evaluator.rs:2001 the function arm is guarded by text.is_ascii(); for non-ASCII text the catch-all Some(value) arm at line 2010 calls text_of, which at line 1395 maps Value::Function to source_of, so the replacement becomes the function's source text. On the base branch to_text() returned None for a function (values.rs:159-162), so replace answered Value::Unknown.
main has one more AOT landing (#1040). No conflicts. Its new code has no call of an import that is made again, no job of the loader and no synchronous load: nothing more takes the async context. 6fc709c is the commit that Bun's main pins (oven-sh/bun#48677).
BunPlugin.cpp uses JSC::SuspendExceptionScope, which FrameTracers.h declares. #48677 took the include out. A release build compiles all the same: its unified source holds 32 files, and BunCPUProfiler.cpp includes the header before BunPlugin.cpp. A debug unified source holds 8 files and none of them includes it, so the debug build stops with BunPlugin.cpp:3528:10: error: no member named 'SuspendExceptionScope' in namespace 'JSC'
main has #48677, which pins WebKit at 6fc709c298. WEBKIT_VERSION is now 5b6deeea52: the head of oven-sh/WebKit#941, which is that commit with the change. Conflicts: the pin, and one list of CI's runner, which takes main's and gets the test of this branch again (test/aot/ci-lists.test.ts).
… awaiting files, known strings (#48771) Bun's side of a round of work on the ahead-of-time compiler (`bun build --compile --aot` and `bun test --aot`, both experimental). The JavaScriptCore side is merged: oven-sh/WebKit#1045, as `de043228c294c72404a687634e1943f549f39048`. This follows #48714. 1,718 files, +70,470 -2,660: test 1,466 files, +55,460 -853; src 250 files, +15,006 -1,803; docs 1 file, +3 -3; scripts 1 file, +1 -1. [skip size check], as in 607519a. Tests - CI compiles a stored program with `--aot`, runs it and compares it with `bun run`; a stored test run likewise, with `bun test --aot` and `bun test`. If you don't use `--aot` - What oven-sh/WebKit#1045 lists: the length of `[, ...list]`, and the message of replaceAll() on null or undefined. - `click(selector)` of `Bun.WebView` could click an element that was about to move. It looks once a frame now, and an element with a pending animation counts as moving. Wrong on main, fixed - Where an entry point imports a file only for its effects, the file's package says `"sideEffects": false`, and the file reads a member that an index of `export *` takes off a built-in module and passes the namespace on, the build stopped, or the member was undefined. - Where a closure captures an alias of `globalThis`, beside a property `get`, `has` or `apply` of an unknown object, the build could stop. - Where a `catch` behind a top-level `await`, or an operator, sets a path, the build did not warn that the program may read another file. - Where the value of a destructuring assignment stores to a field that the pattern reads, or an attribute of an element to one that its `key` reads (JSX's automatic runtime), the field was read as it was before. - With `--minify-syntax`, a function that another one returns and that reads `process.env[key]` gave the value of the build for the key `"NODE_ENV"`. A member of an `enum` under a key that a call passes to such a function was read as it was written, though the program stored into it. Executables and builds - Classes: parents that are parameters, members, results of calls or of `require()` are followed, also through the helpers of Babel, TypeScript and swc. Functions with prototypes assigned later, and decorated classes that are exported, are classes. - Objects: prototypes written into literals, `Object.create()` with a literal, and accessors that a helper defines are read from the source. - Bundles: universal wrappers (UMD), standalone output of Browserify, and exports that are getters are read directly. Requests beside `eval()` and `with` in a bundle are kept. - Strings the build knows: scripts that are joined from constants, names that such code declares, and calls of constants are evaluated when the program is built. - Files: an embedded file that a child process writes is read from the disk. A library that `dlopen()` opens stays the one that was embedded. - Size: an executable leaves out the test runner where nothing in it can reach `Bun.jest`, `bun:test` or `node:test`. With `--trim` it leaves out what only mocks enter, too. `bun test --aot` - The original of a mocked file outlives later runs of the file, and a mock visits only the modules that it concerns. Known issues - #48677 lists what main has; what it says of typed arrays in a file's variable, of the defaults of parameters and of products under `--aot-harden=proof-checks` is fixed since or here. Of the rest, main has too: - Executables, silent: Where a method of a constructor function gives many names to a few of its objects, all of them take the larger size (at most 128 bytes an object). - Executables away from their sources, silent: A route of `Bun.serve()` that is a `Bun.file()` of a file embedded by `--asset` or an import keeps the build's bytes when the program writes or removes that file later by a `__dirname` path, and answers a `Range` request with the whole file (status 200). - Builds, loud: The build of a program whose object literals have more than about 65,000 different numbers as fields stops. - Executables, loud: A method of lists called on a typed array whose prototype was set to that of a class that extends `Array` aborts where nothing else uses that method. - Executables, loud: A store to `process.env` behind `Object.setPrototypeOf(process.env, {})` throws a `RangeError`. - Executables, silent: A CommonJS file whose code threw and that is asked for again by `require.bind()`, `eval()`, `new Function()` or `node:vm` gives its first error instead of running again. - Builds, loud: Under `--aot-harden=proof-checks`, the build of some loops over lists that may be buffers stops; with default flags the answer is right.
Bun's side of a round of work on the ahead-of-time compiler (
bun build --compile --aotandbun test --aot, both experimental). The JavaScriptCore side is oven-sh/WebKit#1040. This follows #48617.Merge order
oven-sh/WebKit#1040 is merged;
scripts/build/deps/webkit.tspins the merged commit.What's in it
2,084 files, +88,328 −2,222 against main.
test: stored programs, test runs and testssrc/aot: the compiler's front endsrc/sema: the type checkerdocsCI compiles a stored program with
--aot, runs it and compares it withbun run; a stored test run likewise, withbun test --aotandbun test.If you don't use
--aotconcat()keeps the elements of a species constructor's result where the argument has holes (a correction of the engine). Reads ofnode:fsandBun.file()ask one word of memory, which is zero outside an executable, and read as before.Files outside the compiler; but for the rows that say otherwise, only compiled programs take these paths:
src/standalone_graph/compiled_program.rs,src/resolver/compiled_program.rssrc/runtime/server/server_body.rs,src/runtime/api/standalone_graph_jsc.rssrc/jsc/bindings/BunCompiledModules.cpp,src/jsc/bindings/BunPlugin.cppand 1 moresrc/jsc/bindings/AsyncStackTrace.cppthen()callssrc/js/ahead-of-time/internal/http.ts,src/js/ahead-of-time/node/_http_incoming.tsand 6 moresrc/bundler/aot/services.rsscripts/build/deps/webkit.tsscripts/verify-baseline-static/allowlist-x64-windows.txt,scripts/verify-baseline-static/allowlist-x64.txtsrc/standalone_graph/executable_file.rssrc/runtime/node/node_fs.rs,src/runtime/node/node_fs_binding.rsand 2 morenode:fsandBun.file()ask one word of memory whether they read an embedded file that another thread may be writingsrc/runtime/test_runner/module_mock.rsExecutables and builds
Wrong on main: a function that calls itself last behind
!,satisfiesor a JSDoc cast, beside such a call without one, loses what the first stores. Insidewith (..), a test of a name (typeof x,!x,if (x)) is decided by the variable outside.Objects, loops, strings and calls are oven-sh/WebKit#1040's text. What the front end adds:
Class.create(), also in TypeScript) is copied for each call in module code, so each class has its own code. Constructors that are kept in variables, fields or namespaces are classes too. TypeScript enums are data.export *, namespaces, interop defaults) are read directly.new Function()oreval()is compiled. The build followsreadFileSync(),require.resolve(),path.join(),import.metaandnew URL()there, and strings that such code evaluates in turn. If the string is another one at run time, that one is interpreted.Bun.file()of an embedded file (routes: { "/x": Bun.file(path) }) threw at its start; it serves the file.bun rungives it in more programs.--explainhas itsunknown.bun test --aotbun test.import()behind an awaited import waits for its own source, so mocks and timers come in the same order.vi.resetModules(), a package that is kept keeps the exports of a built-in module that a later file mocks, and the factory of a hoisted mock over a package is called when something asks for it.bun test.Measured
Linux x64, with the first commit of oven-sh/WebKit#1040, not by the authors of the changes:
bun run= 1bun run= 1Known issues
#48617 lists what main has. Of it, the two builds that stop (two arms that store one parameter into an element; the remainder of a product of BigInts) are fixed here, a direct
evalin strict code of a string that strict code refuses throws, accessors that are defined one after the other each answer by their own definition, the notice aboutimport deferof a mocked CommonJS file is printed again, and fewer files that the program only writes are warned about (TS100098). Not on main:Bun.serve()that is aBun.file()of a file embedded by--assetor an import keeps the build's bytes when the program writes or removes that file later by a__dirnamepath, and answers aRangerequest with the whole file (status 200).eval()is given depends on a variable that is changed only in the default of a parameter of a function with an empty body, the program ends with an error (exit 134).(write = Bun.write) => write(path.join(__dirname, "one.txt")), and the caller passes one that reads:ENOENT.--aot-harden=proof-checksOn main too, not listed there:
fororwhileloop changes a number in a variable of the file or of an enclosing function, and then ends with the engine'sTypeErrorfor a null or undefined element, the last changes are lost, and code that catches the error outside the function reads an older number.returnon the generator's prototype under a computed name, leaving the loop calls the built-inreturn().CI
The engine is oven-sh/WebKit#1040 as merged, 6fc709c298409fe433a60f6aff328a45a8974532. The bound of the compiler's bytes in
test/bundler/compile-aot-size.test.tsis restated; those of executables stay. Linux x64, by a local build:--trim