Repository navigation
perf(vm): add an adaptive bounded compilation cache - #85
Merged
Merged
Conversation
Cache immutable source and serialized program bytecode per VM, with weak decoded-code and importer references. Promote repeated sources to full function bytecode, retain fresh execution state, and evict by a configurable byte budget to avoid JSC shared-cache churn on large module graphs.
steipete
force-pushed
the
claude/w97-vm-bounded-compile-cache
branch
from
October 3, 2026 02:36
69cf88c to
17ec8b2
Compare
steipete
marked this pull request as ready for review
October 3, 2026 03:42
steipete
added a commit
that referenced
this pull request
Oct 3, 2026
Runtime resolution currently discards selected malformed package metadata, allowing optional-dependency fallbacks that Node rejects. Retain package read/parse failures until resolution selects the package or its scope, then throw Node's error with the same code, class, path, and importer context. Explicit module extensions, nested scopes, unused conditions, and fields Node ignores keep their observed behavior. Selected dependency reads materialize both package maps and preserve Node's SyntaxError diagnostics, including UTF-16 context. CommonJS self lookup preserves lazy getters; #imports reads imports first. JSON-encoded maps in string fields follow Node's reader. The metadata reader follows Node's shallow package reader, rather than applying strict JSON validation to unused values. Runtime-specific escaped/duplicate-field semantics use a separate metadata view so Bun's bundler retains its existing field handling. ESM resolve-only lookups validate existing .js/.ts/extensionless scopes, with missing-file and explicit-format exemptions. The resolved-key distinction documented by oven-sh#44473 prevents reapplying ESM scope validation when Bun hands an already-resolved CommonJS file to its ESM loader; this does not port that PR's broader loader rewrite. Conditional target arrays continue after invalid/null alternatives and retain the final error. CommonJS missing-module diagnostics now retain parent filenames. This deliberately targets **Node 24.21** for unreadable selected metadata: it throws `ERR_INVALID_PACKAGE_CONFIG`, whereas 24.19 treated read failures as absent metadata. The compatibility documentation records that choice. This version difference is separate from OpenClaw's malformed dependency fixture: both 24.19 and 24.21 throw `ERR_INVALID_PACKAGE_CONFIG` for import and require, whether the dependency body exists or not (eight checks). Adapts the malformed-scope retention approach from oven-sh#33890 (closed, unmerged) and resolver error identity work from oven-sh#35711 (open). Neither is a complete upstream fix for current Node 24 package-reader behavior. Credit to @robobun and @cirospaciari. Validation of final head `0a08f0c3218d10a886b160ebc60eeb97ac847190`: - 944-case Node oracle: import/require across malformed JSON, field values, empty/missing/BOM metadata, explicit formats, nested scopes, and self references; the immutable final-head binary matches outcomes, codes, and normalized messages in all 944 cases. All 178 package-config errors have the expected Error name; Bun's existing ResolveMessage name remains unchanged for generic missing-module errors. - Final regression control: 87 failures on unpatched fork main across 138 tests. The final head passes 737 targeted tests, with 1 existing skip and 1 todo. All 12 Rust targets and formatting pass. The original 944-case oracle, expanded 376-case package-map oracle and 14 getter-order cases all match Node; 1,850 additional JSON diagnostic cases match. Local and required branch Codex P2 reviews are scoped-clean; the branch review uses merge-base `486288f80d`. - Final-head OpenClaw consumer comparison: conditions 40/44 → 42/44 (Node 44/44); interop 56/56 → 56/56; lazy-alias 24/24 → 24/24. The two remaining conditions failures are independently reproduced OpenClaw capture-adapter defects: missing retained symlink alias materialization and premature nested dependency capture during a compiler preview. No OpenClaw source changes. Final proof has zero skips and verifies the binary hash before and after execution. - Existing Linux #79 proof remains valid: lifetime 8/8 in Node and fork main, with end/close in all four socket combinations. It was not rerun for this change. The patch is rebased onto main `486288f80d` (#85), preserving the changelog append-only. Earlier surrounding execution also reproduced three unchanged `esModule-annotation.test.js` failures on the unpatched control; this PR does not claim that broader file is green. Both fork build/test CI lanes passed in https://github.com/openclaw/bun/actions/runs/37096840734 for exact head `0a08f0c3218d10a886b160ebc60eeb97ac847190`. The live merge gate confirms every non-skipped check is successful. Package-map resolution uses an explicit imports/exports context. Local workspace Clippy for Linux and the complete CI Rust lint workflow (Clippy, Mordant, Miri and vendored tests) pass on this exact head. Upstream submission: oven-sh#44512. Its adaptation uses upstream’s existing resolved-key handling and omits fork-only module-hook integration. Co-authored-by: Ciro Spaciari MacBook <ciro@anthropic.com> Co-authored-by: Dylan Conway <dylan.conway567@gmail.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Repeated
node:vmevaluations of large module graphs fall out of JavaScriptCore’s shared compilation cache near its 2,000-entry working set. This adds a bounded per-VM compilation cache, preserving fresh values and target globals while reusing context-independent bytecode acrossScript,compileFunction, and the run helpers.The cache activates after 1,750 distinct successfully compiled source fingerprints. Before activation, compilation takes the original JSC path followed by bounded fingerprint bookkeeping; it does not serialize or retain bytecode. Activation lasts for that VM’s lifetime.
BUN_VM_COMPILE_CACHE_THRESHOLD=0..2000tunes admission (default 1750).BUN_VM_COMPILE_CACHE_SIZEsets a decimal byte budget (default 256 MiB per VM); 0 disables both caching and tracking.Entries compare complete source text, filename/source URL, offsets, compilation kind/flags, lexical features, and dynamic-import callback identity. Explicit cachedData uses the existing validation path, and debugger/profiler modes bypass the layer. The byte LRU owns copied source/filename and immutable bytecode; decoded code and callbacks are weak references. The budget charges payloads, copied strings and a 1 KiB metadata reservation per entry. Allocator overhead, temporary encoding/decoding allocations and objects held by live Scripts/functions are outside this ownership budget. The bounded admission table is released at activation. No WebKit or dependency changes.
Measured trade-off
Linux c7a.24xlarge, captured async module wrappers, five randomized interleaved processes per size/runtime, eight passes per process; values are medians of process medians for passes 1–7. Main is 0731292. Node is 24.19.0.
A fixed 20-pair follow-up at 500 measured 6.947 → 7.193 ms (+3.54%), with paired 95% bootstrap candidate/main bounds 1.011–1.054: about 0.5 µs per compilation. An initial paired run had approximately 256 KiB higher median peak RSS; later RSS samples overlapped main. The cost is accepted in exchange for 72.5% and 77.1% shorter warm replay at 2,500/4,000 modules, where measured consumer files’ roughly 2,100-module graphs encounter pruning. Candidate/main timing bounds at those sizes are 0.272–0.280 and 0.228–0.232. Each interval uses 20,000 process-level bootstrap samples.
The candidate remains 32.7% slower than Node at 2,500 and 34.3% slower at 4,000. Node parity is not claimed. Main/candidate median process peak RSS in the five-run grid is 129,312/130,464 KiB at 500, 336,048/336,512 at 1,500, 873,540/863,280 at 2,500, and 1,530,288/1,374,156 at 4,000. These measurements are workload-specific.
Validation
The implementation passes 352 Bun VM tests (3 existing skips, 60 existing TODOs), 102/102 ported Node parallel/sequential VM tests, and an identical Node 24 semantic probe. New coverage checks admission, cache keying, stack traces and offsets, cached errors/data, fresh values/contexts, import identities, byte eviction/recency, and decoded-code lifetime. A parser-trace regression fails on unpatched Bun and passes with reuse after GC. Scoped-clean P2 review completed on head
17ec8b2453db2a4bd844b561884ba5d3be0b92c0against merge base3d1c40b8b39e4377a773b26d57440fdbc69bf0b0. Both fork native CI lanes pass on that exact head: run 37090421840. Darwin required one unchanged retry after an existing wall-clock timeout assertion missed its first deadline; the isolated case also passed unchanged. No test assertion was weakened. Formatting, source lint and JavaScript lint pass. The ancillary duplicate-PR bot cannot start because its authentication is not configured.The final cold consumer matrix passed on OpenClaw main ff96d47c3506c50123a555332e6a6cc576b2559f (Vitest 5.0.1). Main Bun is 3d1c40b; candidate is 17ec8b2. Node is 24.19.0. Each config/runtime has five interleaved cold runs on one AWS c7a.24xlarge lease: 30 arms, three pinned four-CPU slots, fresh filesystem overlays and compilation/transform caches, private HOME/state and short 0700 TMPDIRs. Both configs use four thread workers with
isolate:false. No dependency changes.Candidate/main median ratios (20,000 process-level bootstrap resamples):
Every file and every assertion name/status matches across all fifteen runs of each config: Discord 264 files, 3,187 passes, zero skips; auto-reply 233 files, 4,317 passes, one existing skip; zero failures. All 30 invocations exited successfully, published complete reports and left no namespace descendants. The three broker-dependent files also pass identically in their existing fork-based infra owner (27 assertions).
Consumer trade-off: Discord median time improves 10.9%; auto-reply improves 2.6%. Auto-reply median peak RSS rises 2.7% (17.25 → 17.72 GiB); Discord RSS is essentially unchanged (17.63 → 17.55 GiB). Both RSS ratio intervals overlap 1. The candidate remains 17.0% slower than Node on Discord, while auto-reply is 5.1% faster than Node. Node uses substantially less RSS in both configs.
The p95 is interpolated from five process observations. RSS is GNU time’s process/descendant high-water metric, not an aggregate cgroup measurement. Cold means fresh application caches/process state, not an OS page-cache flush. These bounds describe this host/workload. The microbenchmark constructs wrappers without executing module bodies; consumer runs include evaluation and test work, so their percentage gains differ.
Main advanced with #83 during the first matrix. The branch was rebased, both binaries rebuilt, and the entire 30-arm matrix restarted on the same lease. The interrupted old-baseline samples are preserved separately and excluded from the final cohort.
Engine follow-up
Upstream searches found related cached-data work, including oven-sh#41769, but no ordinary-compilation working-set fix to port. No upstream PR is being opened in this round. This may be useful upstream after broader evaluation; no upstream acceptance is implied.
Future engine support could provide (1) an exported retained-size/growth contract for complete unlinked-code graphs, including nested code, profile data and JIT allocations; or (2) ownership references that let decoders borrow storage safely from evictable owned bytecode. Either could reduce copying/decoding cost and needs independent lifetime and performance proof. The runtime docs describe the controls, bounds and measured trade-off; the changelog addition is append-only.