Repository navigation
node:module: implement synchronous module.registerHooks() (+49 tests) - #35690
cirospaciari wants to merge 12 commits into
Conversation
…on hooks [allow size] Adds Node's synchronous module customization hooks API (module.registerHooks, Node 23+, the supported successor to the deprecated off-thread module.register). Resolve and load hook chains run for require(), require.resolve(), static and dynamic import, and import.meta.resolve, with Node's contract: parentURL/conditions/importAttributes context, nextResolve/ nextLoad chaining, the shortCircuit requirement, format overrides (module/commonjs/json/typescript variants), builtin observation with node: URLs and null default source, deregister(), and ERR_INVALID_RETURN_PROPERTY_VALUE / ERR_UNKNOWN_MODULE_FORMAT validation. The chain and validation logic is a port of Node's lib/internal/modules/customization_hooks.js into a new internal module. The native side consults it from the resolver funnel (resolveMaybeNeedsTrailingSlash and its jsc_hooks twin, before the builtin alias fast path), from the module loader (transpileFile, before reading a module off disk), and from fetchBuiltinModule for builtins - all gated on hook counters mirrored onto VirtualMachine so the hook-free path only pays an integer compare. The chain's default resolve step re-enters Bun's native resolution under a reentrancy flag so it cannot recurse into the hooks. module.register() stays a no-op but now emits Node's DEP0205 deprecation warning (once per process, suppressed by --no-deprecation) pointing at registerHooks(); covered by tests in node-module-module.test.js.
Vendors byte-verbatim from Node v26.3.0: - test/js/node/test/module-hooks/: 47 of the 63 synchronous module.registerHooks() tests (plus their fixtures), all passing. The suite directory is added to the node-test runner's inclusion list; a deliberately failing canary run confirmed the runner executes the new directory. - test/js/node/test/es-module/: test-esm-import-meta-resolve-hooks.mjs and test-import-preload-require-cycle.js (with the import-require-cycle fixtures). The es-module runner line matches the pending es-module suite branch, so the two merge cleanly. Not vendored, with reasons: custom-conditions* (3) need per-resolution conditions plumbed into the native resolver; load-builtin-override-* (3) replace a builtin's implementation via load-hook format override, which is unsupported; *inline-typescript* and preload (5) depend on the 11 MB fixtures/snapshot/typescript.js fixture; load-async-and-sync and require-esm (2) need the off-thread module.register() protocol; builtin-require and load-builtin-require (2) require node:sea; create-require-with-url (1) needs URL-string specifiers preserved through createRequire(url); test-esm-import-attributes-identity.mjs needs the module map keyed by import attributes; test-esm-register-deprecation.mjs asserts Node's [DEP0205] stderr format and --throw-deprecation exit behavior, which Bun's warning printer does not produce.
|
Found 4 issues this PR may fix:
🤖 Generated with Claude Code |
|
This PR may be a duplicate of:
🤖 Generated with Claude Code |
…aude/node-esm-loader-hooks # Conflicts: # src/jsc/bindings/ErrorCode.ts
… fallout - Restore src/jsc/bindings/BunHeapProfiler.h (deleted by #36500 on main, but still needed by GeneratedJS2Native.h for the $newCppFunction calls added on this branch). - VirtualMachine.rs / jsc_hooks.rs: resolve_maybe_needs_trailing_slash now takes ResolveMode, not is_esm/is_user_require_resolve; derive the two bools from `mode` for Bun__runModuleResolveHooks. Use `&raw const` for the pointer args. - web_worker.rs: parent_ref binding was lost in the merge; read heap_profiler_config via `(*parent)` like the neighbouring fields. - permission.rs: switch to bun_threading::RwLock (no poisoning) and bun_core::env_var::NODE_OPTIONS per clippy disallowed-types/methods. - clap Diagnostic fields pub (read by bun_runtime::cli::Arguments). - path.rs resolve_posix_t pub(crate) (called from permission.rs). - Minor clippy: then_some, contains(), SAFETY comment placement, unreachable_pub on exec_check.
…aude/node-esm-loader-hooks # Conflicts: # src/jsc/bindings/BunHeapProfiler.h # src/runtime/cli/run_command.rs # src/runtime/permission.rs
…aude/node-esm-loader-hooks
|
Cross-reference: this resolves #27369 ("Bun does not support One small thing the stub had that this does not: the |
Implement synchronous resolve/load hooks with Node-compatible parent and condition overrides, URL identity, source formats, dynamic import attributes, and deregistration. Shared static edges and native builtin handoffs retain their correct records and hook calls. Unsafe separate-record requests and static returned-type identity throw explicit documented errors. Tests expose the remaining JSC limits and Bun's completed import-to-require reuse. Validated the 97-case Node oracle, both native fork CI lanes, 49 upstream tests, 984 surrounding tests, 12 Rust targets, pre-change OpenClaw interoperability, all 112 update tests, and published 2026.9.7 Gateway startup. The zero-hook cold-load comparison passed: +0.354% median versus exact base within 0.793% declared noise across 30 interleaved runs per binary. Ports oven-sh#35690 Co-authored-by: Ciro Spaciari MacBook <ciro@anthropic.com>
…URLs (#78) `module.registerHooks()` exposed Bun's bare native replacement names (such as `ws`) as load-hook URLs. A hook that calls `new URL(url)`, including tsx, throws `ERR_INVALID_URL`. Opaque plugin namespaces had the same URL defect at resolution. Keep the hook URL separate from the native module identity. Installed replacement packages expose their resolved file URL with Node-compatible package conditions; `nextLoad()` still serves Bun's implementation. Missing packages use documented `bun-builtin:` URLs. Opaque native plugin IDs use `bun-virtual:` URLs. Generated identities survive deregistration, while plugin providers and parent importers retain their native IDs. The mapping escapes native namespaces that collide with either generated scheme. Programs that never register hooks retain their existing resolution and replacement behavior. This covers all seven npm replacements, their package subpaths and Next.js aliases, bare `ffi` and `bun`, and Node builtins including `node:sys` and exposed internals. Package-file lookup ignores cached tsconfig aliases and restores ordinary resolver settings afterward. The fork changelog is append-only, and the compatibility documentation describes the schemes and native delegation. The final source passes 348 focused tests (the published base passes 100 and fails 248), 49 imported hook tests, and 1,332 surrounding module tests with no failures. The unchanged 97-case Node 24 oracle remains 82 exact matches plus 15 documented divergences, with zero unexpected results. All 1,867 observed URLs in the final inventory parse in Node 24. Published-package probes match Node's first resolve/load URL and format for all 18 Node-successful resolutions. Full-branch P2 Codex review is scoped-clean. Both Linux x64 and Darwin arm64 pass the final consumer qualification: iOS is 10 runs of 8/8 on each platform versus 10 runs of 4/8 on the published base; historical/current interop is 68/68 and 56/56; updates are 112/112. The wrapper remains 264 passed, two inherited failures, and one skip, with identical failure names, normalized full messages, and diagnostic objects. The published 2026.9.7 Gateway starts with plugins and shared module identity, returns health 200, exits zero, and closes its listener. Every consumer uses isolated state and the exact final runtime. Darwin test starts stayed below load 30 and above 82 GiB free. Both native lanes are green on exact head `65205c81df0b2d195b7192f1d2dd7bb656238993`: [build and test](https://github.com/openclaw/bun/actions/runs/37064039399). [Rust CI](https://github.com/openclaw/bun/actions/runs/37064039381) is also green. The independent Linux proof passes strict Clippy and all 12 Rust target checks. Upstream searches found no existing fix for this defect. The original hooks implementation is tracked at oven-sh#35690; this change is confined to the fork and changes neither WebKit nor workflows.
|
@cirospaciari @robobun What is needed to finish, test, approve and merge this? Can I help? Blocks running Cloudflare |
|
Tried this PR's head (0ae6e5f,
import { registerHooks } from 'node:module'
registerHooks({
resolve(specifier, context, nextResolve) {
if (!specifier.startsWith('repro://')) return nextResolve(specifier, context)
const { specifier: inner, parentURL } = JSON.parse(specifier.slice('repro://'.length))
return nextResolve(inner, { ...context, parentURL })
},
})
const spec = 'repro://' + JSON.stringify({ specifier: './target.mjs', parentURL: new URL('./sub/', import.meta.url).href })
console.log((await import(spec)).default)Node 24.22.2 prints |
|
Sharing a proven fork integration follow-up for this canonical synchronous-hooks implementation. The existing fork work is tracked in openclaw/bun#77 and #78; this is not a competing hooks PR. After the upstream sync, the fork's filelike resolver predicate stopped treating bare names as filelike. That correctly preserves ordinary builtin exclusion and nested plugin recursion guards, but also bypassed a Bun.plugin replacement of bare ws during a registered Node hook's next() default resolution. The existing virtual-URL fixture then loaded native ws instead of the plugin's value. The signed two-file correction admits bare names only in the existing synchronous default-hook scope, while outside an onResolve callback. It retains the builtin-alias veto and prefixed-builtin handling. Using that scope rather than hook counts also preserves deregistration during an active hook. Actual native Debug bun bd red/green proof runs the unchanged 102-case virtual-URL fixture: ten replace-native-name failures before the correction, then 102 passing cases with it. Fifteen selected plugin boundary cases also pass, including ordinary builtin exclusion, nested-helper resolution, bare-name loading and registry/callback behavior. Actual Debug assertion/ASAN and AOT compiler inputs were recorded; source restoration and all 18 proof-archive links were verified. Proof artifact13137121579 has ZIP SHA-256 a29b2b64cecd853c6a745a2a28a4e3b4315442e962c442a63ba5af5c2a035c67. This is a fork integration contract, not a claim that upstream main is broken or that this draft is qualified. Upstream3f3a9ff87cdccf3d32d7ec6b648c664eddaae840 lacks the synchronous-hook binding, customization module and registerHooks export, so porting needs review against the eventual hooks implementation and its own regression run. Completed final matched Release and OpenClaw/Birdclaw qualification are recorded in openclaw/WebKit#45. No shipping release or pin is promoted by this handoff. Final fork qualification is now complete, and the engine controls are landed in openclaw/WebKit#45, commit 94019e6, with the upstream sync history retained. Public signed Bun 30f563c passes Linux Release qualification on engine b44e228: all 44 selected files, 91 namespace cases, the 85-file sync selection, four AOT compiler/runtime cases and nine compile-cache exit cases. Runtime-cache migration proves format 36 to 52, with 56/56 candidate position checks on replacement and marked-output warm replay. Native Darwin passes 53 selected files and four AOT compiler/runtime cases on exact engine b801008. The later Linux-only cache-check correction leaves production engine and Darwin control inputs identical; that reuse was verified and the older proof is not relabeled. Same-binary downstream qualification also passes: OpenClaw's ten packaged-runtime steps and two SQLite cases, plus Birdclaw's twelve tests and one CLI/server runtime iteration with its logical database digest preserved. These remain fork-side results, not qualification of the current upstream-only stack or this PR's wider callers. Earlier full Debug timing failures and filesystem lock timeouts remain recorded. Immutable shipping artifacts, a coherent Bun manifest promotion and fresh stock CI are still prerequisites for openclaw/bun#221; this handoff publishes no release or shipping pin. |
|
Anything holding back this PR? |
Implements the synchronous module customization hooks API —
module.registerHooks()(Node 23+, the supported successor tomodule.register()) — and vendors the upstreamtest/module-hooks/suite that exercises it.What this does
module.registerHooks({ resolve, load })now works in Bun for bothrequire()andimport:require,require.resolve, static/dynamicimport,import.meta.resolve), with Node's context contract (parentURL,conditions,importAttributes),nextResolvechaining, theshortCircuitrequirement, andERR_INVALID_RETURN_PROPERTY_VALUEvalidation.{url, context.format}contract and can replace the source; returnedformat(module,commonjs,json, the*-typescriptvariants) maps onto Bun's loader + module-type pipeline. Builtins are observed withnode:-prefixed URLs and anulldefault source (source overrides for builtins are ignored, like Node ignores them forbuiltinformat).hook.deregister(), hook chaining/merged contexts, and virtual specifiers (e.g.virtual:xids produced by hooks) work.module.register()stays a no-op but now emits Node'sDEP0205deprecation warning pointing atregisterHooks().How it's wired
src/js/internal/modules/customization_hooks.ts— port of Node'slib/internal/modules/customization_hooks.jschain/validation logic, plus the two Bun-facing entry points (runResolveHooksBun,runLoadHooksBun).VirtualMachine::resolve_maybe_needs_trailing_slash, and itsjsc_hooks.rstwin) consults the JS resolve chain before the builtin-alias fast path; the module loader (transpile_file) consults the load chain before reading a module off disk;fetch_builtin_moduleconsults it for builtins. All gated on plain integer counters mirrored ontoVirtualMachine, so the hook-free hot path only pays an integer compare.fs.readFileSync.ERR_INVALID_RETURN_PROPERTY_VALUEandERR_UNKNOWN_MODULE_FORMAT(message builders match Node's text).Tests
Vendored byte-verbatim from Node v26.3.0 (each verified failing on system Bun and passing on this build):
test/js/node/test/module-hooks/— 47 of the 63 sync-hooks tests (the suite is added to the CI runner's inclusion list; a canary run verified the runner executes the new directory).test/js/node/test/es-module/—test-esm-import-meta-resolve-hooks.mjsandtest-import-preload-require-cycle.js(the runner line matches the es-module suite PR; trivial merge overlap).Not covered (dropped, with reasons)
module.register()protocol — the 20test-esm-loader-*failures in the es-module suite all depend on it (--experimental-loader/--loaderchildren or directregister()calls);register()is deprecated upstream (DEP0205). Tests: gated on that feature.custom-conditions*(3) — per-resolutioncontext.conditionsoverrides need conditions plumbed through the native resolver per-call.load-builtin-override-{commonjs,json,module}(3) — replacing a builtin's implementation via load-hook format override is unsupported.*-inline-typescript*+preload(5) — depend on the 11 MBfixtures/snapshot/typescript.jsupstream fixture.load-async-and-sync,require-esm(2) — need off-threadregister().builtin-require,load-builtin-require(2) — requirenode:sea, which Bun does not implement.create-require-with-url(1) — needs URL-string specifiers preserved throughcreateRequire(url).test-esm-import-attributes-identity.mjs(1) — needs the ES module map keyed by import attributes.test-esm-register-deprecation.mjs(1) — the DEP0205 warning is implemented (with Bun-native tests intest/js/node/module/node-module-module.test.js), but the upstream test asserts Node's[DEP0205]stderr format and--throw-deprecationexit behavior, which Bun's warning printer does not produce.