Repository navigation
build: run create-hash-table.ts and generate-jssink.ts with cfg.jsRuntime - #41309
Conversation
…g.jsRuntime The codegen rule now runs its script with cfg.jsRuntime, the runtime that runs configure. The new codegen_bun rule keeps bun for the scripts that still need it. These two scripts run under node, and both runtimes write the same files. The root package.json has no "type" field, so node prints MODULE_TYPELESS_PACKAGE_JSON once for each codegen process. The node command hides that warning.
Node reads an absolute Windows path in import() as a URL with a drive-letter scheme, and fails with ERR_UNSUPPORTED_ESM_URL_SCHEME.
emitCodegen() assigns cppAll from cppHeaders, cppSources, and bindgenV2Cpp after the steps run, so a header pushed to cppAll is lost.
Node needs the file extension in a relative import. Node and bun both link zstd 1.5.7, so both write the same compressed bytes.
readdirRecursive() now sorts the entries of each directory. Node sorts readdirSync() results and bun does not, so the audit comment at the end of generated_host_exports.rs listed tied files in a different order under each runtime. The thunks were already sorted by symbol.
…time create-hash-table.ts runs perl with spawnSync from node:child_process, not spawn from bun. It now exports createHashTable(), and generate-jssink.ts calls it in-process. Before, generate-jssink.ts spawned a second runtime with "run" and ignored its exit code.
There was a problem hiding this comment.
LGTM — mechanical codegen_bun → codegen migration following the documented path in scripts/build/CLAUDE.md.
What was reviewed:
create-hash-table.ts:Bun.spawn→spawnSyncwithproc.errorthrown andstatus !== 0checked; output post-processing andwriteIfNotChangedunchanged, sorestat = 1idempotency holds.import.meta.dirname/import.meta.main/.ts-suffixed import already used by sibling codegen scripts, so no new Node-version constraint introduced here.generate-jssink.tsnow callscreateHashTablein-process;TARGET_PLATFORMis read at module load from the same env thecodegenrule sets, so the OS-block preprocessing still matches the target.
Extended reasoning...
Overview
This PR migrates two codegen steps — create-hash-table.ts (the eleven *.lut.h object-LUT edges) and generate-jssink.ts (the JSSink edge) — from the Bun-only codegen_bun ninja rule to the runtime-agnostic codegen rule. create-hash-table.ts is refactored from a top-level-await Bun script into an exported synchronous createHashTable(input, output) function using child_process.spawnSync, guarded by if (import.meta.main) for CLI use. generate-jssink.ts drops its Bun.spawnSync([bun, "run", ...]) subprocess (which ignored the child's exit code) in favor of an in-process call. scripts/build/codegen.ts updates the two emitters to use rule: "codegen" and drops the "run" subcommand from the args.
Security risks
None. This is build-time tooling that runs perl over in-tree source files to generate JSC property hash tables. No network, no untrusted input, no runtime code paths touched.
Level of scrutiny
Low-to-moderate. The change is mechanical and directly implements the migration described in scripts/build/CLAUDE.md ("Move a script to the codegen rule once it runs under node, and check that both runtimes write the same output"). The output-shaping logic — the @ begin…@ end extraction, #if OS(...) stripping, and post-perl regex cleanup — is byte-for-byte unchanged; only the process-spawn plumbing and module shape changed. The Node-compat idioms used (import.meta.dirname, import.meta.main, .ts import suffix) are already established in generate-string-map.ts, bundle-functions.ts, and several other codegen files, so this introduces no new runtime-version floor.
Other factors
Error handling is strictly improved: the old generate-jssink.ts discarded the subprocess exit code, so a perl failure would silently produce a stale or missing .lut.h; the in-process call now hits process.exit(1) on nonzero status and throws on spawn error (e.g. perl missing or maxBuffer exceeded). TARGET_PLATFORM continues to flow correctly because both callers run under the codegen rule, which sets it in the environment, and the module reads it once at load. No CODEOWNERS entry covers scripts/build/ or src/codegen/, and there are no prior reviews on the PR. The bug hunt ran to dry_streak with no findings.
…gen-compress-embed # Conflicts: # scripts/build/configure.ts
…c7912907/codegen-hash-table
|
No actionable comments were generated in the recent review. 🎉 ℹ️ Recent review info⚙️ Run configurationConfiguration used: Path: .coderabbit.yaml Review profile: ASSERTIVE Plan: Essentials Run ID: 📒 Files selected for processing (3)
Included review availability: Your plan provides up to 10 included reviews per hour; 5 remain after this review. WalkthroughChangesThe hash-table generator now exports a synchronous function. JSSink generation calls it directly. JSSink and object LUT build rules now use the generic codegen runtime. Codegen integration
Suggested reviewers: Merge Risk: ⚪ Minimal · up to This change moves hash-table and JSSink generation to the configured JavaScript runtime while preserving synchronous failure propagation and direct CLI support. No merge-blocking risk remains. 🚥 Pre-merge checks | ✅ 4✅ Passed checks (4 passed)
Comment |
There was a problem hiding this comment.
LGTM — mechanical codegen_bun → codegen migration following the documented pattern, and the in-process createHashTable() call now propagates perl failures that the old Bun.spawnSync silently swallowed.
What was reviewed:
spawnSyncwithinput:/encoding: "utf8"preserves the old stdin-pipe/stdout-text semantics;proc.errorand non-zerostatusboth fail the step.TARGET_PLATFORMis set bycodegenCommandfor both rules, so the module-level env read still sees the right value when called in-process fromgenerate-jssink.ts.- Ninja edge inputs (
script,hashTableScript,perlScript) andwriteIfNotChangedare unchanged, sorestatincrementality is preserved. import.meta.dirname/import.meta.main/.tsimport extension match the Node-compat conventions already used by sibling migrated scripts.
Extended reasoning...
Overview
This PR migrates two codegen steps — JSSink generation and the eleven object-LUT .lut.h files — from the Bun-only codegen_bun ninja rule to the runtime-agnostic codegen rule. src/codegen/create-hash-table.ts is refactored from a top-level-await script using Bun.spawn into an exported synchronous createHashTable() using Node's child_process.spawnSync, with import.meta.dir → import.meta.dirname, a .ts extension on the helpers import, and CLI dispatch gated on import.meta.main. src/codegen/generate-jssink.ts now calls createHashTable() in-process instead of spawning bun run create-hash-table.ts as a subprocess. scripts/build/codegen.ts swaps the two edges to rule: "codegen" and drops the "run" arg accordingly.
Security risks
None. This is build-time tooling that shells out to a vendored perl script over repository source files; no untrusted input, network, auth, or crypto is involved. The spawnSync("perl", [...]) call uses an argv array (no shell interpolation) and the input path comes from ninja-declared build edges.
Level of scrutiny
Low-to-moderate. The change is mechanical and follows the exact migration pattern documented in scripts/build/CLAUDE.md ("Move a script to the codegen rule once it runs under node"). Sibling scripts in the same stacked series (generate-string-map.ts, generate-node-errors.ts, compress-embed.ts, generate-host-exports.ts) have already been migrated the same way. I verified that both codegen and codegen_bun rules share codegenCommand, so the TARGET_PLATFORM/TARGET_ARCH env vars are set identically — the module-level process.env.TARGET_PLATFORM read in create-hash-table.ts sees the correct value whether invoked as a CLI or imported by generate-jssink.ts. The args shape ([script, ...] without "run") matches every other codegen-rule edge in the file.
Other factors
The refactor is a net correctness improvement: the old generate-jssink.ts called Bun.spawnSync and discarded its result, so a perl failure would produce a stale/missing JSSink.lut.h without failing the build. The new in-process call hits process.exit(1) on non-zero perl status. writeIfNotChanged is retained, so ninja's restat = 1 optimization still short-circuits downstream C++ recompiles when LUT output is byte-identical. Ninja dependency declarations (hashTableScript, perlScript as inputs) are unchanged. No CODEOWNERS cover these paths, and there are no outstanding CHANGES_REQUESTED reviews. The bug hunt exited on dry_streak with no findings.
Stacked on #41309. The diff is against that branch. ### Problem - `cppbind.ts` runs under bun only. When `@lezer/cpp` is missing, it runs `process.argv[0] install` and then runs itself again. Under node, `argv[0]` is node. - Node cannot strip a parameter property (`constructor(public position: Srcloc)`), and it has no `Bun.stringWidth()`. ### Fix - Remove the self-install. The cppbind edge depends on the root install stamp, so `@lezer/cpp` is there before the script runs. The imports are static now. - `PositionedErrorClass` assigns `position` in its constructor. `LineInfo` drops the `source` field, which nothing reads. - The caret under an error uses `before.length`. The text before the caret is C++ source, which is ASCII. - Verified: node and bun write the same `cpp.rs` as main, on Linux and on Windows. A bad `ZIG_EXPORT` tag prints the same error under both runtimes. ### Background - `cppbind.ts` scans the C++ files for `[[ZIG_EXPORT]]` functions and writes their Rust declarations to `cpp.rs`. - The self-install came from #21439, before the build listed the root install as an input of this step. <details><summary>Notes</summary> - Without `node_modules`, a manual run now fails with node's or bun's error that `@lezer/cpp` is missing. </details> --------- Co-authored-by: autofix-ci[bot] <114827586+autofix-ci[bot]@users.noreply.github.com> Co-authored-by: Dylan Conway <dylan.conway567@gmail.com>
Stacked on #41308. The diff is against that branch.
Problem
create-hash-table.tsruns perl withspawnfrom thebunmodule. Node has no such module.generate-jssink.tsrunscreate-hash-table.tsas a second process withprocess.execPath run <script>. Under node,runis not a subcommand. The script also ignored the exit code of that process.Fix
create-hash-table.tsruns perl withspawnSyncfromnode:child_process. It exportscreateHashTable(input, output), and runs it from the command line whenimport.meta.mainis set.generate-jssink.tscallscreateHashTable()in-process. A perl failure now fails the step.JSSinkedge and the eleven*.lut.hedges move to thecodegenrule.Background
.lut.hfile holds the JSC property tables for one C++ file. The perl scriptcreate_hash_tablereads the@begin ... @endblocks from the source and writes the tables.TARGET_PLATFORMpicks the#if OS(...)blocks that stay. Thecodegenrule sets it for the process, so the in-process call reads the same value.Notes
import.meta.mainneeds node 24.2 or later. CI uses node 26.3.0.generate-string-map.tsalready depends on it.spawnSync("perl")findsperl.exeonPATH, asBun.spawndid.