Skip to content

build: run create-hash-table.ts and generate-jssink.ts with cfg.jsRuntime - #41309

Merged
dylan-conway merged 11 commits into
mainfrom
robobun/c7912907/codegen-hash-table
Sep 4, 2026
Merged

dylan-conway merged 11 commits into
mainfrom
robobun/c7912907/codegen-hash-table

Conversation

@robobun

@robobun robobun commented Sep 4, 2026

Copy link
Copy Markdown
Collaborator

Stacked on #41308. The diff is against that branch.

Problem

  • create-hash-table.ts runs perl with spawn from the bun module. Node has no such module.
  • generate-jssink.ts runs create-hash-table.ts as a second process with process.execPath run <script>. Under node, run is not a subcommand. The script also ignored the exit code of that process.

Fix

  • create-hash-table.ts runs perl with spawnSync from node:child_process. It exports createHashTable(input, output), and runs it from the command line when import.meta.main is set.
  • generate-jssink.ts calls createHashTable() in-process. A perl failure now fails the step.
  • The JSSink edge and the eleven *.lut.h edges move to the codegen rule.
  • Verified: node and bun write the same 141 codegen files as main, on Linux and on Windows.

Background

  • A .lut.h file holds the JSC property tables for one C++ file. The perl script create_hash_table reads the @begin ... @end blocks from the source and writes the tables.
  • TARGET_PLATFORM picks the #if OS(...) blocks that stay. The codegen rule sets it for the process, so the in-process call reads the same value.
Notes
  • import.meta.main needs node 24.2 or later. CI uses node 26.3.0. generate-string-map.ts already depends on it.
  • On Windows, spawnSync("perl") finds perl.exe on PATH, as Bun.spawn did.

…g.jsRuntime

The codegen rule now runs its script with cfg.jsRuntime, the runtime
that runs configure. The new codegen_bun rule keeps bun for the scripts
that still need it. These two scripts run under node, and both runtimes
write the same files.

The root package.json has no "type" field, so node prints
MODULE_TYPELESS_PACKAGE_JSON once for each codegen process. The node
command hides that warning.
Node reads an absolute Windows path in import() as a URL with a
drive-letter scheme, and fails with ERR_UNSUPPORTED_ESM_URL_SCHEME.
emitCodegen() assigns cppAll from cppHeaders, cppSources, and
bindgenV2Cpp after the steps run, so a header pushed to cppAll is lost.
Node needs the file extension in a relative import. Node and bun both
link zstd 1.5.7, so both write the same compressed bytes.
readdirRecursive() now sorts the entries of each directory. Node sorts
readdirSync() results and bun does not, so the audit comment at the end
of generated_host_exports.rs listed tied files in a different order
under each runtime. The thunks were already sorted by symbol.
…time

create-hash-table.ts runs perl with spawnSync from node:child_process,
not spawn from bun. It now exports createHashTable(), and
generate-jssink.ts calls it in-process. Before, generate-jssink.ts
spawned a second runtime with "run" and ignored its exit code.

@claude claude Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM — mechanical codegen_bun → codegen migration following the documented path in scripts/build/CLAUDE.md.

What was reviewed:

  • create-hash-table.ts: Bun.spawn → spawnSync with proc.error thrown and status !== 0 checked; output post-processing and writeIfNotChanged unchanged, so restat = 1 idempotency holds.
  • import.meta.dirname / import.meta.main / .ts-suffixed import already used by sibling codegen scripts, so no new Node-version constraint introduced here.
  • generate-jssink.ts now calls createHashTable in-process; TARGET_PLATFORM is read at module load from the same env the codegen rule sets, so the OS-block preprocessing still matches the target.
Extended reasoning...

Overview

This PR migrates two codegen steps — create-hash-table.ts (the eleven *.lut.h object-LUT edges) and generate-jssink.ts (the JSSink edge) — from the Bun-only codegen_bun ninja rule to the runtime-agnostic codegen rule. create-hash-table.ts is refactored from a top-level-await Bun script into an exported synchronous createHashTable(input, output) function using child_process.spawnSync, guarded by if (import.meta.main) for CLI use. generate-jssink.ts drops its Bun.spawnSync([bun, "run", ...]) subprocess (which ignored the child's exit code) in favor of an in-process call. scripts/build/codegen.ts updates the two emitters to use rule: "codegen" and drops the "run" subcommand from the args.

Security risks

None. This is build-time tooling that runs perl over in-tree source files to generate JSC property hash tables. No network, no untrusted input, no runtime code paths touched.

Level of scrutiny

Low-to-moderate. The change is mechanical and directly implements the migration described in scripts/build/CLAUDE.md ("Move a script to the codegen rule once it runs under node, and check that both runtimes write the same output"). The output-shaping logic — the @ begin…@ end extraction, #if OS(...) stripping, and post-perl regex cleanup — is byte-for-byte unchanged; only the process-spawn plumbing and module shape changed. The Node-compat idioms used (import.meta.dirname, import.meta.main, .ts import suffix) are already established in generate-string-map.ts, bundle-functions.ts, and several other codegen files, so this introduces no new runtime-version floor.

Other factors

Error handling is strictly improved: the old generate-jssink.ts discarded the subprocess exit code, so a perl failure would silently produce a stale or missing .lut.h; the in-process call now hits process.exit(1) on nonzero status and throws on spawn error (e.g. perl missing or maxBuffer exceeded). TARGET_PLATFORM continues to flow correctly because both callers run under the codegen rule, which sets it in the environment, and the module reads it once at load. No CODEOWNERS entry covers scripts/build/ or src/codegen/, and there are no prior reviews on the PR. The bug hunt ran to dry_streak with no findings.

Base automatically changed from robobun/c7912907/codegen-host-exports to main September 4, 2026 04:12
@coderabbitai

coderabbitai Bot commented Sep 4, 2026

Copy link
Copy Markdown
Contributor

Review Change Stack

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info
⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: ASSERTIVE

Plan: Essentials

Run ID: e489aa21-2bb5-4fe5-b04a-44f052b7bc03

📥 Commits

Reviewing files that changed from the base of the PR and between 1719834 and 8eb66a0.

📒 Files selected for processing (3)
  • scripts/build/codegen.ts
  • src/codegen/create-hash-table.ts
  • src/codegen/generate-jssink.ts

Included review availability: Your plan provides up to 10 included reviews per hour; 5 remain after this review.


Walkthrough

Changes

The hash-table generator now exports a synchronous function. JSSink generation calls it directly. JSSink and object LUT build rules now use the generic codegen runtime.

Codegen integration

Layer / File(s) Summary
Synchronous hash-table generation
src/codegen/create-hash-table.ts, src/codegen/generate-jssink.ts
createHashTable handles preprocessing and process results synchronously. JSSink generation calls it directly with LUT paths.
Generic codegen build rules
scripts/build/codegen.ts
JSSink and object LUT generation use the generic codegen rule and invoke the hash-table script directly.

Suggested reviewers: jarred-sumner

Merge Risk: ⚪ Minimal · up to 8eb66

This change moves hash-table and JSSink generation to the configured JavaScript runtime while preserving synchronous failure propagation and direct CLI support. No merge-blocking risk remains.

🚥 Pre-merge checks | ✅ 4
✅ Passed checks (4 passed)
Check name Status Explanation
Title check ✅ Passed The title clearly identifies the main change: running the hash-table and JSSink generators with cfg.jsRuntime.
Description check ✅ Passed The description explains the problem, fix, affected build rules, verification results, and relevant background. It does not use the exact template headings, but it provides the required change summary…
Linked Issues check ✅ Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check ✅ Passed Check skipped because no linked issues were found for this pull request.

Comment @coderabbitai help to get the list of available commands.

@dylan-conway
dylan-conway merged commit edc668c into main Sep 4, 2026
9 of 10 checks passed
@dylan-conway
dylan-conway deleted the robobun/c7912907/codegen-hash-table branch September 4, 2026 04:19

@claude claude Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM — mechanical codegen_bun → codegen migration following the documented pattern, and the in-process createHashTable() call now propagates perl failures that the old Bun.spawnSync silently swallowed.

What was reviewed:

  • spawnSync with input:/encoding: "utf8" preserves the old stdin-pipe/stdout-text semantics; proc.error and non-zero status both fail the step.
  • TARGET_PLATFORM is set by codegenCommand for both rules, so the module-level env read still sees the right value when called in-process from generate-jssink.ts.
  • Ninja edge inputs (script, hashTableScript, perlScript) and writeIfNotChanged are unchanged, so restat incrementality is preserved.
  • import.meta.dirname / import.meta.main / .ts import extension match the Node-compat conventions already used by sibling migrated scripts.
Extended reasoning...

Overview

This PR migrates two codegen steps — JSSink generation and the eleven object-LUT .lut.h files — from the Bun-only codegen_bun ninja rule to the runtime-agnostic codegen rule. src/codegen/create-hash-table.ts is refactored from a top-level-await script using Bun.spawn into an exported synchronous createHashTable() using Node's child_process.spawnSync, with import.meta.dir → import.meta.dirname, a .ts extension on the helpers import, and CLI dispatch gated on import.meta.main. src/codegen/generate-jssink.ts now calls createHashTable() in-process instead of spawning bun run create-hash-table.ts as a subprocess. scripts/build/codegen.ts swaps the two edges to rule: "codegen" and drops the "run" arg accordingly.

Security risks

None. This is build-time tooling that shells out to a vendored perl script over repository source files; no untrusted input, network, auth, or crypto is involved. The spawnSync("perl", [...]) call uses an argv array (no shell interpolation) and the input path comes from ninja-declared build edges.

Level of scrutiny

Low-to-moderate. The change is mechanical and follows the exact migration pattern documented in scripts/build/CLAUDE.md ("Move a script to the codegen rule once it runs under node"). Sibling scripts in the same stacked series (generate-string-map.ts, generate-node-errors.ts, compress-embed.ts, generate-host-exports.ts) have already been migrated the same way. I verified that both codegen and codegen_bun rules share codegenCommand, so the TARGET_PLATFORM/TARGET_ARCH env vars are set identically — the module-level process.env.TARGET_PLATFORM read in create-hash-table.ts sees the correct value whether invoked as a CLI or imported by generate-jssink.ts. The args shape ([script, ...] without "run") matches every other codegen-rule edge in the file.

Other factors

The refactor is a net correctness improvement: the old generate-jssink.ts called Bun.spawnSync and discarded its result, so a perl failure would produce a stale/missing JSSink.lut.h without failing the build. The new in-process call hits process.exit(1) on non-zero perl status. writeIfNotChanged is retained, so ninja's restat = 1 optimization still short-circuits downstream C++ recompiles when LUT output is byte-identical. Ninja dependency declarations (hashTableScript, perlScript as inputs) are unchanged. No CODEOWNERS cover these paths, and there are no outstanding CHANGES_REQUESTED reviews. The bug hunt exited on dry_streak with no findings.

dylan-conway added a commit that referenced this pull request Sep 4, 2026
Stacked on #41309. The diff is against that branch.

### Problem
- `cppbind.ts` runs under bun only. When `@lezer/cpp` is missing, it
runs `process.argv[0] install` and then runs itself again. Under node,
`argv[0]` is node.
- Node cannot strip a parameter property (`constructor(public position:
Srcloc)`), and it has no `Bun.stringWidth()`.

### Fix
- Remove the self-install. The cppbind edge depends on the root install
stamp, so `@lezer/cpp` is there before the script runs. The imports are
static now.
- `PositionedErrorClass` assigns `position` in its constructor.
`LineInfo` drops the `source` field, which nothing reads.
- The caret under an error uses `before.length`. The text before the
caret is C++ source, which is ASCII.
- Verified: node and bun write the same `cpp.rs` as main, on Linux and
on Windows. A bad `ZIG_EXPORT` tag prints the same error under both
runtimes.

### Background
- `cppbind.ts` scans the C++ files for `[[ZIG_EXPORT]]` functions and
writes their Rust declarations to `cpp.rs`.
- The self-install came from #21439, before the build listed the root
install as an input of this step.

<details><summary>Notes</summary>

- Without `node_modules`, a manual run now fails with node's or bun's
error that `@lezer/cpp` is missing.

</details>

---------

Co-authored-by: autofix-ci[bot] <114827586+autofix-ci[bot]@users.noreply.github.com>
Co-authored-by: Dylan Conway <dylan.conway567@gmail.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants