Skip to content

build: run generate-host-exports.ts with cfg.jsRuntime - #41308

Merged
dylan-conway merged 8 commits into
mainfrom
robobun/c7912907/codegen-host-exports
Sep 4, 2026
Merged

dylan-conway merged 8 commits into
mainfrom
robobun/c7912907/codegen-host-exports

Conversation

@robobun

@robobun robobun commented Sep 4, 2026

Copy link
Copy Markdown
Collaborator

Stacked on #41307. The diff is against that branch.

Problem

  • generate-host-exports.ts runs under bun only. It reads import.meta.dir and imports ./helpers with no extension. Node supports neither.
  • Under node, generated_host_exports.rs was different. readdirRecursive() (src/codegen/helpers.ts:18) returns entries in the order of readdirSync(). Node sorts those entries. Bun returns the file system order.

Fix

  • readdirRecursive() sorts the entries of each directory by name. The other caller, internal-module-registry-scanner.ts, sorts its result again, so its output does not change.
  • The script uses import.meta.dirname and ./helpers.ts, and its edge moves to the codegen rule.
  • Verified: node and bun write the same generated_host_exports.rs, on Linux and on Windows. All other codegen files match main.

Background

  • generated_host_exports.rs holds the Rust thunks for the #[host_fn] exports. A comment at the end counts the extern "C" blocks in each file.
  • The thunks were already sorted by symbol. Only the comment depended on the order, for files with the same count.
Notes
  • Compared with main, the comment has the same 190 lines in a new order. No other line changes.
  • ls -f on this machine gives beta alpha gamma omega zeta mid. fs.readdirSync() gives that order in bun and the sorted order in node.

…g.jsRuntime

The codegen rule now runs its script with cfg.jsRuntime, the runtime
that runs configure. The new codegen_bun rule keeps bun for the scripts
that still need it. These two scripts run under node, and both runtimes
write the same files.

The root package.json has no "type" field, so node prints
MODULE_TYPELESS_PACKAGE_JSON once for each codegen process. The node
command hides that warning.
Node reads an absolute Windows path in import() as a URL with a
drive-letter scheme, and fails with ERR_UNSUPPORTED_ESM_URL_SCHEME.
emitCodegen() assigns cppAll from cppHeaders, cppSources, and
bindgenV2Cpp after the steps run, so a header pushed to cppAll is lost.
Node needs the file extension in a relative import. Node and bun both
link zstd 1.5.7, so both write the same compressed bytes.
readdirRecursive() now sorts the entries of each directory. Node sorts
readdirSync() results and bun does not, so the audit comment at the end
of generated_host_exports.rs listed tied files in a different order
under each runtime. The thunks were already sorted by symbol.

@claude claude Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM — mechanical migration following the documented codegen_bun → codegen path.

What was reviewed:

  • emitHostExports arg shape matches other codegen-rule edges (emitErrorCode, emitStringMaps, emitCompressedEmbeds): script as first arg, no "run".
  • import.meta.dirname and the .ts import extension both work in Bun and Node 24+ --experimental-strip-types.
  • The other readdirRecursive caller (internal-module-registry-scanner.ts:11) already .sort()s the flattened result, so the new per-directory sort changes nothing there; readdirRecursiveWithExclusionsAndExtensionsSync is a separate function and untouched.
Extended reasoning...

Overview

This PR moves generate-host-exports.ts from the bun-only codegen_bun ninja rule to the runtime-agnostic codegen rule so it can run under whichever runtime started configure (cfg.jsRuntime — node in CI, bun for bun bd). Three files change: scripts/build/codegen.ts swaps the rule name and drops the leading "run" arg (matching every other codegen-rule edge in the file); src/codegen/generate-host-exports.ts swaps import.meta.dir → import.meta.dirname and adds the .ts extension to the helpers import; src/codegen/helpers.ts sorts each directory's readdirSync entries by name so bun (filesystem order) and node (sorted) produce byte-identical generated_host_exports.rs.

Security risks

None. This is build-system plumbing with no user-facing surface, no network I/O, no auth/crypto/permissions, and no untrusted input. The sort comparator is a plain string compare over Dirent.name values from the local source tree.

Level of scrutiny

Low. scripts/build/CLAUDE.md explicitly documents this migration ("Move a script to the codegen rule once it runs under node, and check that both runtimes write the same output"), and the change follows it to the letter. The codegen rule invokes ${cfg.jsRuntime} $args directly, so dropping "run" is required and matches emitErrorCode/emitStringMaps/emitCompressedEmbeds. import.meta.dirname is standard in Node 20.11+ and supported by Bun; the .ts extension is required by Node's type-stripping loader and accepted by Bun. The per-directory sort is the correct fix for restat = 1 idempotency and the node/bun parity contract — without it, the trailing per-file counts comment in generated_host_exports.rs would reorder between runtimes.

Other factors

I checked that the only other caller of readdirRecursive (internal-module-registry-scanner.ts) already .sort()s its flattened result, so the added per-directory sort cannot change its output. readdirRecursiveWithExclusionsAndExtensionsSync (used by bindgen.ts and generate-js2native.ts) is a separate function and untouched. The new comment in helpers.ts names a real runtime divergence that would otherwise invite someone to delete the sort as redundant, so it earns its place under the repo's comment bar. No outstanding reviews or objections in the timeline; exit reason was dry_streak.

…gen-compress-embed

# Conflicts:
#	scripts/build/configure.ts
Base automatically changed from robobun/c7912907/codegen-compress-embed to main September 4, 2026 04:06
Comment thread src/codegen/helpers.ts Resolved
@dylan-conway
dylan-conway merged commit 1719834 into main Sep 4, 2026
9 of 10 checks passed
@dylan-conway
dylan-conway deleted the robobun/c7912907/codegen-host-exports branch September 4, 2026 04:12
@coderabbitai

coderabbitai Bot commented Sep 4, 2026

Copy link
Copy Markdown
Contributor

Review Change Stack

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info
⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: ASSERTIVE

Plan: Essentials

Run ID: 51f72462-893c-4a2c-89a2-5e782a6473c2

📥 Commits

Reviewing files that changed from the base of the PR and between c83c9c3 and 5878ab0.

📒 Files selected for processing (3)
  • scripts/build/codegen.ts
  • src/codegen/generate-host-exports.ts
  • src/codegen/helpers.ts

Included review availability: Your plan provides up to 10 included reviews per hour; 6 remain after this review.


Walkthrough

Changes

Host export generation

Layer / File(s) Summary
Runtime-compatible host export generation
scripts/build/codegen.ts, src/codegen/generate-host-exports.ts
The host-exports build edge uses the generic codegen rule and configured JavaScript runtime. The generator uses explicit .ts helper resolution and import.meta.dirname.
Deterministic directory traversal
src/codegen/helpers.ts
readdirRecursive sorts directory entries by name before recursive collection.

Suggested reviewers: jarred-sumner

Merge Risk: ⚪ Minimal · up to 5878a

This change runs host-export generation with the configured JavaScript runtime and makes traversal deterministic across Node and Bun. No current merge-readiness risk remains.

🚥 Pre-merge checks | ✅ 4
✅ Passed checks (4 passed)
Check name Status Explanation
Description check ✅ Passed The description clearly explains the problem, the fix, and verification results on Linux and Windows. It provides the required information from the repository template, although it uses different sect…
Title check ✅ Passed The title clearly identifies the main change: running generate-host-exports.ts with cfg.jsRuntime. It is concise and directly related to the pull request.
Linked Issues check ✅ Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check ✅ Passed Check skipped because no linked issues were found for this pull request.

Comment @coderabbitai help to get the list of available commands.

@claude claude Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code review found no issues

No high-confidence issues detected in this change.

dylan-conway pushed a commit that referenced this pull request Sep 4, 2026
…time (#41309)

Stacked on #41308. The diff is against that branch.

### Problem
- `create-hash-table.ts` runs perl with `spawn` from the `bun` module.
Node has no such module.
- `generate-jssink.ts` runs `create-hash-table.ts` as a second process
with `process.execPath run <script>`. Under node, `run` is not a
subcommand. The script also ignored the exit code of that process.

### Fix
- `create-hash-table.ts` runs perl with `spawnSync` from
`node:child_process`. It exports `createHashTable(input, output)`, and
runs it from the command line when `import.meta.main` is set.
- `generate-jssink.ts` calls `createHashTable()` in-process. A perl
failure now fails the step.
- The `JSSink` edge and the eleven `*.lut.h` edges move to the `codegen`
rule.
- Verified: node and bun write the same 141 codegen files as main, on
Linux and on Windows.

### Background
- A `.lut.h` file holds the JSC property tables for one C++ file. The
perl script `create_hash_table` reads the `@begin ... @end` blocks from
the source and writes the tables.
- `TARGET_PLATFORM` picks the `#if OS(...)` blocks that stay. The
`codegen` rule sets it for the process, so the in-process call reads the
same value.

<details><summary>Notes</summary>

- `import.meta.main` needs node 24.2 or later. CI uses node 26.3.0.
`generate-string-map.ts` already depends on it.
- On Windows, `spawnSync("perl")` finds `perl.exe` on `PATH`, as
`Bun.spawn` did.

</details>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants