Skip to content

ci(asan): enable JSC validateGraphAtEachPhase on ASAN test lanes - #34665

Closed
robobun wants to merge 1 commit into
mainfrom
claude/farm/9a031f57/asan-dfg-validation
Closed

robobun wants to merge 1 commit into
mainfrom
claude/farm/9a031f57/asan-dfg-validation

Conversation

@robobun

@robobun robobun commented Jul 19, 2026 •

Copy link
Copy Markdown
Collaborator

What

Set BUN_JSC_validateGraphAtEachPhase=1 in scripts/runner.node.mjs for ASAN test processes (both the bun test path and the Node compat test path).

Why

There is a cluster of ~5900 Sentry events over the last 90 days in JITWorklistThread -> DFG::Plan::compileInThreadImpl -> CFAPhase::performBlockCFA -> AbstractInterpreter<InPlaceAbstractState>::executeEffects, grouped across BUN-2YP4 / BUN-2VC6 / BUN-2WRJ / BUN-2WR9 / BUN-2XV5 / BUN-36RW / BUN-3QAE / BUN-36ZX and a few more. ~98% Windows, but confirmed on Linux and macOS too.

Disassembly of the 1.3.14 windows-x64 PDB pins the BUN-2YP4 crash to:

mov  rax, [r13]          ; r13 = &AbstractInterpreter::m_state (this+0x18)
movzx eax, [rax+0x50]    ; m_state->m_isValid

with fault addresses of the form (N<<32) | 0x150, i.e. the 8-byte m_state reference on the CFAPhase stack has had its low dword overwritten (to 0x100, which matches asArrayModes(NonArrayWithContiguous)) and its high dword set to a small monotone integer (matches AbstractValueClobberEpoch). That is exactly the {m_arrayModes; m_effectEpoch} pair inside an AbstractValue, so the shape is consistent with an AbstractValue write landing one slot off.

None of the relevant JSC options default to ASSERT_ENABLED:

  • validateGraph / validateGraphAtEachPhase: default false
  • validateAbstractInterpreterState: default false (Restricted)

so today's ASAN/assertions lanes do not run them. validateGraphAtEachPhase runs DFGValidate.cpp after every DFG phase and checks Node/Edge/block invariants, so if a prior phase is leaving the graph inconsistent CI will crash with a VALIDATE((node, edge), ...) message naming the broken node instead of segfaulting in CFA.

What about validateAbstractInterpreterState

It is the more targeted check (re-runs CFA during FTL lowering and diffs the state) but it is far too slow for CI: a 5000-iteration sum over a 10k-element array takes 26s on release bun and times out on a debug build. validateGraphAtEachPhase alone adds <1s to the same loop.

Smoke test

BUN_JSC_validateGraphAtEachPhase=1 against inspect.test.js, text-encoder.test.js, bun-serve-static.test.ts on the debug build: 175 pass / 0 fail.


no test proof · iteration 2 · build/CI scripts only; test-proof not applicable

@coderabbitai

coderabbitai Bot commented Jul 19, 2026 •

Copy link
Copy Markdown
Contributor

Warning

Review limit reached

@robobun, you've reached your PR review limit, so we couldn't start this review.

Next review available in: 1 minute

Enable usage-based reviews in Billing to review now. Otherwise, wait until the next included review is available.
You're only billed for reviews past your plan's rate limits ($0.25/file).

How can I continue?

After more reviews become available, a review can be triggered using the @coderabbitai review command as a PR comment. Alternatively, push new commits to this PR.

To avoid repeated limits, reduce automatic review volume by pausing incremental auto-reviews earlier, using label-based review opt-in, excluding WIP or generated PR titles, or requesting reviews manually when the PR is ready. If your team needs uninterrupted high-volume reviews, an organization admin can enable usage-based reviews.

How do review limits work?

CodeRabbit enforces per-developer PR review limits for each organization. Most developers receive the normal plan review availability.

For paid Pro and Pro+ PR reviews, CodeRabbit uses adaptive limits for sustained high-volume activity. When a developer's recent PR review activity reaches the 95th percentile or higher among CodeRabbit users, additional reviews become available more gradually as earlier reviews age out of the rolling window.

Please refer docs for additional details.

Review details
⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: ASSERTIVE

Plan: Pro

Run ID: d68f4099-2593-4046-90a7-59b9c854d72a

📥 Commits

Reviewing files that changed from the base of the PR and between f5ba155 and a05f842.

📒 Files selected for processing (1)
  • scripts/runner.node.mjs

Comment @coderabbitai help to get the list of available commands.

@robobun

robobun commented Jul 19, 2026 •

Copy link
Copy Markdown
Collaborator Author
Updated 10:07 PM PT - Jul 18th, 2026

❌ @robobun, your commit a05f842 has 3 failures in Build #75612 (All Failures):


🧪   To try this PR locally:

bunx bun-pr 34665

That installs a local version of the PR into your bun-34665 executable, so you can run:

bun-34665 --bun

@claude claude Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM — CI-only env var addition scoped to ASAN lanes.

What was reviewed:

  • Both spawn paths (Node-compat runOneTest and spawnBunTest) gate on the same basename(execPath).includes("asan") check as neighboring validateExceptionChecks / LSAN blocks.
  • BUN_JSC_-prefixed option name matches the established JSC-option env convention; no production code touched.
  • Worst-case blast radius is ASAN test lanes getting slower or surfacing new DFG validation failures, which is the stated intent.
Extended reasoning...

Overview

Adds env.BUN_JSC_validateGraphAtEachPhase = "1" in two places in scripts/runner.node.mjs: the Node-compat test spawn path inside runOneTest and the spawnBunTest path. Both are gated on basename(execPath).includes("asan"), identical to the existing gates for BUN_JSC_validateExceptionChecks, BUN_DESTRUCT_VM_ON_EXIT, and BUN_FEATURE_FLAG_NO_ORPHANS in the same functions. No other files changed.

Security risks

None. This sets an internal JSC diagnostics flag in the CI test-runner script; it does not touch runtime code, does not accept external input, and does not affect release builds or user-facing binaries.

Level of scrutiny

Low. This is CI infrastructure only — the runner script that spawns test processes. The change is additive, mechanical, and mirrors the exact idiom of the surrounding lines. If the flag turns out to be too slow or noisy on ASAN lanes it will show up immediately in CI and can be reverted with no user impact.

Other factors

The PR description is thorough (Sentry cluster IDs, disassembly analysis, why validateAbstractInterpreterState was rejected, smoke-test results). The two insertion points are the correct and complete set — they cover every path in the runner that spawns bun with ASAN for tests. The comment on each block is a durable why-comment naming the Sentry issue and the rejected alternative, matching the file's existing comment style. No prior reviewer feedback to address.

@robobun

robobun commented Jul 19, 2026 •

Copy link
Copy Markdown
Collaborator Author

Build #75612 (rebased): all 20 :debian: 13 x64-asan - test-bun shards passed with BUN_JSC_validateGraphAtEachPhase=1 in effect. No DFG VALIDATE() failures, no timeouts on the ASAN lane.

This diff only sets the env var when basename(execPath).includes("asan"), so it cannot affect non-ASAN lanes.

Remaining red (all non-ASAN, unrelated to this diff):

Ready for review.

Validate DFG graph invariants after every compiler phase so a corrupted
Node*/Edge fails with a VALIDATE() crash naming the offending node, instead
of a garbage-pointer segfault deep inside CFA/executeEffects.

Targets the ~5900-event Sentry cluster around BUN-2YP4 / BUN-2VC6 / BUN-2WRJ
(JITWorklistThread -> DFG::CFAPhase::performBlockCFA -> AbstractInterpreter),
where the AbstractInterpreter::m_state reference is getting stomped on the JIT
thread stack with what looks like an AbstractValue {m_arrayModes, m_effectEpoch}
pair.

validateAbstractInterpreterState would be the more targeted FTL-side check but
it is far too slow (26s+ for a trivial hot loop even on release).
@robobun
robobun force-pushed the claude/farm/9a031f57/asan-dfg-validation branch from 8ae4e89 to a05f842 Compare July 19, 2026 03:35

@claude claude Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM — CI-only env var addition, scoped to ASAN lanes.

Checked that the basename(execPath).includes("asan") gate matches the neighboring validateExceptionChecks / BUN_FEATURE_FLAG_NO_ORPHANS blocks and that both env-setup sites (node-compat path in runOneTest and spawnBunTest) are covered. The env object is already spread into bunEnv after defaults in spawnBun, so the new key propagates to the child process. No production/runtime code touched.

Extended reasoning...

Overview

This PR adds BUN_JSC_validateGraphAtEachPhase=1 to the environment of test subprocesses in scripts/runner.node.mjs, gated on the ASAN binary. It is inserted in two places: the node-compat test path inside runOneTest and the spawnBunTest path. Both insertions mirror the existing BUN_JSC_validateExceptionChecks / shouldValidateLeakSan blocks immediately above and below them, using the same basename(execPath).includes("asan") predicate that already gates BUN_FEATURE_FLAG_NO_ORPHANS and the LSAN options.

Security risks

None. This is a CI test-runner script change that sets a diagnostic JSC option in the environment of already-instrumented ASAN test processes. No auth, crypto, network, or user-facing surface is touched.

Level of scrutiny

Low. The change is a mechanical config tweak to an internal CI script following an established pattern in the same file. Worst-case failure mode is that ASAN CI gets slower or surfaces new VALIDATE() crashes — which is the explicit intent (hunting the DFG CFA crash cluster described in the PR body). It cannot affect release builds or non-ASAN lanes.

Other factors

The PR description is thorough (Sentry cluster, disassembly, why validateAbstractInterpreterState was not chosen, smoke test on three test files). The inline comment (3 lines) explains the intent and the alternative that was rejected, matching the repo's comment conventions. The earlier CI failures were traced to an unrelated main-branch regression (#34605, fixed by #34667) and the PR was rebased. No CODEOWNERS cover scripts/runner.node.mjs. No outstanding reviewer comments.

@github-actions

Copy link
Copy Markdown
Contributor

Closing this PR because it has been inactive for more than 90 days.

@github-actions github-actions Bot closed this Oct 18, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant