Skip to content

test: keep JSC's MarkedBlock warm-up out of the structuredClone RSS test - #41280

Open
robobun wants to merge 2 commits into
mainfrom
robobun/7b9ac857/structured-clone-rss-warmup-pool
Open

robobun wants to merge 2 commits into
mainfrom
robobun/7b9ac857/structured-clone-rss-warmup-pool

Conversation

@robobun

@robobun robobun commented Sep 3, 2026 •

Copy link
Copy Markdown
Collaborator

Problem

  • test/js/web/structured-clone-fastpath.test.ts fails on Linux aarch64 in the parallel batch: "structuredClone should use a constant amount of memory for simple object inputs", Expected: < 1048576, Received: 1572864 (build 109723).
  • The WebKit upgrade in Two full collections + payload page-out once the heap goes quiet #41083 added a warm-up supply of MarkedBlocks. A helper thread keeps up to 32 blocks resident and refills it when fewer than 8 are left. A MarkedBlock is 64 KiB on Linux arm64 (max(16 KB, CeilingOnPageSize)), so one refill makes up to 2 MiB resident. A refill in the measured loop adds 1 to 1.5 MiB.

Fix

  • The test keeps one clone in 101 of a first batch alive through the GC. The 10000 measured clones fit in the cells of the 10000 that died. So the loop takes no new blocks and cannot start a refill.
  • The loop runs in a helper that JSC compiles during the first batch. The clone count and the bound do not change.
  • Verified on x64 with a 2 MiB supply (BUN_JSC_warmUpMarkedBlockCount=128). Run alone, the old test fails 8 of 8 (+1.7 to 2.0 MiB). The new test passes 10 of 10 (+0 to 100 KiB). A retained 100-byte array per clone still fails it. bun bd test passes.

Background

  • A MarkedBlock is a fixed-size chunk of the JSC heap that holds cells of one size.
  • Bun.gc(true) frees only blocks with no live cells. New cells go into the free cells of the other blocks first.
  • The supply (useWarmUpMarkedBlocks, from upstream WebKit) moves the page faults of new blocks to a helper thread.
Notes

Sightings. I scanned the annotations of builds 108400 to 109731. The test failed in the batch 46 times in 44 builds, from build 109059 (2026-09-01 14:48 UTC) to 109723. All runs were on Linux aarch64: 27 on debian 13, 18 on ubuntu 25.04, 1 on alpine 3.23. The received values are 1048576 to 1572864. The 659 builds before 109059 have no failure. #41083 merged at 10:01 UTC on 2026-09-01, and all 44 builds have it in their base. Until #41204 the runner ran the file again alone, where it passed, so the earlier runs show as flaky.

Why only in the batch. The supply level at the start of the test depends on what ran before it in the same worker. On x64 with a 2 MiB supply, a run of the whole file puts the refill in the string test, which has an 8 MiB bound, and the simple-object test passes. Run alone, the simple-object test fails.

Why only arm64. On x64 and darwin a MarkedBlock is 16 KiB, so a refill is at most 512 KiB. The measured loop takes about 30 blocks there, about as many as a refill adds, so the supply changes the old delta very little.

Not the cause. e7e8fe0 (#36545) changes only objects with no properties. This test clones an object with one property.

Precise RSS delta of the measured loop, x64 release (KiB, 4 to 10 runs each). From /proc/<pid>/smaps. /proc/self/stat (what process.memoryUsage.rss() reads) moves in steps of max(32, 2 * CPUs) pages: 1 MiB on the 128-CPU machine I used, 128 KiB on the 4-vCPU CI hosts.

test supply whole file this test alone
old 32 x 16 KiB (x64 default) 250 to 530 400 to 840
old 128 x 16 KiB (2 MiB) 4 to 52 1690 to 1990
new 32 x 16 KiB 0 to 44 28 to 88
new 128 x 16 KiB 0 to 36 36 to 96

With BUN_JSC_useWarmUpMarkedBlocks=0 the old test alone is 610 to 670 KiB: most of the old delta on x64 is new blocks, which the new test does not take.

Other checks.

  • In a serial run with --isolate after 10 or 57 files from the same batch, the new delta is -12 to 36 KiB.
  • The new test with the aarch64 release binary under qemu-user: -32 to 40 KiB.
  • Cost: 10000 more clones, about 3 ms in a release build and 0.7 s in a debug ASAN build.
  • The string-input test is unchanged. A refill fits in its 8 MiB bound.
  • The patch applies on top of test: call matchers once per loop in structured-clone-fastpath.test.ts #41276, which edits other tests in this file.

For the WebKit fork. warmUpMarkedBlockCount counts blocks. oven-sh/WebKit#553 describes the supply as 512 KiB, which holds where a MarkedBlock is 16 KiB. On Linux arm64 it is 2 MiB.


[auto-merge] gate passed · iteration 1 · 1 files touched

passes on PR (with fix)
Test-only change.

Debug/ASAN (expected pass):
$ bun bd test 'test/js/web/structured-clone-fastpath.test.ts'
$ BUN_DEBUG_QUIET_LOGS=1 bun scripts/build.ts --profile=debug --quiet test test/js/web/structured-clone-fastpath.test.ts
bun test v1.4.1 (a6c4cc276)

test/js/web/structured-clone-fastpath.test.ts:
(pass) Structured Clone Fast Path > structuredClone should work with empty object [15.70ms]
(pass) Structured Clone Fast Path > structuredClone({}) result is spreadable after adding a property [4.83ms]
(pass) Structured Clone Fast Path > structuredClone should work with empty string [1.23ms]
(pass) Structured Clone Fast Path > structuredCloneDeOptimization [1.91ms]
(pass) Structured Clone Fast Path > structuredCloneDeOptimization [0.22ms]
(pass) Structured Clone Fast Path > structuredCloneDeOptimization [0.56ms]
(pass) Structured Clone Fast Path > structuredCloneDeOptimization [0.20ms]
(pass) Structured Clone Fast Path > structuredCloneDeOptimization [0.34ms]
(pass) Structured Clone Fast Path > structuredClone should use a constant amount of memory for string inputs [267.13ms]
(pass) Structured Clone Fast Path > structuredClone should use a constant amount of memory for simple object inputs [1391.90ms]
(pass) Structured Clone Fast Path > structuredClone should work with empty array [2.42ms]
(pass) Structured Clone Fast Path > structuredClone should work with array of numbers [1.74ms]
(pass) Structured Clone Fast Path > structuredClone should work with array of strings [1.59ms]
(pass) Structured Clone Fast Path > structuredClone should work with array of mixed primitives [1.84ms]
(pass) Structured Clone Fast Path > structuredClone should work with array of special numbers [2.85ms]
(pass) Structured Clone Fast Path > structuredClone should work with large array of numbers [27.14ms]
(pass) Structured Clone Fast Path > structuredClone should work with array of simple objects [2.14ms]
(pass) Structured Clone Fast Path > structuredClone should work with large array of same-shape objects [18.46ms]
(pass) Structured Clone Fast Path > structuredClone should work wit
... (truncated)
Exit: 0
diff hotspot
test/js/web/structured-clone-fastpath.test.ts | 18 ++++++++++++++----
 1 file changed, 14 insertions(+), 4 deletions(-)

gate history · 2 passed · 0 rejected · iteration 1

evidence per changed file
file                                           reads  edits  tests
test/js/web/structured-clone-fastpath.test.ts      5      7     19

"structuredClone should use a constant amount of memory for simple object
inputs" measured RSS over 10000 clones with a 1 MiB bound. Since the
WebKit upgrade in #41083, JSC hands out new MarkedBlocks from a supply
that a helper thread keeps resident. A refill of that supply makes up to
32 blocks resident. A MarkedBlock is 64 KiB on Linux arm64, so a refill
in the measured loop added 1 to 1.5 MiB there, and the test failed in
the parallel batch.

The test now keeps one clone in 101 of a first batch alive through the
GC. The measured clones fit in the cells of the clones that died, so the
heap takes no new blocks in the measured loop. The loop runs in a helper
that the first batch gets compiled. The clone count and the bound do not
change.
@robobun

robobun commented Sep 3, 2026 •

Copy link
Copy Markdown
Collaborator Author

Status: the diff is ready for a maintainer to merge. It changes one test and no source. The red CI lanes come from tests that this PR does not touch.

How I reproduced it. The failure needs the warm-up supply at its Linux arm64 size, 2 MiB. On x64 a MarkedBlock is 16 KiB, so 128 blocks give the same size:

# old test: fails every run (RSS +1.7 to 2.0 MiB)
BUN_JSC_warmUpMarkedBlockCount=128 bun test test/js/web/structured-clone-fastpath.test.ts -t "simple object inputs"

With this branch the same command passes (RSS +0 to 100 KiB). bun bd test test/js/web/structured-clone-fastpath.test.ts passes: 93 pass, 0 fail.

CI. The changed test passes (93 pass, 0 fail) in builds 109735 and 109738 on these lanes:

  • Linux aarch64: debian 13, ubuntu 25.04, alpine 3.23
  • Linux x64: debian 13, ubuntu 25.04, and the ASAN lane
  • Windows x64 and aarch64, macOS aarch64 and x64

These red failures are not related to this diff. I reported each one as a main break:

  • test/js/bun/spawn/spawn-stdin-destroy.test.ts on alpine 3.23 aarch64: EPIPE: broken pipe, write. It also fails on main in build 109667.
  • test/js/sql/postgres-failed-connection-resurrection.test.ts on the x64 ASAN lane: the worker crashed with SIGABRT.
  • test/cli/install/bun-patch.test.ts on windows 2019 x64. It also fails on main in builds 109640, 109667 and 109692.

What CI on this PR cannot show. The original failure happens only in the parallel batch on Linux aarch64. CI runs the test files that a PR changes alone, so the lanes above show only that the new test passes. On main, the old test failed 46 times in the batch between builds 109059 and 109723. The batch runs on main after the merge are the check on arm64.

@coderabbitai

coderabbitai Bot commented Sep 3, 2026 •

Copy link
Copy Markdown
Contributor

Review Change Stack

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info
⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: ASSERTIVE

Plan: Essentials

Run ID: 34bb274e-bcbc-45a4-95bb-ac7f62dbde42

📥 Commits

Reviewing files that changed from the base of the PR and between e0a2b82 and cf2731c.

📒 Files selected for processing (1)
  • test/js/web/structured-clone-fastpath.test.ts

Included review availability: Your plan provides up to 10 included reviews per hour; 6 remain after this review.


Walkthrough

The structured-clone memory test now uses a reusable cloneMany helper. It performs warm-up cloning before RSS sampling, retains 100 clones during measurement, measures 10,000 subsequent clones, and verifies the retained clone count. The ASAN-dependent allocation threshold remains unchanged.

Changes

Structured clone memory validation

Layer / File(s) Summary
Clone allocation and retention measurement
test/js/web/structured-clone-fastpath.test.ts
The test adds cloneMany, performs warm-up cloning, measures 10,000 clones, retains 100 clones, and asserts the retained count.

Suggested reviewers: jarred-sumner

Merge Risk: ⚪ Minimal · up to d79e8

The structured-clone RSS test now warms allocations before measurement while preserving the measured clone count and memory bound. No current merge-blocking risk remains.

🚥 Pre-merge checks | ✅ 4
✅ Passed checks (4 passed)
Check name Status Explanation
Linked Issues check ✅ Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check ✅ Passed Check skipped because no linked issues were found for this pull request.
Description check ✅ Passed The description clearly explains the problem, fix, background, and verification results. It does not use the template headings exactly, but it provides the required information and is complete.
Title check ✅ Passed The title is concise, specific, and accurately describes the main change: preventing JSC MarkedBlock warm-up from affecting the structuredClone RSS test.

Comment @coderabbitai help to get the list of available commands.

@robobun

robobun commented Sep 3, 2026

Copy link
Copy Markdown
Collaborator Author

The automated review has no findings, so the diff stays as it is. Buildkite build 109735 is running. Its Linux aarch64 lanes are the real check for this change, because the failure occurs only there.

@robobun

robobun commented Sep 3, 2026

Copy link
Copy Markdown
Collaborator Author

A correction to my earlier comment. CI runs the test files that a PR changes alone, before the parallel batch (scripts/runner.node.mjs:1278). So on this PR the test runs alone on Linux aarch64, where the old test also passed. The aarch64 lanes here show only that the new test passes on arm64. The batch runs on main after the merge are the check for the fix.

@claude claude Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM — test-only de-flake with the leak threshold unchanged.

What was reviewed:

  • Checked against CLAUDE.md rule #15: this is not a stack-scanner workaround — the identified cause is useWarmUpMarkedBlocks refilling 32 × 64 KiB blocks on arm64, and the fix arranges for measured clones to reuse dead cells rather than moving where GC runs.
  • Verified the 1 MB non-ASAN bound is unchanged, so leak-detection sensitivity is preserved; expect(kept).toHaveLength(100) keeps the interspersed live cells rooted past both Bun.gc(true) calls.
  • Confirmed the sibling string-input test was considered and intentionally left alone (its 8 MiB bound absorbs a refill), and no CODEOWNERS entry covers this file.
Extended reasoning...

Overview

This PR modifies a single memory-leak test in test/js/web/structured-clone-fastpath.test.ts — the "structuredClone should use a constant amount of memory for simple object inputs" case. The warm-up phase now creates 10,100 clones (100 kept in an array, 10,000 discarded via a cloneMany helper) instead of 100, so that after Bun.gc(true) the 10,000 measured clones reuse free cells in blocks pinned by the 100 kept clones rather than allocating fresh MarkedBlocks. An expect(kept).toHaveLength(100) assertion after the second GC keeps the array live through measurement. The RSS-delta thresholds (400 MB ASAN / 1 MB otherwise) are untouched. No production code changes.

Security risks

None. This is a test-only change with no effect on runtime behavior, no external I/O, and no user-controlled input.

Level of scrutiny

Low-to-moderate. The diff is small (~15 net lines) and confined to one test's warm-up structure. The primary review question was CLAUDE.md rule #15 — whether this is a disguised conservative-stack-scanner workaround. It is not: the change does not alter where or how many times GC runs, nor add sleeps; it changes the allocation pattern so the measured window doesn't request new MarkedBlocks and therefore can't trip JSC's useWarmUpMarkedBlocks refill (which on Linux arm64 makes up to 2 MiB resident because MarkedBlock is 64 KiB there vs 16 KiB on x64). The PR description ties the flake's onset to the WebKit upgrade that introduced the warm-up supply, reproduces it on x64 by raising BUN_JSC_warmUpMarkedBlockCount, and provides /proc/<pid>/smaps deltas — this is a mechanism-level explanation, not a "GC ran at a different depth" hand-wave.

Other factors

REVIEW.md's de-flake rule ("keep asserting the property the original assertion protected") is satisfied: the 1 MB bound is unchanged and the PR notes a real per-clone retention still fails it. The sibling string-input leak test was considered and intentionally excluded because its 8 MiB bound already absorbs a refill — matching REVIEW.md's "if a site is intentionally excluded, say so." The added comment is dense but earns its place per rule #14: the next reader would otherwise have to dig through WebKit's MarkedBlock sizing and the warm-up-supply option to understand why the warm-up is shaped this way. Added cost is ~0.7 s in debug+ASAN, within the file budget. No CODEOWNERS entry covers this path, the timeline shows no outstanding reviewer objections, and the bug hunt exited on dry_streak with zero findings.

@robobun

robobun commented Sep 3, 2026 •

Copy link
Copy Markdown
Collaborator Author
Updated 4:36 AM PT - Sep 3rd, 2026

❌ @robobun, your commit d79e8c4 has 2 failures in Build #109738 (All Failures):


🧪   To try this PR locally:

bunx bun-pr 41280

That installs a local version of the PR into your bun-41280 executable, so you can run:

bun-41280 --bun

@robobun

robobun commented Sep 3, 2026

Copy link
Copy Markdown
Collaborator Author

Both automated reviews have no findings, so the diff stays as it is. In build 109735 the changed test passed on Linux aarch64 (debian 13, ubuntu 25.04, alpine 3.23) and on the x64 ASAN lane. It ran alone there, so this shows that the new test works on arm64. It does not show the batch case.

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant