Skip to content

[JSC] A generator that suspends saves only the locals written since it was entered or resumed - #751

Open
dylan-conway wants to merge 4 commits into
mainfrom
claude/generator-frame-pruning
Open

dylan-conway wants to merge 4 commits into
mainfrom
claude/generator-frame-pruning

Conversation

@dylan-conway

@dylan-conway dylan-conway commented Oct 1, 2026 •

Copy link
Copy Markdown
Member

Problem

When a generator, async function or async generator is turned into a state machine, BytecodeGeneratorification emits, at every suspend point, one put_to_scope per live local before suspending and one get_from_scope per live local after resuming. Bytecode is O(suspend points × live locals). The saves are the expensive half: in Baseline a frame save is 172 bytes of code and a restore is 24, and in the largest function of a real application the saves alone are 52.6% of its Baseline code.

This has to be fixed here: the copies are emitted by JSC's own bytecode pass, and nothing an embedder does can remove them.

What this does

One thing: a suspend point does not save a local that has not been written since the function was entered or last resumed.

Each local already has its own slot in the generator frame. Resuming loads every live local from its slot, exactly as before, so afterwards register and slot agree. If nothing writes the register before the next suspend point, the slot is still right and the save is dead.

saveOnlyWrittenLocals is one forward may-analysis over the existing BytecodeGraph: what may have been written by the time each block is reached. It is emptied after each op_yield, and it flows along ordinary edges and from every block to its exception handler. It runs off a worklist, so the work does not depend on how the code is laid out. The save loop then goes over liveness & written instead of liveness.

Restores are untouched, nothing new is inserted anywhere, BytecodeRewriter is untouched, and no opcode is special. At every suspend point the frame holds exactly what it held before, so nothing changes for the GC, the DFG or the debugger.

There are no heuristics and no thresholds. The one shortcut is a proof, not a tuning knob: a function with a single suspend point is skipped, because that point is reached from the function's entry and everything live has been written on the way from there.

useGeneratorSavePruning (default true) turns it off.

The engine change is 84 added lines in BytecodeGeneratorification.cpp and one in OptionsList.h.

History

This PR used to do much more: it also pruned restores, and chose per local between copying at suspend points and keeping the local in the frame. That got further (−39% Baseline code on the functions below instead of −33%), but it needed a cost model, a rule about loops, new insertion points in BytecodeRewriter, and special cases for several opcodes. Review found a hang in the rewriter change, and my own review found that it could pin dead objects in the frame and that two code shapes compiled slower than before, one of them 3×. All of that is removed here rather than patched. The first two commits are that version; the last one replaces it.

Results

Release build, one binary, option off vs. on. Program output is identical in every row.

Synthetic async generator with S suspend points and L live locals (each activation touches one local):

S / L saves bytecode bytes Baseline code bytes Baseline compile
50 / 10 615 → 109 19,817 → 12,897 142,496 → 58,112 0.37 → 0.18 ms
50 / 40 2,295 → 139 78,345 → 44,577 480,608 → 111,424 1.09 → 0.42 ms
200 / 10 2,473 → 410 91,855 → 58,639 564,064 → 219,520 1.27 → 0.69 ms
200 / 40 9,223 → 440 322,813 → 182,285 1,902,752 → 398,752 3.96 → 1.23 ms
400 / 40 18,448 → 840 650,701 → 368,973 3,796,448 → 781,248 8.97 → 2.73 ms
400 / 80 36,448 → 880 1,265,221 → 696,133 7,350,816 → 1,229,216 20.14 → 4.56 ms

(The save counts include a few put_to_scope that are not frame saves.)

A real 16 MB application bundle (4,291 generator and async functions, every function compiled with generateBytecodeCacheFile): 41,232 → 19,388 saves (−53%).

suspend points functions saves
1 2,236 3,086 → 3,086
2–3 1,331 7,493 → 5,409 (−28%)
4–10 634 12,913 → 6,571 (−49%)
11–30 77 6,664 → 2,302 (−65%)
> 30 13 11,076 → 2,020 (−82%)

Across that bundle and a second 9 MB one, 1,930 of 6,013 functions have fewer saves, 4,083 are unchanged, and none has more.

The three largest functions, each extracted and called once with --jitPolicyScale=0 to get its real Baseline compile:

suspends / locals / handler ranges bytecode Baseline code
152 / 116 / 338 173,131 → 121,307 1,216,704 → 660,320 (−46%)
90 / 200 / 48 142,524 → 115,900 898,592 → 611,904 (−32%)
69 / 114 / 105 94,131 → 68,307 693,952 → 416,288 (−40%)

The 11 largest together: bytecode 657,079 → 516,671 (−21%), Baseline code 4,825,536 → 3,251,776 (−33%).

That application's real agent loop in Bun, driven offline by a local mock of its API (streamed text and tool calls), medians of 15 runs. The option-off case was measured twice, interleaved, to show the noise:

session JIT memory peak RSS user CPU
20 turns 8,108 / 8,108 → 7,136 KB (−12%) 248 / 247 → 245 MB 796 / 769 → 791 ms
50 turns 10,608 / 10,608 → 9,520 KB (−10%) 372 / 371 → 352 MB (−5%) 1,129 / 1,135 → 1,095 ms (−3%)
100 turns 11,188 / 11,188 → 10,108 KB (−10%) 370 / 365 → 349 MB (−5%) 1,617 / 1,617 → 1,592 ms (−2%)
300 turns 12,440 / 12,508 → 11,344 KB (−9%) 413 / 408 → 409 MB 3,504 / 3,510 → 3,449 ms (−2%)

JIT memory is the solid result: its runs differ by about 1%. Peak RSS varies by up to 10% from run to run at 300 turns, so only the 50 and 100 turn rows say anything.

This is a memory and compile-time win for programs with large async functions, and ordinary programs will not notice it. hono and fastify under load, rollup bundling svelte's compiler, web streams and a CLI's --help, medians of 9 runs: JIT memory, peak RSS and CPU are all within the difference between two runs with the option off. (Two readings were not at first, rollup's peak RSS and the streams CPU time; repeated 15 and 25 times in four interleaved groups, the medians are 500 / 509 MB off against 495 / 500 on, and 710 / 708 ms off against 708 / 699 on.) bun build --bytecode output: hono unchanged, fastify 1,580,696 → 1,580,392 bytes.

Cost of the analysis: about 1% of bytecode generation. User-mode instructions for the whole parse + generate + encode pipeline (perf stat), in millions, medians of 7 interleaved runs (the runs of one configuration differ by 0.15% at most):

input off on
20,000 small async functions and generators 4,704 4,782 +1.7%
16 MB bundle 11,726 11,856 +1.1%
9 MB bundle 6,079 6,114 +0.6%

That is one more decode of each instruction in the functions that have two or more suspend points. It is paid once, when a function is first compiled to bytecode. (An earlier version of this description said +0.1% for the 16 MB bundle. That came from a single run with a bad reading and was wrong.)

Shapes chosen to be hard on it, 100 locals unless noted, medians of 3:

shape off on
1,000 awaits, no try 513 316 0.62×
1,000 / 4,000 awaits in one try/catch/finally 792 / 11,792 590 / 8,246 0.74× / 0.70×
4,000 awaits in one try, 600 locals 50,355 29,233 0.58×
1,000 / 4,000 loops that each await 773 / 11,559 778 / 11,581 1.01× / 1.00×
1,000 / 4,000 consecutive try/catch/finally that each await 2,822 / 42,309 2,837 / 42,369 1.01× / 1.00×
40 / 160 loops nested in each other, each awaiting 19.1 / 181.6 17.4 / 126.3 0.91× / 0.70×

None is slower than before by more than 1%, and no ratio grows with size. The worklist visits each block 1.00 to 1.55 times in these, and 1.26 times on average over the 16 MB bundle (2.63 in its worst function). (The pass as a whole is quadratic in several of these, before and after: the liveness analysis calls handlerForBytecodeIndex, a linear scan, for every instruction on every iteration, and BytecodeBasicBlock::computeImpl links blocks by scanning all of them.)

Speed. Nine microbenchmarks, medians of 9 processes, best of 40 rounds in each, option off measured twice:

benchmark off off again on
one await in a loop 5.46 ms 5.46 5.46
four awaits in a row 6.41 6.41 6.27
awaits in try/catch/finally 5.50 5.55 5.43
generator, one yield in a loop 6.27 6.31 6.07
generator, 4 yields, 8 locals 5.39 5.26 4.89
hot loop that never suspends, in a try, among awaits 12.47 12.51 12.49
loop that suspends rarely 17.07 15.49 17.83
for-await over an async generator 6.17 6.17 6.21
for-of with awaits in the body 3.83 3.84 3.75

Nothing is slower. A few are a little faster, most of all the generator with many locals; the loop that suspends rarely is too noisy to read.

Not done here

Restores are still O(suspend points × live locals), so bytecode only roughly halves in the worst cases. They are cheap in Baseline, and a follow-up that gives frame copies their own compact opcodes makes them cheaper still, so I would rather see whether pruning them is still worth its complexity after that.

The two largest functions above stay over maximumOptimizationCandidateBytecodeCost (100,000), so this does not change which functions the DFG will compile.

Testing

  • New JSTests/stress/generator-saves-written-locals.js: 12 small functions, each one way for a write to reach a suspend point with no other way for it to get there: in the suspend point's own block, in an earlier block, on one of two paths, in an earlier loop iteration, in a try block that throws (early and late in the block, in the inner of two, and where the handler throws too), by op_enter, part way through an instruction that can still throw, in an async function, and not at all. It runs in the default modes and once with the option off. Slowest of the 18: 115 ms. V8 agrees with the expectations of the generator cases.

  • Negative testing, case by case. I broke the change in 11 ways, one per thing it relies on, and ran every check in the test against every breakage in its own process:

    breakage caught by
    writes do not flow to handlers the four try-block cases, the async case
    a block is not revisited when more reaches it earlier iteration, the try-block cases, async
    op_enter writes nothing on entry, and five others
    only an instruction's last checkpoint counts part way through an instruction
    nothing flows along back edges earlier iteration, the try-block cases, async
    a block's own writes do not flow out earlier block, and ten others
    a block keeps only the last thing that flowed to it one of two paths, and six others
    the outermost handler wins instead of the innermost the inner of two try blocks
    only the first block of a handler's range is claimed late in a try block, handler throws too, async
    writes in the suspend point's own block do not count in the same block, async
    resuming loads only what was saved never written again, part way through an instruction

    Every breakage is caught by the case written for it, and every case catches at least one.

  • Object lifetime. A WeakRef test of what a suspended generator keeps alive gives the same answer with the option off and on in all five of its cases.

  • Differential fuzzing with a generator of random generator / async / async generator programs (nested try/catch/finally, labeled break and continue, loops that suspend always, sometimes and never, switches with gaps, for-of and for-await with closable iterators, yield*, TDZ, closures, suspends inside expressions, driven by random next / throw / return). Output with the option off in the LLInt is compared with the option on in 9 configurations (LLInt, Baseline, DFG, FTL with eager thresholds, debugger bytecode, type and control flow profilers, the bytecode optimizer, collectContinuously). 640 seeds on the final code, all identical; it finds each of the 11 breakages above in 4 to 102 of 120 seeds.

@github-actions

github-actions Bot commented Oct 1, 2026 •

Copy link
Copy Markdown

Preview build of e88805c: autobuild-preview-pr-751-e88805c5

@coderabbitai

coderabbitai Bot commented Oct 1, 2026 •

Copy link
Copy Markdown

Review in Change Stack →

Navigate logical layers of code changes, visualize relationships, and explore their blast radius.

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info
⚙️ Run configuration

Configuration used: Organization UI

Review profile: ASSERTIVE

Plan: Essentials

Run ID: f2d28466-da8d-45b4-af98-b6a488299e76

📥 Commits

Reviewing files that changed from the base of the PR and between b0bbc2e and e88805c.

📒 Files selected for processing (3)
  • JSTests/stress/generator-saves-written-locals.js
  • Source/JavaScriptCore/bytecode/BytecodeGeneratorification.cpp
  • Source/JavaScriptCore/runtime/OptionsList.h

Included review availability: This review used your included allowance. 1 included review remains after this review. Your included PR review attempts over the past 7 days set your current allowance at 2 reviews per hour.


Walkthrough

The change adds written-local tracking to generator liveness analysis. Generatorification saves only locals that are live and may have been written since entry or the previous suspension. A new stress test covers generator and async-function suspension paths.

Changes

Generator save pruning

Layer / File(s) Summary
Track and prune saved locals
Source/JavaScriptCore/runtime/OptionsList.h, Source/JavaScriptCore/bytecode/BytecodeGeneratorification.cpp
The enabled-by-default useGeneratorSavePruning option controls analysis that tracks possibly written locals. Generatorification saves locals that are both live and possibly written since entry or the previous suspension. Resume still loads every live local.
Exercise suspension paths
JSTests/stress/generator-saves-written-locals.js
The stress test checks unchanged and rewritten locals, conditional writes, loops, exception handlers, captured variables, iterator progress, and async-function suspension.

Priority: ➖ Normal

Merge Risk: ⚪ Minimal · up to e8880

The change prunes generator frame saves behind an option that defaults to on, and resume still loads all live locals. No concrete merge-blocking risk was found in review.

🚥 Pre-merge checks | ✅ 3 | ❌ 1

❌ Failed checks (1 warning)

Check name Status Explanation Resolution
Description check ⚠️ Warning The description gives a detailed explanation, implementation summary, results, and testing evidence. However, it omits required template information, including a bug title, Bugzilla URL, reviewer line… Add the associated Bugzilla bug title and URL, include the required reviewer line, and provide the changed-file and function list in the repository template format.
✅ Passed checks (3 passed)
Check name Status Explanation
Linked Issues check ✅ Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check ✅ Passed Check skipped because no linked issues were found for this pull request.
Title check ✅ Passed The title clearly and specifically describes the main change: generators save only locals written since entry or resume.
Full details: Description check

Explanation

The description gives a detailed explanation, implementation summary, results, and testing evidence. However, it omits required template information, including a bug title, Bugzilla URL, reviewer line, and the template-style changed-file/function list.

  • Fix all pre-merge checks with AI
  • Autopilot · Keep fixing CodeRabbit findings and required CI, and resolving merge conflicts

Autopilot is currently an internal CodeRabbit preview.

Warning

Git: CodeRabbit could not clone the repository, so clone-backed analysis was skipped and this review may be incomplete. Verify repository clone access, such as SSH credentials, before requesting another full review. If clone access is intentionally unavailable, use path_filters to narrow the review scope.


Comment @coderabbitai help to get the list of available commands.

@claude claude Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Beyond the inline finding on the range jump-table entries, I also checked two things that turned out fine: write-through stores are keyed to the next instruction's offset and inserted at Position::Before, which sorts ahead of LabelPoint, so a jump to that offset skips the store and only the fallthrough path runs it, as intended; and the one-handler-per-block lookup in buildSegments is sound because computePreciseJumpTargets adds every handler's start, end and target as block leaders, so no block straddles a handler range boundary.

Extended reasoning...

The change replaces the per-yield liveness copy in BytecodeGeneratorification.cpp with a ~650-line segment dataflow analysis (GeneratorFrameAnalysis) that prunes saves/restores and adds a write-through mode, changes BytecodeRewriter offset-adjustment semantics (jump sources now use OriginalBytecodePoint, new adjustInstructionOffset for profiler tables), adds four JSC options, and adds stress tests. It touches no security-sensitive surface such as auth or injection, but it is core bytecode-rewriting logic in the JIT pipeline where an off-by-one produces hangs or wrong values. One confirmed finding (0-valued range jump-table entries now get a nonzero adjustment when a load is inserted before the switch) plus the size and complexity of the dataflow rule out approval; the ruled-out items above are the concrete extra checks worth recording.

@claude claude Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The follow-up commit fixes the switch jump-table hole I flagged on the previous push, and this re-review found no new bugs; because the core of the change is a new ~650-line fixpoint dataflow analysis in a bytecode pass that every generator and async function goes through, a human look would still be worthwhile.

What was reviewed:

  • BytecodeRewriter.h adjustJumpTarget(X, X) now returns 0, so 0 entries in op_switch_imm / op_switch_char range tables stay holes when a load is inserted after the switch's label; checked that no other caller of that overload relies on a self-jump (the bytecode generator never emits one, loops always target an op_loop_hint).
  • The offset += before + after reorder in adjustJumpTargets() so out-of-line jump targets are keyed by the instruction's final offset, consistent with the new adjustInstructionOffset() used for the type and control-flow profiler maps.
  • The new "a switch with gaps between its cases" test exercises integer, single-char, and in-loop switches with holes under all four option files, each driven by testLoopCount and //@ requireOptions.
Extended reasoning...

The diff adds a GeneratorFrameAnalysis class to Source/JavaScriptCore/bytecode/BytecodeGeneratorification.cpp that replaces the per-yield save/restore of all live locals with per-local pruning and an optional write-through policy, fixes jump-offset arithmetic in BytecodeRewriter for fragments inserted after an instruction's label, adds four options, and adds a shared stress suite run under four option combinations. It touches no security-sensitive surface (no auth, injection, or data exposure), but it is correctness-critical: wrong analysis silently corrupts generator locals across suspends, and the ASSERT_ENABLED-only validate() does not run in the release-only CI lanes. The second commit addresses the one finding from the prior run with a targeted fix and regression test, and the hunt ran dry this time, which is why the body acknowledges progress; the size and complexity of the dataflow code and its effect on all generator and async bytecode in every tier are what rule out approving without a human reviewer.

…t was entered or resumed, and does nothing else new
@dylan-conway dylan-conway changed the title [JSC] Generators copy a local to or from the generator frame only where an activation needs it [JSC] A generator that suspends saves only the locals written since it was entered or resumed Oct 1, 2026

@claude claude Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I re-reviewed the rewritten version (b0bbc2e) and found no bugs; the BytecodeRewriter change my earlier comment was about is gone entirely, and the engine change is now a single forward may-written pass plus a one-line save filter. Since this is a new dataflow analysis in a pass every generator and async function goes through, under /Source/JavaScriptCore ownership, a human look is still worthwhile.

What was reviewed:

  • Soundness of liveness & written: a local live at a yield but not in written must have been live at the previous resume (no def between), so it was reloaded there and its slot still matches; the entry path is covered because op_enter defs all numVars and the state switch is inserted after the analysis, so no switch edges leak entry writes into resume blocks.
  • Per-block handler lookup on the leader offset: computePreciseJumpTargets adds every handler start/end/target as a jump target, so blocks never straddle a handler range; flowing the pre-reset set at each yield and at block end covers every intermediate throw point.
  • Worklist/bitvector reuse: in() is only reused after liveness finished and data.liveness was copied out; flowTo is monotone with the in-worklist guard; checkpointed defs are iterated the same way stepOverInstruction does.
  • Test follows JSTests/README conventions (testLoopCount, throws on failure, noInline/drainMicrotasks are jsc-shell APIs); I could not run it here, so the 152 ms vs. the 200 ms budget claim is unverified.
Extended reasoning...

The diff touches Source/JavaScriptCore/bytecode/BytecodeGeneratorification.cpp (new GeneratorLivenessAnalysis::computeWrittenLocals, 71 lines, and the save loop now iterating liveness & written), one new Bool option useGeneratorFramePruning in OptionsList.h, and a new 539-line stress test; it touches no auth, injection or data-exposure surface. The prior inline finding (switch range-table gap hang in BytecodeRewriter.h) is moot because that code is no longer part of the PR. Approval is withheld because the changed paths are covered by .github/CODEOWNERS (/Source/JavaScriptCore and /JSTests to the jsc-reviewers team) and because a new dataflow analysis whose correctness rests on a liveness/resume invariant is not a mechanical change, even though my reading found the argument sound.

…its own table, and has one test case for each thing it relies on

@claude claude Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I reviewed the pared-down version of this PR (the last push, which replaced the per-activation design) and did not find any bugs. Because it changes what every generator and async function saves at every suspend point, and the correctness rests on a dataflow argument rather than a local code change, a human look is still worthwhile.

What was reviewed:

  • The per-block handler table: handler start, end and target are all fed to computePreciseJumpTargets, so they are block leaders and no block straddles a try range; the exit block's UINT_MAX leader terminates the range walk, and empty/inverted ranges are already dropped in BytecodeGenerator::generate before this pass runs.
  • The "clear written at op_yield" step: resume reloads every live local (liveness is unchanged), op_yield itself has no defs, and op_enter defs every var, so a local that is live at a yield and unwritten on every path must have been loaded at the previous resume and still equals its slot.
  • Handler flow at a yield happens before the clear and again at block end with all checkpoint defs, so a write before a throw in the same block (including a mid-instruction throw after a checkpoint def) reaches the handler.
  • The test directive uses the run(kind, *options) form the runner provides, and a brand-new stress test does not fall under the BUN-TEST-DIFFERENCES.md rule for changed upstream tests.
Extended reasoning...

The net diff adds an 84-line forward may-written dataflow in BytecodeGeneratorification.cpp that narrows the save set at each op_yield to liveness & written, a default-on option in OptionsList.h, and a stress test that runs with the option on and off. It touches no security-sensitive surface (no auth, injection or data exposure), but it alters bytecode emitted for every generator, async function and async generator in the fork. I traced the invariants the pruning relies on (handler boundaries as block leaders, exit-block sentinel, op_enter defs, resume loading all live locals) and found them to hold, and the bug hunt ran dry. Deferring rather than approving because the change is in core engine codegen with a non-local correctness argument and runs in CI only on no-assertion release lanes, so the ASSERTs in the new code never fire there.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant