Repository navigation
[JSC] A generator that suspends saves only the locals written since it was entered or resumed - #751
dylan-conway wants to merge 4 commits into
Conversation
…re an activation needs it
|
Preview build of e88805c: |
|
Navigate logical layers of code changes, visualize relationships, and explore their blast radius. No actionable comments were generated in the recent review. 🎉 ℹ️ Recent review info⚙️ Run configurationConfiguration used: Organization UI Review profile: ASSERTIVE Plan: Essentials Run ID: 📒 Files selected for processing (3)
Included review availability: This review used your included allowance. 1 included review remains after this review. Your included PR review attempts over the past 7 days set your current allowance at 2 reviews per hour. WalkthroughThe change adds written-local tracking to generator liveness analysis. Generatorification saves only locals that are live and may have been written since entry or the previous suspension. A new stress test covers generator and async-function suspension paths. ChangesGenerator save pruning
Priority: ➖ Normal Merge Risk: ⚪ Minimal · up to The change prunes generator frame saves behind an option that defaults to on, and resume still loads all live locals. No concrete merge-blocking risk was found in review. 🚥 Pre-merge checks | ✅ 3 | ❌ 1❌ Failed checks (1 warning)
✅ Passed checks (3 passed)
Full details: Description checkExplanation The description gives a detailed explanation, implementation summary, results, and testing evidence. However, it omits required template information, including a bug title, Bugzilla URL, reviewer line, and the template-style changed-file/function list.
Warning Git: CodeRabbit could not clone the repository, so clone-backed analysis was skipped and this review may be incomplete. Verify repository clone access, such as SSH credentials, before requesting another full review. If clone access is intentionally unavailable, use Comment |
There was a problem hiding this comment.
Beyond the inline finding on the range jump-table entries, I also checked two things that turned out fine: write-through stores are keyed to the next instruction's offset and inserted at Position::Before, which sorts ahead of LabelPoint, so a jump to that offset skips the store and only the fallthrough path runs it, as intended; and the one-handler-per-block lookup in buildSegments is sound because computePreciseJumpTargets adds every handler's start, end and target as block leaders, so no block straddles a handler range boundary.
Extended reasoning...
The change replaces the per-yield liveness copy in BytecodeGeneratorification.cpp with a ~650-line segment dataflow analysis (GeneratorFrameAnalysis) that prunes saves/restores and adds a write-through mode, changes BytecodeRewriter offset-adjustment semantics (jump sources now use OriginalBytecodePoint, new adjustInstructionOffset for profiler tables), adds four JSC options, and adds stress tests. It touches no security-sensitive surface such as auth or injection, but it is core bytecode-rewriting logic in the JIT pipeline where an off-by-one produces hangs or wrong values. One confirmed finding (0-valued range jump-table entries now get a nonzero adjustment when a load is inserted before the switch) plus the size and complexity of the dataflow rule out approval; the ruled-out items above are the concrete extra checks worth recording.
…nserted in front of the switch
There was a problem hiding this comment.
The follow-up commit fixes the switch jump-table hole I flagged on the previous push, and this re-review found no new bugs; because the core of the change is a new ~650-line fixpoint dataflow analysis in a bytecode pass that every generator and async function goes through, a human look would still be worthwhile.
What was reviewed:
BytecodeRewriter.hadjustJumpTarget(X, X)now returns 0, so 0 entries in op_switch_imm / op_switch_char range tables stay holes when a load is inserted after the switch's label; checked that no other caller of that overload relies on a self-jump (the bytecode generator never emits one, loops always target an op_loop_hint).- The
offset += before + afterreorder inadjustJumpTargets()so out-of-line jump targets are keyed by the instruction's final offset, consistent with the newadjustInstructionOffset()used for the type and control-flow profiler maps. - The new "a switch with gaps between its cases" test exercises integer, single-char, and in-loop switches with holes under all four option files, each driven by
testLoopCountand//@ requireOptions.
Extended reasoning...
The diff adds a GeneratorFrameAnalysis class to Source/JavaScriptCore/bytecode/BytecodeGeneratorification.cpp that replaces the per-yield save/restore of all live locals with per-local pruning and an optional write-through policy, fixes jump-offset arithmetic in BytecodeRewriter for fragments inserted after an instruction's label, adds four options, and adds a shared stress suite run under four option combinations. It touches no security-sensitive surface (no auth, injection, or data exposure), but it is correctness-critical: wrong analysis silently corrupts generator locals across suspends, and the ASSERT_ENABLED-only validate() does not run in the release-only CI lanes. The second commit addresses the one finding from the prior run with a targeted fix and regression test, and the hunt ran dry this time, which is why the body acknowledges progress; the size and complexity of the dataflow code and its effect on all generator and async bytecode in every tier are what rule out approving without a human reviewer.
…t was entered or resumed, and does nothing else new
There was a problem hiding this comment.
I re-reviewed the rewritten version (b0bbc2e) and found no bugs; the BytecodeRewriter change my earlier comment was about is gone entirely, and the engine change is now a single forward may-written pass plus a one-line save filter. Since this is a new dataflow analysis in a pass every generator and async function goes through, under /Source/JavaScriptCore ownership, a human look is still worthwhile.
What was reviewed:
- Soundness of
liveness & written: a local live at a yield but not inwrittenmust have been live at the previous resume (no def between), so it was reloaded there and its slot still matches; the entry path is covered becauseop_enterdefs allnumVarsand the state switch is inserted after the analysis, so no switch edges leak entry writes into resume blocks. - Per-block handler lookup on the leader offset:
computePreciseJumpTargetsadds every handler start/end/target as a jump target, so blocks never straddle a handler range; flowing the pre-reset set at each yield and at block end covers every intermediate throw point. - Worklist/bitvector reuse:
in()is only reused after liveness finished anddata.livenesswas copied out;flowTois monotone with the in-worklist guard; checkpointed defs are iterated the same waystepOverInstructiondoes. - Test follows JSTests/README conventions (
testLoopCount, throws on failure,noInline/drainMicrotasksare jsc-shell APIs); I could not run it here, so the 152 ms vs. the 200 ms budget claim is unverified.
Extended reasoning...
The diff touches Source/JavaScriptCore/bytecode/BytecodeGeneratorification.cpp (new GeneratorLivenessAnalysis::computeWrittenLocals, 71 lines, and the save loop now iterating liveness & written), one new Bool option useGeneratorFramePruning in OptionsList.h, and a new 539-line stress test; it touches no auth, injection or data-exposure surface. The prior inline finding (switch range-table gap hang in BytecodeRewriter.h) is moot because that code is no longer part of the PR. Approval is withheld because the changed paths are covered by .github/CODEOWNERS (/Source/JavaScriptCore and /JSTests to the jsc-reviewers team) and because a new dataflow analysis whose correctness rests on a liveness/resume invariant is not a mechanical change, even though my reading found the argument sound.
…its own table, and has one test case for each thing it relies on
There was a problem hiding this comment.
I reviewed the pared-down version of this PR (the last push, which replaced the per-activation design) and did not find any bugs. Because it changes what every generator and async function saves at every suspend point, and the correctness rests on a dataflow argument rather than a local code change, a human look is still worthwhile.
What was reviewed:
- The per-block handler table: handler start, end and target are all fed to computePreciseJumpTargets, so they are block leaders and no block straddles a try range; the exit block's UINT_MAX leader terminates the range walk, and empty/inverted ranges are already dropped in BytecodeGenerator::generate before this pass runs.
- The "clear written at op_yield" step: resume reloads every live local (liveness is unchanged), op_yield itself has no defs, and op_enter defs every var, so a local that is live at a yield and unwritten on every path must have been loaded at the previous resume and still equals its slot.
- Handler flow at a yield happens before the clear and again at block end with all checkpoint defs, so a write before a throw in the same block (including a mid-instruction throw after a checkpoint def) reaches the handler.
- The test directive uses the run(kind, *options) form the runner provides, and a brand-new stress test does not fall under the BUN-TEST-DIFFERENCES.md rule for changed upstream tests.
Extended reasoning...
The net diff adds an 84-line forward may-written dataflow in BytecodeGeneratorification.cpp that narrows the save set at each op_yield to liveness & written, a default-on option in OptionsList.h, and a stress test that runs with the option on and off. It touches no security-sensitive surface (no auth, injection or data exposure), but it alters bytecode emitted for every generator, async function and async generator in the fork. I traced the invariants the pruning relies on (handler boundaries as block leaders, exit-block sentinel, op_enter defs, resume loading all live locals) and found them to hold, and the bug hunt ran dry. Deferring rather than approving because the change is in core engine codegen with a non-local correctness argument and runs in CI only on no-assertion release lanes, so the ASSERTs in the new code never fire there.
Problem
When a generator, async function or async generator is turned into a state machine,
BytecodeGeneratorificationemits, at every suspend point, oneput_to_scopeper live local before suspending and oneget_from_scopeper live local after resuming. Bytecode is O(suspend points × live locals). The saves are the expensive half: in Baseline a frame save is 172 bytes of code and a restore is 24, and in the largest function of a real application the saves alone are 52.6% of its Baseline code.This has to be fixed here: the copies are emitted by JSC's own bytecode pass, and nothing an embedder does can remove them.
What this does
One thing: a suspend point does not save a local that has not been written since the function was entered or last resumed.
Each local already has its own slot in the generator frame. Resuming loads every live local from its slot, exactly as before, so afterwards register and slot agree. If nothing writes the register before the next suspend point, the slot is still right and the save is dead.
saveOnlyWrittenLocalsis one forward may-analysis over the existingBytecodeGraph: what may have been written by the time each block is reached. It is emptied after eachop_yield, and it flows along ordinary edges and from every block to its exception handler. It runs off a worklist, so the work does not depend on how the code is laid out. The save loop then goes overliveness & writteninstead ofliveness.Restores are untouched, nothing new is inserted anywhere,
BytecodeRewriteris untouched, and no opcode is special. At every suspend point the frame holds exactly what it held before, so nothing changes for the GC, the DFG or the debugger.There are no heuristics and no thresholds. The one shortcut is a proof, not a tuning knob: a function with a single suspend point is skipped, because that point is reached from the function's entry and everything live has been written on the way from there.
useGeneratorSavePruning(default true) turns it off.The engine change is 84 added lines in
BytecodeGeneratorification.cppand one inOptionsList.h.History
This PR used to do much more: it also pruned restores, and chose per local between copying at suspend points and keeping the local in the frame. That got further (−39% Baseline code on the functions below instead of −33%), but it needed a cost model, a rule about loops, new insertion points in
BytecodeRewriter, and special cases for several opcodes. Review found a hang in the rewriter change, and my own review found that it could pin dead objects in the frame and that two code shapes compiled slower than before, one of them 3×. All of that is removed here rather than patched. The first two commits are that version; the last one replaces it.Results
Release build, one binary, option off vs. on. Program output is identical in every row.
Synthetic async generator with S suspend points and L live locals (each activation touches one local):
(The save counts include a few
put_to_scopethat are not frame saves.)A real 16 MB application bundle (4,291 generator and async functions, every function compiled with
generateBytecodeCacheFile): 41,232 → 19,388 saves (−53%).Across that bundle and a second 9 MB one, 1,930 of 6,013 functions have fewer saves, 4,083 are unchanged, and none has more.
The three largest functions, each extracted and called once with
--jitPolicyScale=0to get its real Baseline compile:The 11 largest together: bytecode 657,079 → 516,671 (−21%), Baseline code 4,825,536 → 3,251,776 (−33%).
That application's real agent loop in Bun, driven offline by a local mock of its API (streamed text and tool calls), medians of 15 runs. The option-off case was measured twice, interleaved, to show the noise:
JIT memory is the solid result: its runs differ by about 1%. Peak RSS varies by up to 10% from run to run at 300 turns, so only the 50 and 100 turn rows say anything.
This is a memory and compile-time win for programs with large async functions, and ordinary programs will not notice it. hono and fastify under load, rollup bundling svelte's compiler, web streams and a CLI's
--help, medians of 9 runs: JIT memory, peak RSS and CPU are all within the difference between two runs with the option off. (Two readings were not at first, rollup's peak RSS and the streams CPU time; repeated 15 and 25 times in four interleaved groups, the medians are 500 / 509 MB off against 495 / 500 on, and 710 / 708 ms off against 708 / 699 on.)bun build --bytecodeoutput: hono unchanged, fastify 1,580,696 → 1,580,392 bytes.Cost of the analysis: about 1% of bytecode generation. User-mode instructions for the whole parse + generate + encode pipeline (
perf stat), in millions, medians of 7 interleaved runs (the runs of one configuration differ by 0.15% at most):That is one more decode of each instruction in the functions that have two or more suspend points. It is paid once, when a function is first compiled to bytecode. (An earlier version of this description said +0.1% for the 16 MB bundle. That came from a single run with a bad reading and was wrong.)
Shapes chosen to be hard on it, 100 locals unless noted, medians of 3:
None is slower than before by more than 1%, and no ratio grows with size. The worklist visits each block 1.00 to 1.55 times in these, and 1.26 times on average over the 16 MB bundle (2.63 in its worst function). (The pass as a whole is quadratic in several of these, before and after: the liveness analysis calls
handlerForBytecodeIndex, a linear scan, for every instruction on every iteration, andBytecodeBasicBlock::computeImpllinks blocks by scanning all of them.)Speed. Nine microbenchmarks, medians of 9 processes, best of 40 rounds in each, option off measured twice:
Nothing is slower. A few are a little faster, most of all the generator with many locals; the loop that suspends rarely is too noisy to read.
Not done here
Restores are still O(suspend points × live locals), so bytecode only roughly halves in the worst cases. They are cheap in Baseline, and a follow-up that gives frame copies their own compact opcodes makes them cheaper still, so I would rather see whether pruning them is still worth its complexity after that.
The two largest functions above stay over
maximumOptimizationCandidateBytecodeCost(100,000), so this does not change which functions the DFG will compile.Testing
New
JSTests/stress/generator-saves-written-locals.js: 12 small functions, each one way for a write to reach a suspend point with no other way for it to get there: in the suspend point's own block, in an earlier block, on one of two paths, in an earlier loop iteration, in a try block that throws (early and late in the block, in the inner of two, and where the handler throws too), byop_enter, part way through an instruction that can still throw, in an async function, and not at all. It runs in the default modes and once with the option off. Slowest of the 18: 115 ms. V8 agrees with the expectations of the generator cases.Negative testing, case by case. I broke the change in 11 ways, one per thing it relies on, and ran every check in the test against every breakage in its own process:
op_enterwrites nothingEvery breakage is caught by the case written for it, and every case catches at least one.
Object lifetime. A
WeakReftest of what a suspended generator keeps alive gives the same answer with the option off and on in all five of its cases.Differential fuzzing with a generator of random generator / async / async generator programs (nested try/catch/finally, labeled break and continue, loops that suspend always, sometimes and never, switches with gaps, for-of and for-await with closable iterators,
yield*, TDZ, closures, suspends inside expressions, driven by randomnext/throw/return). Output with the option off in the LLInt is compared with the option on in 9 configurations (LLInt, Baseline, DFG, FTL with eager thresholds, debugger bytecode, type and control flow profilers, the bytecode optimizer,collectContinuously). 640 seeds on the final code, all identical; it finds each of the 11 breakages above in 4 to 102 of 120 seeds.