Repository navigation
test(aot): run the layered runs of callbacks-named-by-another-program-of-a-test-run.test.ts concurrently and state every line - #47663
Conversation
… concurrently and state every line Each `bun test --aot` run in layers is a test of its own, and the six tests are concurrent, so `bun test --max-concurrency` says how many of them are built at once. A run compiles all of Bun's own JavaScript, and the six runs went one after the other. Every run, plain Bun's too, must print the 18 lines that the file states and the summary of two passing test files, and exit with 0. Before, two of the 18 lines were checked, by substring. A debug build skips the file and says why. Plain Bun lists a private helper of flatMap in a stack there, so the file failed at its first comparison, and one run in layers takes five minutes or more.
|
No actionable comments were generated in the recent review.
|
| Layer / File(s) | Summary |
|---|---|
Run output assertions test/aot/callbacks-named-by-another-program-of-a-test-run.test.ts |
The test parses the final summary from stderr and checks expected stack lines, successful two-test summaries, and exit code 0 for each run configuration. Debug builds skip the tests. The timeout increases from 240 to 300 seconds. |
Suggested reviewers: jarred-sumner
Merge Risk: 
Minimal · up to 90e55
-
The change strengthens AOT output checks while isolating concurrent test fixtures. No actionable merge-blocking issue was identified; normal checks should pass before merging.
Pre-merge checks: 
4
-
Full details: Description check
-
Explanation
The description clearly explains what the PR changes and how the change was verified. It includes implementation details, performance measurements, debug-build behavior, CI results, and test evidence. The required template headings are not used verbatim, but the required information is present and complete.
-
- Autofix · Keep fixing CodeRabbit findings and required CI, and resolving merge conflicts
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.
|
Status: closed without a merge. #47942 rewrites the front end of the ahead-of-time compiler and its tests, and it deletes What this pull request measured, for the record:
The description has every run and how it was taken. |
There was a problem hiding this comment.
Nothing blocking. The comments below are optional suggestions. There is no need to push a fix for them before merging.
Beyond the inline note on the per-test timeout, I also checked the hardcoded summary lines against the runner's summary printer in src/cli/test_command.rs (N pass, N fail, N expect() calls, Ran N tests across N files. plus a [N.NNms]/[N.NNs] suffix from src/bun_core/output.rs) — the trim-and-strip regex in seenIn matches that output exactly, and the AOT test run | ... lines that could land after the summary are gated on BUN_JSC_logAOTPhaseTimes, so they only matter when a developer has that set. The test.skipIf(isDebug).concurrent.each chain is also handled by the alwaysSkipped proxy in test/aot/supported.ts when --aot is unsupported.
Extended reasoning...
The change rewrites one test file, test/aot/callbacks-named-by-another-program-of-a-test-run.test.ts, splitting six layered bun test --aot runs into concurrent per-layer tests and replacing substring checks with the full expected 18-line stdout plus a stderr summary and exit code; it touches no production code and no security-sensitive surface. The restructure is correct as far as I can trace it, but the choice to run six Bun-library builds concurrently (the author reports up to ~9.5 GB peak) and the exact-output coupling to stack-frame rendering are design trade-offs a test/aot owner should weigh, so a human look is still worthwhile rather than an automated approve.
…ot 600 s A test holds one run in layers. With six of them at once a test took 158 s on the lane with AddressSanitizer (build 136542) and at most 275 s in the local runs. On main a test holds two runs in layers and has 240 s.
There was a problem hiding this comment.
LGTM — the second commit brings the per-test ceiling down to 300 s, close to the measured worst case for one layered run, and the rest is a test-only tightening that replaces two substring checks with the full 18-line output, the summary and the exit code.
What was reviewed:
- Summary parsing against
print_summary/print_elapsedin src: durations are onlymsors,bunEnvsetsNO_COLOR, and theAOT test run | largest workerline prints only underBUN_JSC_logAOTPhaseTimes, so nothing lands afterRan 2 tests across 2 files.in the slice. - The
isDebugskip:showPrivateScriptsInStackTracesis set underBUN_DEBUGin ZigGlobalObject.cpp, so the flatMap helper frame is debug-only and theSKIPPED:stderr line follows the test/aot/README.md convention. test.skipIf(...).concurrent.each(...)is a chain used across the suite, andsupported.ts'salwaysSkippedproxy handles it when--aotis unsupported.
Extended reasoning...
The PR touches one test file, test/aot/callbacks-named-by-another-program-of-a-test-run.test.ts, splitting three sequential tests into six concurrent ones and asserting every output line, the bun test summary and the exit code with a single toEqual. It touches no security-sensitive surface. The only prior review comment was this bot's nit on the 600 s timeout, which the second commit lowered to 300 s; no CODEOWNERS entry covers test/aot, the bug hunt ran dry, and the stderr parsing and skip condition were checked against the runtime source.
|
Closing: #47942 rewrites the front end of the ahead-of-time compiler and its tests for the engine's new compiler (oven-sh/WebKit#997, merged), so this no longer applies and would conflict with it. If the problem still shows once that is in, please open a new pull request against |
|
Understood. #47942 deletes |
Problem
test/aot/callbacks-named-by-another-program-of-a-test-run.test.tstakes 467 s of the 496 s of its parallel batch on debian 13 x64-asan.bun test --aotruns in layers go one after another. Each compiles Bun's own JavaScript library again.Fix
bun test --max-concurrencysays how many build at once.flatIntoArrayWithCallbackthere, so main's file fails at line 91.bun bd test: 3 fail in 5.5 s before, 6 skip in 2.4 s after.Background
Notes
What the file asks, and what stays as it was. The project, its three variants (the package, the application or the test file gives the names), the commands and the words
TestRunLayers:1,TestRunPartFiles:1andTestRunLayers:2,TestRunPartFiles:1are unchanged. Each variant still runs under plain Bun and in both forms of layers: nine commands before, twelve now, because each of the six tests starts its own plain run (0.05 s with the release build, 0.2 to 0.4 s with release-asan).run()ofharness/run.tsstill fails a run that was not built in the layers it asked for. A test has 300 s. It holds one run in layers, where a test of main holds two and has 240 s, but up to six run at once: a test then took 158 s in CI and at most 275 s locally. With two at once in a batch of CI a test took about 124 s (372 s for three rounds of two).Assertions, before and after.
2 pass,0 fail,2 expect() calls,Ran 2 tests across 2 files.for every runTwo faults show the difference. Each was put into the project text of both versions of the file and run with the release build.
a.test.jsprints the line "one that the package calls" before it nameseach, so every run printseachthereb.test.jsskips its test1 pass,1 skip,1 expect() callsWhere the time goes. One run in two layers with the release build and
BUN_JSC_logAOTPhaseTimes=1: the layer that holds the package and Bun's own code takes 18.4 s of 19.4 s (250 units, 11,028 functions). The layer of the application, both test files and both workers take under 1.3 s together. So the file is six builds of Bun's library, and the processor time stays the same: 152 and 158 s before, 152 and 154 s after (release, 4 CPUs). Only the order changes. The layer holds that much of the library because the project makes two functions from text at run time (see the last section).All local runs. Linux x64, 12 vCPUs, 15.6 GB,
tasksetfor fewer CPUs. Each row is one round, taken in turn. The host takes a changing part of the machine: the rounds differ more than the columns of a round.release-asan (
bun run build:asan), 8 CPUs, the environment thatscripts/runner.node.tsgives a file on the lane with AddressSanitizer:--max-concurrency=8--max-concurrency=2scripts/runner.node.tsitself (--exec-path build/release-asan/bun-asan, 8 CPUs), this PR took 190 and 131 s. Main's file failed there after 722 s, and with the raised limit the runner ended it at the 1080 s that it gives a file.--max-concurrency=8is what the runner passes when the file runs in a process of its own.--max-concurrency=2is what a parallel batch gets there.r7i.2xlarge).release (
bun run build:release,bun-profile), 4 CPUs, the environment of a lane without a sanitizer:--max-concurrency=4--max-concurrency=2debug (
bun bd test):Received: "SEEN one of the test file | probed < visited < step < for each < in a package < flatIntoArrayWithCallback < forEachOf < work < <anonymous>". A debug build setsshowPrivateScriptsInStackTraces(src/jsc/bindings/ZigGlobalObject.cpp, underBUN_DEBUG), so plain Bun lists the private helper thatflatMapcalls. Compiled code does not list it. No build with AddressSanitizer alone does this, so the lane with it is not affected.SKIPPED:line on stderr with the reason.CI, before. Build 136323: 467 s on debian 13 x64-asan, 153 s on x64 and 102 s on aarch64, the only lanes that run the file. It ran inside a batch of 35 files:
bun test --parallel=7 --max-concurrency=2on x64-asan (the batch took 496 s),--parallel=3 --max-concurrency=2on x64 (153 of 265 s). The file is in such a batch becausetest/parallel-allowlist.jsonlists the folder and was made before the file existed. This PR leaves that list alone.CI, this PR. The same shard of each lane held the file in all three builds.
--max-concurrency=2, beside six other files on x64-asan and two on the others. That column is like for like with main.--max-concurrency=8beside two other files on x64-asan,--max-concurrency=4beside one on the others.Designs that were not taken.
scripts/build/embed-compiled-library.ts, asthe-library-that-bun-holds-compiled-is-the-lowest-layer.test.tsmakes one): one build of the library. The package is then a program of its own above Bun's own code. In the Bun under test the two are one program.new Function("f", "return f"). Because of them the lowest layer holds about twice as much of Bun's library: 249 units and 11,025 functions, against 128 and 4,007 when both are writtenf => f, and one run takes 10.6 s and not 6.5 s (release, 4 CPUs). They hand each callback through code that the compiler cannot read, and the tests ask about callbacks that it has lost sight of. No commit on main has both layers and the fault that these tests were written for, so a cheaper project cannot be shown to find it.[auto-merge] gate passed · iteration 0 · 1 files touched
passes on PR (with fix)
diff hotspot
gate history · 1 passed · 0 rejected · iteration 0
evidence per changed file