Skip to content

test/aot: check stderr, exit code and signal of both runs of each request shape - #48723

Open
robobun wants to merge 1 commit into
mainfrom
robobun/03259465/request-shapes-whole-result
Open

robobun wants to merge 1 commit into
mainfrom
robobun/03259465/request-shapes-whole-result

Conversation

@robobun

@robobun robobun commented Nov 12, 2026

Copy link
Copy Markdown
Collaborator

Problem

  • test/aot/node-http-calls-what-another-file-laid-over-a-method-of-a-request.test.ts reads only stdout of plain Bun, and stdout and the exit code of the compiled program. A run that warns twice or writes another line to stderr passes. So does a plain run that exits with 3.
  • The file takes 175 s on debian 13 x64-asan (median of 10 builds). Its 36 builds each compile node:http again. This PR does not make it faster (Notes).

Fix

  • Plain Bun and the compiled program must each give the whole result. That is called 2, one DeprecationWarning with its hint line on stderr, exit code 0 and no signal.
  • Only the process number and the executable's name are open in stderr.
  • Verified: bun bd test of the file takes 1,058 s, before 1,295 s and 670 s. In each run the debug compiler crashes in 12 to 16 of the 36 builds (AOT: debug/ASAN compiler crash in LinkFlow::findWhatSomeLaneWasGiven #48169). 36 pass on release builds, one with ASAN.

Background

  • A shape is a program of two files. One lays a function over push on the prototype of an heir of http.IncomingMessage. The other serves one request and counts the calls of that function.
  • Compiled node:http ends a request's body on a short road while the request finds the push of its class (src/js/ahead-of-time/node/_http_incoming.ts:1173).
  • Considered one build for several shapes. The compiler makes other code of node:_http_incoming for 9 of the 36. A shape would not run there as it runs alone.
  • Considered builds without a copy of Bun (AOT_PROGRAMS_ALONE, a switch for the whole tree): 8% less CPU time with ASAN.
Notes

What the file starts. 36 builds, 36 runs of plain Bun and 36 runs of what was built. This PR changes none of them.

CI, the file as on main. Seconds, from the logs of the test-bun jobs of ten builds of other pull requests (2026-11-11 and 2026-11-12). By runsOnThisLane() of scripts/runner.node.ts no other lane runs the file.

Build debian 13 x64-asan debian 13 x64 debian 13 aarch64
140086 177.56 53.50 34.72
140389 171.03 45.29 36.84
140391 182.94 41.68 37.14
140396 175.32 43.67 33.55
140403 168.44 50.72 34.12
140423 157.79 46.31 33.14
140435 181.54 46.98 37.38
140454 129.14 44.50 32.19
140457 187.66 43.82 34.81
140459 175.25 44.55 38.66
Median 175.3 44.9 34.8
  • The ASAN lane has 8 processors. It runs the file beside two other files, with --max-concurrency=8.
  • The x64 and aarch64 lanes have 4 processors. They run the file beside one other file, with --max-concurrency=4.
  • The file passed at the first try in each of these runs.

Why still 36 builds. test/aot/README.md says to use one build for many cases, and to keep one case to a program where a case can change what the compiler knows about the others. Here it can.

  • I built each shape with BUN_JSC_aotMapFilePath. The map lists every compiled function with its size and its operations. A build has 4,830 functions, 4,816 of them outside the two files of the shape.
  • Those library functions are not the same in every build. The 36 builds make 5 different sets of them. The set follows the way to lay, never the way to inherit.
Way to lay Shapes Library functions with other code than in the first shape Where More library functions
Ask.prototype.push = o, Object.assign, util._extend, Reflect.set, a value by Object.defineProperty or Object.defineProperties, const held = Ask.prototype; held.push = o 21 0 0
a key from Object.entries(), the key '' + unseen('push') 6 32 node:_http_incoming, internal:http, internal:http1_server_fallback, the streams 17
__defineGetter__ 3 29 the same modules 0
a getter by Object.defineProperty 3 15 the streams 0
a key from Object.keys() 3 3 internal:util/inspect, internal:primordials 17
  • Among the 32 are the functions with the short roads: endsItsBodyUnwatched, takeBodyData, endBody and keepsItsMethods of node:_http_incoming, onHttp1Body and onHttp1MessageComplete of internal:http1_server_fallback, addChunk of internal:streams/readable.
  • Two builds of one shape give the same map.
  • So a program with several ways to lay is compiled as the most careful of them asks. A way that the compiler must notice by itself would then run on code that another way has already made careful.
  • The 21 shapes with the same library code still differ in what the compiler knows of Ask. One program for them holds three classes, or one class with seven stores, and the file asks about one class with one store.

Assertions, before and after.

Before After
The build built(): exit code 0 and no line of stderr that begins with error the same
stdout of plain Bun called 2 the same
stderr of plain Bun not read one DeprecationWarning: m and the hint line below it, and nothing else
How plain Bun ends not read exit code 0, no signal
stdout of the compiled program what plain Bun printed called 2
stderr of the compiled program not read as for plain Bun
How the compiled program ends exit code 0 exit code 0, no signal
  • expect() calls: 108, as before. Each of the two for the runs now compares the whole result of spawned().
  • No snapshot: the tests are concurrent, and a snapshot matcher throws there (Snapshot matchers are not supported in concurrent tests, src/runtime/test_runner/expect.rs:1417).
  • A failure shows the whole stderr of the run.

A fault made by hand, in 4 of the 36 shapes, on the release build:

Fault The file of main This file
None 4 pass 4 pass
The program writes one more line to stderr, in both runs 4 pass 4 fail
Plain Bun exits with 3 behind its line 4 pass 4 fail
The compiled program does not warn 4 pass 4 fail
The compiled program warns twice 4 pass 4 fail
The compiled program counts one call fewer 4 fail 4 fail

Where the time goes. Linux x64, a VM with 12 processors and 16 GB that shares its host. All binaries are built from 7f964e6, the parent of this branch when it was measured. The branch is on ccb3611 now, and neither the file nor a helper that it imports differs between the two. On the release build of ccb3611 the file passes too: 36 pass.

  • Release build: the bun of bun-linux-x64.zip that the build of main at 7f964e6 made (140544). Started as scripts/runner.node.ts starts the file: its environment, a TMPDIR of its own, 4 processors (taskset), --max-concurrency=4. The file takes 33 s to 48 s and about 120 s of CPU time: 3.3 s for each shape. CI takes 45 s there, beside another file.
  • One build there takes 4.0 s of user time and 0.9 s of kernel time, measured beside other work: 0.3 s to bundle, 0.7 s to check, the rest to compile.
  • Release build with ASAN: bun scripts/build.ts --profile=release-asan, stripped with llvm-strip --strip-debug as the runner strips it (616 MB). One build on one compiler thread takes 15.3 s and 15.9 s (12.0 s to 12.7 s of user time): 0.7 s to bundle, 2.2 s to check, 13.0 s to compile. Plain Bun runs the program in 0.74 s, the executable runs in 0.26 s.
  • With ASAN, 8 processors and --max-concurrency=8, the 36 builds take 1,545 s of the 1,731 s that the steps of all cases take together: 89%. The runs of plain Bun take 115 s and the runs of the executables 71 s.
  • So a lane is as fast as it compiles node:http 36 times. Nothing in the file waits for a clock, a port or a fixed sleep.

Builds that write no copy of Bun: measured, not in this PR. With BUN_AOT_WRITES_PROGRAM_ALONE=1 a build writes the program with a launcher at its head, and not a copy of Bun with the program behind it (test/aot/harness/program-alone.ts). test/aot/run-parallel.ts builds the stored programs so.

Executables, as on main The program alone
Bytes that the 36 builds write, release build 36 x 75.0 MB = 2.7 GB 36 x 5.0 MB = 0.2 GB
The same with ASAN 21.3 GB 0.2 GB
One build with ASAN, one compiler thread 15.3 s, 15.9 s (2.4 s of kernel time) 14.1 s, 13.4 s (1.2 s, 1.3 s)
The file, release build, 4 processors: time 48.1, 32.9, 36.3, 34.8 s 37.9, 40.6, 35.8 s
The same: CPU time of the machine 124, 116, 121, 118 s 117, 119, 116 s
The file with ASAN, 8 processors: time 244.9, 215.3, 211.0, 178.1, 209.7, 299.6 s 213.3, 215.3, 102.6, 179.0, 267.5, 254.3 s
The same: CPU time of the machine 718, 704, 701, 683, 718, 737 s 654, 652, 596, 655, 699, 676 s
  • The two files ran in turns. 36 pass in every run.
  • The CPU time of the machine is user and kernel time from /proc/stat. It does not count the time that the host takes from this VM. With ASAN that time was 0.4 to 2.4 times the CPU time of a run, so the times of two runs do not compare.
  • With ASAN the program alone takes 7.7% less CPU time (655 s against 710 s, the means of 6 runs). That is about 1.5 s for each build: the copy of 616 MB.
  • On the release build it takes 2% less (117 s against 120 s), and no time that these runs can show.
  • So the ASAN lane would take about 8% less, and the release lanes the same. That is not enough to change what the file starts: with the program alone, CI starts no shape from an executable.
  • AOT_PROGRAMS_ALONE=1 around a run is that switch for every file of test/aot. It is off in CI.

Plain Bun beside the build: measured, not in this PR. A case can start its plain run while its build runs. On the release build the file then takes 33.7, 36.0 and 37.4 s, and 36.0, 32.9 and 35.3 s with the program alone too. That is the same as without. The processors are busy with builds either way.

bun bd test. A debug build at 7f964e6, the time that bun test prints. A Bun with ASAN has 5 cases under way.

Run File Time Pass, fail CPU time of the machine Time that the host took from the VM
1 of main 670.4 s 20, 16 3,700 s 3,912 s
2 this PR 1,058.1 s 20, 16 4,046 s 8,146 s
3 of main 1,295.3 s 23, 13 4,186 s 10,591 s
  • Runs 2 and 3 ran back to back, two hours after run 1. A run of this PR's file between 1 and 2 was cut at 33 of 36 cases when the machine restarted: 21 pass, 12 fail.
  • Every red case but one is a build that a race of the debug compiler ended: AddressSanitizer: SEGV on unknown address 0x000000000008 in WTF::removeIterator, then Stopped while compiling in scan and Link. That is AOT: debug/ASAN compiler crash in LinkFlow::findWhatSomeLaneWasGiven #48169, which says that only an engine built without NDEBUG has it. The release build with ASAN did not crash in 13 runs of the file here. The one other red case is a build of run 3 that was ended at its limit of 240 s.
  • A case whose build crashes ends early. In runs 2 and 3 a case with a whole build takes 137 s to 249 s, and a case with a crashed build 66 s to 149 s. So the time of a run follows how many builds crash and what the host takes from the VM.
  • This PR adds no process and no work to a case.

What I could not run. macOS and Windows. CI runs the file on the three Debian lanes only.

…uest shape

node-http-calls-what-another-file-laid-over-a-method-of-a-request.test.ts read
only stdout of plain Bun, and stdout and the exit code of the compiled program.

Plain Bun and the compiled program must now each give the whole result of a
run: `called 2` on stdout, one DeprecationWarning with its hint line on stderr,
exit code 0 and no signal. The expression for stderr leaves open the number of
the process and the name of the executable.
@robobun

robobun commented Nov 12, 2026 •

Copy link
Copy Markdown
Collaborator Author

Status: ready for a maintainer. This PR is #48723. The diff is green on every lane that ran.

CI on 0032206 (build 140691, finished):

  • All 184 jobs that ran passed.
  • The changed file passed at the first try on the three lanes that run it: 146.7 s on debian 13 x64-asan, 46.8 s on debian 13 x64 and 29.6 s on debian 13 aarch64. As the body says, this PR does not change those times.
  • The build is red for one lane: the four darwin-x64-any-test-bun jobs never found a host. They expired in the queue four times between 13:51 and 19:05 UTC. That lane does not run this file.
  • The lane has no host for any pull request now: in the 30 builds created between 17:14 and 19:07 UTC, none of its jobs has started. So I did not start CI again.
  • One test passed on a retry on debian 13 x64: test/aot/mock-factories.test.ts. This PR does not touch it.

How the numbers in the body were made:

  • CI: the time that the log of each test-bun job prints for the file, in ten builds of other pull requests.
  • Locally: bun bd test test/aot/node-http-calls-what-another-file-laid-over-a-method-of-a-request.test.ts on a debug build. Also the file under a release build and under a release build with ASAN, each started with the environment and flags that scripts/runner.node.ts gives it.

@robobun

robobun commented Nov 12, 2026 •

Copy link
Copy Markdown
Collaborator Author
Updated 11:05 AM PT - Nov 12th, 2026

❌ @robobun, your commit 0032206 has some failures in Build #140691 (All Failures)


🧪   To try this PR locally:

bunx bun-pr 48723

That installs a local version of the PR into your bun-48723 executable, so you can run:

bun-48723 --bun

@coderabbitai

coderabbitai Bot commented Nov 12, 2026

Copy link
Copy Markdown
Contributor

Explain Change →

No actionable comments were generated in the recent review.

Walkthrough

The test now checks plain Bun and AOT execution against the same expected stdout, exit code, signal, and deprecation warning on stderr.

Changes

AOT HTTP call test

Layer / File(s) Summary
Execution result assertions
test/aot/node-http-calls-what-another-file-laid-over-a-method-of-a-request.test.ts
The test defines shared expected results for stdout, exit code, signal, and the deprecation warning. It compares both plain Bun and AOT execution results against those expectations.

Suggested reviewers: jarred-sumner

Merge Risk: Minimal · up to 00322

This change only tightens an AOT test's checks on stdout, stderr, exit code and signal. It has no production impact, and no actionable merge-blocking risk remains.

Pre-merge checks: Passed 4

Check name Status Explanation
Title check Passed The title names the main change: the AOT test now checks stderr, exit code and signal for both the plain Bun run and the compiled run of each request shape. It is specific and matches the raw summary,…
Description check Passed The description covers the problem, the fix, verification, and background. It fills the 'What does this PR do?' content through its Problem and Fix sections, and the 'How did you verify your code work…
Linked Issues check Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check Passed Check skipped because no linked issues were found for this pull request.
Full details: Title check

Explanation

The title names the main change: the AOT test now checks stderr, exit code and signal for both the plain Bun run and the compiled run of each request shape. It is specific and matches the raw summary, which describes the same assertion changes in the single test file. It is slightly long at 79 characters, but it remains clear.

Full details: Description check

Explanation

The description covers the problem, the fix, verification, and background. It fills the 'What does this PR do?' content through its Problem and Fix sections, and the 'How did you verify your code works?' content through the timing, fault-injection, release/ASAN, and bun bd test measurements. It is very long, with much detail in collapsed notes, but the template's required information is present.

  • Autofix · Keep fixing CodeRabbit findings and required CI, and resolving merge conflicts

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

Share

Recent review info

Run configuration

Commits

Reviewing files that changed from the base of the PR and between ccb3611 and 0032206.

Files selected for processing: 1

  • test/aot/node-http-calls-what-another-file-laid-over-a-method-of-a-request.test.ts

Included review availability: This review used your included allowance. Your plan provides up to 10 included reviews per hour; 0 remain after this review.

@claude claude Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code review found no issues

No high-confidence issues detected in this change.

@robobun

robobun commented Nov 12, 2026

Copy link
Copy Markdown
Collaborator Author

The diff of this PR is ready. I cannot repeat bun bd test on this branch: a debug build of its base, main at ccb3611, does not compile. This PR changes no source file.

  • bun bd stops at src/jsc/bindings/BunPlugin.cpp:3528: error: no member named 'SuspendExceptionScope' in namespace 'JSC'.
  • AOT: strings that packages evaluate, modules in bundles, class factories, mocks in import cycles #48677 removed #include <JavaScriptCore/FrameTracers.h> from that file. The file still uses JSC::SuspendExceptionScope.
  • A release build compiles it. There the file is in a bundle of 32 sources, behind BunCPUProfiler.cpp, which includes the header. A debug build makes bundles of 8 (scripts/build/unified.ts:194), and no file in front of BunPlugin.cpp in its bundle includes the header.
  • With the include back in BunPlugin.cpp, that translation unit compiles here.

The bun bd test times in the body are from 7f964e6, one commit earlier. CI builds release binaries, and those check this PR.

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant