Repository navigation
Conversation
|
Updated 11:05 PM PT - Nov 12th, 2026
❌ @robobun, your commit 9a291fe has 1 failures in
🧪 To try this PR locally: bunx bun-pr 48718That installs a local version of the PR into your bun-48718 --bun |
|
Status: ready for review. This PR is #48718. CI:
How the numbers in the description were measured:
|
|
No actionable comments were generated in the recent review. WalkthroughThe module-identity harness validates test and file totals and exit status for each plain test run. It also rejects duplicate known-list entries and includes mismatch details in the expected comparison. ChangesModule identity harness
Suggested reviewers: Merge Risk:
|
There was a problem hiding this comment.
LGTM — a small, self-contained tightening of the module-identity test harness with no production code touched.
What was reviewed:
- The new
ran/exitCodeexpectations against the runner source:print_summaryalways prints "Ran N test(s) across M file(s). ",filesis counted once per file (not per--rerun-eachrepeat) while each repeat's(pass)line and summary count increment, the " N fail" line is always printed, and exit code 1 is set iff failures, unhandled errors between tests, or load failures occur — consistent with whattestFilesOfandplain.answer.tests.lengthcompare against forstraight,looped,shared,reducedandheldcases (including--preload ./setup.mjsand--rerun-each=3args, which do not match.test.<ext>). expect(actual, message): Bun's custom label replaces only theexpect(...).toBe(...)signature and keeps the string diff, so the replacedconsole.errorloses nothing.- Not run here:
bun bd test test/aot/module-identity-*.test.tsand the planted-fault checks the description reports; this reading relies on the runner source and the author's reported runs.
Extended reasoning...
The diff changes only test/aot/harness/module-identity.ts (+33/-20): the plain bun test run of each case must now report a summary line whose test and file counts match the printed test lines and the test files named on the command, with an exit code that matches the fail count; the known-list check also flags keys listed twice; and a console.error dump becomes the custom message of the final expect. It touches no production code and no security-sensitive surface, and the file is not covered by CODEOWNERS. Approval rests on cross-checking the hard-coded summary and exit-code expectations against src/runtime/cli/test_command.rs and on every case definition in module-identity-cases.ts and module-identity-held-cases.ts naming its test files explicitly as .test.mjs/.test.ts args; the four candidate issues from the ruled-out list (exit 1 with "0 fail", rerun load failures, non-.test. file names, summary wording changes) are all intended tightenings or hypothetical future drift rather than defects in this change.
The run of a case under `bun test` is its specification. It passed with one SAID line and no crash. So a project whose two runs both ended before the last test file passed. - The `bun test` run must count every test file that its command names, have a line for each test that it counts, and exit with the code of the failures that it counts. - "every listed case exists" also fails on a case that module-identity.known.txt names twice. The later line counted in silence. - A case whose line differs from the listed one fails with the differing columns in its message. They went to stderr before.
395e175 to
9a291fe
Compare
Problem
test/aot/module-identity-15.test.tstakes 189 s on debian 13 x64-asan (build 140086). Its tests are intest/aot/harness/module-identity.ts. This PR does not make it faster (Notes).bun testrun, the specification, only has to print oneSAIDline and not crash.test/aot/module-identity.known.txtcan name a case twice. The later line counts in silence.Fix
bun testrun must be whole. Its summary counts every test file, each counted test has its line, and the exit code matches the failures.bun bd test test/aot/module-identity-15.test.tstakes 733 s and 782 s before, 967 s and 1,025 s after. At the same time on 27 cases: 369 s and 371 s (Notes). The release canary passes all 20 files, and three planted faults fail only here.Background
bun testandbun test --aot. The list names the known differences.bun test --aot: 0.7 to 0.9 s of CPU in a release build.bun testneeds 15 ms.bun test. A change of its words fails every case.Behaviour change: none
Notes
The files. The shared module drives these 20 wrapper files. None of them changes.
test/aot/module-identity-1.test.tstest/aot/module-identity-2.test.tstest/aot/module-identity-3.test.tstest/aot/module-identity-4.test.tstest/aot/module-identity-5.test.tstest/aot/module-identity-6.test.tstest/aot/module-identity-7.test.tstest/aot/module-identity-8.test.tstest/aot/module-identity-9.test.tstest/aot/module-identity-10.test.tstest/aot/module-identity-11.test.tstest/aot/module-identity-12.test.tstest/aot/module-identity-13.test.tstest/aot/module-identity-14.test.tstest/aot/module-identity-15.test.tstest/aot/module-identity-16.test.tstest/aot/module-identity-held-1.test.tstest/aot/module-identity-held-2.test.tstest/aot/module-identity-held-3.test.tstest/aot/module-identity-held-4.test.tsTime of
test/aot/module-identity-15.test.ts, before and after. The diff adds a pattern match over lines that are already read. So the two sides do the same work.bun bd test test/aot/module-identity-15.test.tsbun test test/aot/module-identity-15.test.tsstealin/proc/stat). That share swings by the minute.loopedcases took a median of 16.6 s in the second run before and 71.2 s in the second run after. The 85straightcases took 27.8 s and 40.6 s before, 40.6 s and 37.1 s after.mainlay beside the changed one under another name, with a wrapper file of its own. On 27 cases (-t "straight \| (named|namespace) ") the module ofmaintakes 369.3 s and 903 s of CPU. The changed one takes 370.9 s and 884 s of CPU. Each side passes its 27 cases.src/options_types/context.rs:648). CI passes--max-concurrency=8to this file on its 8 processors.Where the time is. Release canary of 7f964e6, one case at a time.
bun testbun test --aotbun test --aotcompilesBUN_JSC_logAOTPhaseTimes=1)vi.doMock()bun testruns together take 44 s of CPU. The 20 wrapper files together take about 2,100 s of CPU.test/aot/module-identity-15.test.tsis the slowest of the 16 in builds 140086 and 140295. That holds on debian 13 x64-asan and on debian 13 aarch64. Its cases are not heavier. In a run of every case its part costs 129 s of CPU. The 16 parts cost 124 s to 137 s. The runner puts the same files beside it in each build. One of them istest/aot/node-http-calls-what-another-file-laid-over-a-method-of-a-request.test.ts(178 s and 164 s on the ASAN lane).What does not make it faster.
heldproject. They differ in the environment alone, butbun test --aothas no way to run a program that is already compiled.test/harness.tsgives the children of every file undertest/aotMalloc=1and its ASAN options. The runner sets the exception checks (BUN_JSC_validateExceptionChecks) for the file, and the children inherit them. Other files use both, so this PR leaves them. I have no release build with ASAN to measure what they cost.The new rule holds today. I ran all 1,967 projects under
bun testbefore I wrote it. Every run prints the summary line. It counts as many files as its command names, and as many tests as it has lines for. 1,962 runs exit with 0 and "0 fail". 5 exit with 1 and "1 fail": arequire()of an async module throws inside their test. That was the canary of 7f964e6. The canary of ccb3611 gives the same counts.Every case with the change. All 20 wrapper files pass with the changed module on the release canary of 7f964e6: 1,968 tests. The branch is now on
mainat 9ea3c69. The canary of that commit passes all 20 files again: 1,968 tests.CI of this PR, first push. Build 140670, on
mainat ccb3611. All 20 wrapper files pass at the first try on the three lanes that run them. Build 140651 is the build ofmainat the same commit. A build ofmaindoes not run the ASAN lane.module-identity-15,mainmodule-identity-15, this PRmainmainat ccb3611.test/aot/parallelthat this PR does not touch.one-data-file-under-two-types/untyped-import-answers-before-text/main.mjsfails on ubuntu 25.04 aarch64.lists/elements-beside-numeric-arrays.jsfails on ubuntu 25.04 x64, and also in build 140651 ofmain. Two builds ofmainbehind AOT: update the engine; stores in defaults of parameters, paths that stores rule out #48714 pass both programs (140786, 140802). So the branch is rebased ontomain.CI of this PR, second push. Build 140933, on
mainat 9ea3c69. All 20 wrapper files pass again on the three lanes.module-identity-15module-identity-15took 189.3 s, 170.3 s and 227.0 s in builds 140086, 140670 and 140933. The files beside it decide that, as above.TypeError: Invalid URLingetFileUrl(scripts/buildkite.ts:367). It was writing the report ofvendor/elysia/test/response/stream.test.ts, which failed its first try and passed its second.Planted faults. Release canary, each run filtered to the named case.
mainprocess.exit(0)after theSAIDline ofreduced | unmocked below a barrelprocess.exit(0)in the first of the two test files ofreduced | loaded earlier through a barrelstraight | namespace dependency mock.module resetin the listBUN_AOT_REWRITE_MODULE_IDENTITY=1a run of two cases leaves the list byte for byte as it is.test/aot/module-identity.known.txtdoes not change.no test proof · iteration 5 · no src or test change; test-proof not applicable