Repository navigation
Conversation
|
Status: open, ready for review. How this was measured:
The description has the numbers. |
|
Warning Review limit reachedYou've used all free OSS reviews for now. Wait for the free limit to reset to keep reviewing this public repository. Next included review available in 53 seconds. View limit details
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
|
Updated 12:53 PM PT - Oct 21st, 2026
✅ @robobun, your commit c6881cbbc1cc5ee4250f89796d898de43a64cefa passed in 🧪 To try this PR locally: bunx bun-pr 45880That installs a local version of the PR into your bun-45880 --bun |
There was a problem hiding this comment.
Nothing blocking. The comments below are optional suggestions. There is no need to push a fix for them before merging.
Beyond the inline findings, I also checked the two shared-run matrices (formatted and contained via once()) for vacuous rows: a missing per-file result fails through expect(undefined).toEqual([]) rather than passing, and each row's expected line number or path is unique to its file. The temp-dir names used by the lazy stdout/stderr getters are captured before using dir is disposed, so late normalization still strips the path.
Extended reasoning...
The change rewrites assertions and run-sharing in a single integration test file for bun lint, with no production code and no security-sensitive surface. Two inline findings were posted; this note records what else was examined and ruled out.
2d22f28 to
bbc7858
Compare
…ends lint() normalizes stdout and stderr when a test first reads them. It normalized both for every run before. In a debug build that took 41 s of the test runner's thread, 29 s of it for two 6 MB outputs that no test reads, and no other test made progress in that time. Rows of three tables share their runs, each row with its own file: one run for each format lints the four files of the line-break tests, one run lints the 13 files of the directive tests, and one run fixes the 15 files of the emphasis tests. Six tests start their two runs together. The file starts 186 `bun lint` processes. It started 242. resultsOf() reads the results of a `-f json` run and expects the run to end the way a run with these results ends: standard error has the summary line for that number of files and nothing else, and the exit code is 1 only if a problem is an error. That only compares a run with itself, so each test also states the exit code it expects. Checks that a text is absent are now checks of the exact output, and the tests of linear time expect the number of problems that ESLint reports. On a debug build the file sets its default limit for a test to 120 s. `bun bd test` gives a test 5 s, and a run with a configuration file in JavaScript starts two debug processes. Release builds and CI keep their limits.
1bb64be to
c6881cb
Compare
There was a problem hiding this comment.
I reviewed the squashed commit and found no bugs; the points from my earlier passes are addressed here (the --fix count at test/cli/lint/js-parsers.test.ts:1517 is now exact, and reported/realms keep an explicit exitCode 1 beside resultsOf). Because this is a 675-line restructuring of a test file that I could not execute, a human look at the new exact expectations would still be worthwhile.
What was reviewed:
lint()'s lazy<dir>replacement: it captures realpath-then-given before the temp dir is disposed, in the same order harness'snormalizeBunSnapshot(…, dir)uses, so output is unchanged.pathOf'sbun-lint-js-parsers_\w+matches harness'smkdtempsuffix (six alphanumerics); the?.messages/?.suppressedMessagesrows fail on an undefined result rather than passing vacuously.- Shared
once()runs: each row asserts on its own file so a single-row regression still fails alone; a shared-run failure rejects every dependent row, which is the stated tradeoff. - The one thing I could not verify by reading: the per-rule counts in "runs in linear time" (e.g.
count - 2for emphasis markers) were derived atcount = 2000on debug/ASAN; the release lane runs them at 20000.
Extended reasoning...
The diff rewrites only test/cli/lint/js-parsers.test.ts (+437/-238): memoized output normalization, a resultsOf helper that checks stderr and exit code against the parsed JSON, three parameterized suites collapsed onto shared runs, exact stderr/exitCode assertions added across most tests, and the "runs in linear time" suite moved and tightened to exact per-rule counts. No production code and no security-sensitive surface is touched; no CODEOWNERS entry covers the path. The latest push addresses my prior inline comments, and the setDefaultTimeout call is now gated to debug builds with a stated reason. Deferring rather than approving because the change is large, replaces many loose assertions with exact values I could not execute, and the new count expectations are only stated as verified on the debug and ASAN lanes.
|
About the counts in
|
Problem
test/cli/lint/js-parsers.test.tstakes 68 s to 77 s on debian 13 x64-asan, 1 s to 6 s elsewhere. 56 of its 242bun lintruns start a JavaScript engine, and with LeakSanitizer each ends about 5 s later.lint()normalized every output: 41 s of the runner's thread in a debug build, 29 s for outputs no test reads.Fix
lint()normalizes an output when a test first reads it.--fixtests 1. Six tests start both runs together. 186 runs.resultsOf()checks that a-f jsonrun ends as its results say, and each test states its exit code. Still 196 tests.expect()calls: 490, now 629.bun bd test test/cli/lint/js-parsers.test.ts --timeout 270000, debug build, in turns: 264, 259, 308 s, then 202, 230, 218 s. ASAN lane, four runs: 63.6, 73.3, 63.5, 62.6 s.Background
src/options_types/context.rs:534). A waiting test holds one place.test/leaksan.suppsuppresses by name, so LeakSanitizer startsllvm-symbolizer(test(lint): make rule-throws.test.ts faster and strengthen its assertions #45755): 4 s of CPU for each of the 56 runs. That bounds the lane, and only asrc/change removes it.Behaviour change: none
Notes
The lane, before and after. The time of the file on
debian 13 x64-asan - test-bun:llvm-symbolizer, and 5 tests at a time use 5 of the 8 cores for it. This PR starts the same 56 runs. It removes 56 runs without an engine, which take about 0.5 s each there.Where the time goes on the lane. The log of the ASAN lane has a time for each batch of tests that end. In build 127781 the file takes 75.6 s:
processorsandlanguages: 22 tests, 34 of their 35 runs start an engineThe log does not name the tests, so this is what fits the log, not what the log says.
Which runs end with the symbolizer. I put a script at
ASAN_SYMBOLIZER_PATHthat writes down its parent and then startsllvm-symbolizer. With the variables thatscripts/runner.node.tssets on the lane, 56 runs start it, on main and here: the runs with a processor, with a rule of a plugin in JavaScript, and the runs that send a file to theeslintpackage. One such run on the debug build: 2.0 s to 2.2 s, and 6.3 s to 6.4 s with the variables. A run with aneslint.config.mjsand built-in rules only: 1.4 s to 1.5 s, and 2.0 s with the variables.A model of the lane. 5 tests at a time in the order of the file, 5.5 s for a run with an engine, 0.6 s for a run without. For the file with 180 tests it gave 74.6 s for main and 62.6 s for the first commit. The model lets runs with an engine overlap at no cost. On the lane they share 8 cores, so it says too much.
The test runner's thread.
normalizeBunSnapshotmakes about 15 passes over a text. On the debug build it needs 14.3 s for 6.4 MB.files whose parser options request types all go to one engineprints 6.4 MB twice and reads onlyraw. In one run of the file on main the normalization of all outputs took 41.5 s. With the default limit of 5 s the tests that waited for the thread then failed after 15 s to 17 s. The getters keep what they normalized, so the four rows that read one SARIF output normalize it once.The runs that are shared.
U+2028 and U+2029 aren't line breaks in JSON: 4 files (line feeds, byte order mark, carriage returns, both) and 10 formats were 40 runs. Now one run for each format lints the four files. Each file is ten lines longer than the one before it, so the second"a"is on line 54, 64, 74 or 84, and a place in the output belongs to one file. A test still fails alone: with the expected line of one file changed, its 10 rows fail and the other 30 pass.disables every rule it lists: 13 rows, each with its own file, in one run. A row reads the result of its file.--fix turns .. into emphasis: 15 rows thatbun check: 1.4x to 7.8x faster on large projects. Fixes forbun check,bun lintandbun fmt#45800 added, each with its own file, in one run with--fix -f json. A row expects the fixed text of its file and no problem left in it. With the expected text of one row changed, that row fails and the other 14 pass.once). A row can still run alone with-t.The runs that start together.
handles a processor that throws, ..,a file with a language and a processor ..,extglobs and brace ranges ..,files whose parser options request types ..,handle many keys out of order, with commentsand--fix removes what is unused, and nothing elseeach started two runs one after the other.What keeps its own run, and why.
options are validated,languageOptions: .. is rejectedandan invalid regex in the options of ..: an error in the configuration ends a run.the whole file goes to the eslint packageandeslint-plugin-json is treated like any other package ..: each row has its own configuration or flag.runs in linear time: a row that is slow fails alone, and names the input.Order.
runs in linear timeis now the last block of the Markdown tests, afterthe whole file goes to the eslint package. Six tests with an engine ended the file: 5 at a time, then one alone. Now the 14 tests without an engine run beside them. The model gives 2.4 s for this.Assertions.
resultsOf(run)parses the output of a-f jsonrun. It expects stderr to beLinted N filesor✓ No problems in N filesfor the N results and nothing else, and the exit code to be 1 only if a message is an error. A run that ended with another exit code fails with its stderr, not with a JSON syntax error.problems(run)is thesummaryof these results. 73 of the 186 runs go through them.not.toContain("error")andnot.toContain("not linted")(3 tests) are the exact stderr.not.toContain("the eslint package linted")(2 tests) isJavaScript: 0 engines loaded 0 modulesof--timing.--doctor(2 tests) compares the wholefilesobject.several such files among otherscompares the path and the number of problems of all 22 results.if a named block is the whole file, ..compares the paths of the results.with an .eslintrc, ..compares all of stdout and stderr. Onenot.toContainis left: the six formats ofif eslint isn't installeddo not name the file that was skipped.toContainon stderr.runs in linear time(14 rows) expected no fatal message and an exit code below 2. Each row now expects how often each rule reports. ESLint 10.13 with @eslint/markdown 8.0.3 reports the same numbers for 200 and for 2,000 repeats.handle many keys out of order, with commentsexpected more than half of the keys. It now expects the number, which is what ESLint 10.13 with @eslint/json 2.1.1 reports for 300 and for 3,000 keys.--stdintests went aroundlint()and did not read stderr. They uselint()now, compare stderr, and have an inline snapshot of theprettyoutput where they looked for one line.2 deprecated formatting rules did not run: indent, semi.--fixtests expect the line about what was fixed, for exampleFixed 25 problems in 4 files. The excerpt tests expect the line of the problem with the mark under it.Timeouts.
bun bd test <file>gives a test 5 s. A run with a configuration file in JavaScript starts two processes, and on a debug build that takes seconds. On this machine 90 of the 196 tests ran out of the 5 s (144 and 95 of 180 with main's file). On a debug build the file now sets its default limit to 120 s, which the tests of the first two groups already have. Release builds and the lanes of CI keep their limits: the condition isisDebug.Local numbers. Debug ASAN build on Linux x64, a VM with 12 vCPUs that gets about 3 cores. Before and after in turns. The time of one run differs by up to 50 s from the next.
bun bd test <file> --timeout 270000, 3 runs, 196 testsbun bd test <file>, 2 runs, 180 tests--timeouta test has 5 s. That is too short for a debug build on this machine, on main and here. Every test that fails ran out of it.llvm-symbolizerat once on main, 15 here. The least available memory of the 15 GB: 4.0 GB on main, 2.7 GB here.[auto-merge] gate passed · iteration 1 · 1 files touched
passes on PR (with fix)
diff hotspot
gate history · 1 passed · 1 rejected · iteration 1
evidence per changed file