Repository navigation
Conversation
The dev server harness killed a module-global set of spawned processes when any test ended. That made test.concurrent impossible: the first test to finish killed every other test's server and clients. The harness now tracks the processes per test (Dev.processes) and kills only those at teardown. A new `concurrent` option on devTest registers the test with test.concurrent. Dev server and client log lines carry the test's directory name, so concurrent output stays readable. bundle.test.ts opts every test in. Two expectMessage calls are now awaited. The two "is a Response" checks assert the response status. "commonjs forms" loops over the forms without debug logs and without the timeout multiplier. Release: 10.7s to 1.2s. Debug+ASAN: 47.8s to 13.4s.
|
Status: ready for review. Reproduced the slow run with Review follow-ups, both verified with probes and a full
|
|
No actionable comments were generated in the recent review. 🎉 ℹ️ Recent review info⚙️ Run configurationConfiguration used: Path: .coderabbit.yaml Review profile: ASSERTIVE Plan: Pro Run ID: 📒 Files selected for processing (2)
Included review availability: Your plan provides up to 5 included reviews per hour; 0 remain after this review. WalkthroughThe bake harness now supports opt-in concurrent tests, per-test subprocess cleanup, deadline-bounded lifecycle operations, and exit reaping. DevServer bundle tests use concurrent execution and stronger HMR and HTTP response assertions. ChangesBake concurrency and cleanup
🚥 Pre-merge checks | ✅ 4✅ Passed checks (4 passed)
Full details: Linked Issues checkExplanation The PR retains all 21 bundle regression cases and adds opt-in concurrency, per-test process cleanup, timeout handling, and file-level reaping. These changes support the relevant testing, resource cleanup, and execution objectives in [ Full details: Out of Scope Changes checkExplanation The changes remain within scope. Concurrency flags, awaited assertions, response status checks, removal of debug logs, and harness cleanup all support the stated test execution and reliability objectives. Full details: Description checkExplanation The description clearly explains the problem, implementation, verification results, cleanup behavior, and scope. It does not use the exact template headings, but it provides the required purpose and verification information in equivalent sections. Comment |
There was a problem hiding this comment.
LGTM — test-infrastructure only, opt-in concurrency with per-test process tracking.
Checked that the moved using _ disposer leaves no gap: await using devProcess covers startup failures before Dev is constructed, and clients only spawn after. The module-global danglingProcesses set is retained and still feeds the process.on("exit") handler, so nothing leaks on abnormal exit. The activeClient module-global is shared across concurrent tests but is only read in interactive mode. The "commonjs forms" table-driven rewrite preserves all seven original cases in order; the two added awaits and the Response→status assertions are strict tightenings.
Extended reasoning...
Overview
This PR touches only test infrastructure: test/bake/bake-harness.ts gains an opt-in concurrent?: boolean on DevServerTest that routes registration through test.concurrent, and the per-test cleanup disposer is changed from iterating the module-global danglingProcesses set to iterating a new per-Dev-instance processes set (so a finishing concurrent test kills only its own subprocesses). OutputLineStream prefixes now carry the test's temp-directory basename so interleaved output stays attributable. test/bake/dev/bundle.test.ts opts all 21 cases into concurrent: true, adds two missing awaits on c.expectMessage(...), strengthens two toBeInstanceOf(Response) liveness checks into concrete status/body assertions, and rewrites the "commonjs forms" test as a table-driven loop (dropping debug console.log scaffolding and timeoutMultiplier: 2). No production code is touched.
Security risks
None. This is test-harness and test-file code only; no user-facing surface, parsing, auth, crypto, or network-exposed paths are modified.
Level of scrutiny
Moderate for the harness change (shared by ~22 bake test files), low for the test-file edits. The harness change is deliberately opt-in: only bundle.test.ts sets concurrent: true, so the other 21 files keep their sequential behavior. I traced the disposer move: the old using _ sat before Bun.spawn and killed everything in danglingProcesses; the new one sits after new Dev(...) and kills only dev.processes. The window between spawn and Dev construction (which includes await stream.waitForLine) is already covered by await using devProcess, and no clients exist yet, so no process can leak from that gap. The module-global danglingProcesses is still populated (lines 909, 2126) and still drained in the process.on("exit") handler (line 2286), so the last-resort cleanup on abnormal exit is unchanged. The activeClient module-global is written per client but only read by maybeWaitInteractive, which runs single-test — harmless under concurrency.
Other factors
The "commonjs forms" refactor was diffed case-by-case: all seven [source, field] pairs match the original sequence exactly, and the initial { field: {} } assertion is preserved before the loop. The two now-awaited expectMessage calls fix a real hazard (unawaited rejection would misattribute under concurrency). The res.status === 500 assertion replaces a vacuous toBeInstanceOf(Response) and the PR notes the route stays failing regardless of the pending #40115 fix; the 200 + /_bun/client/ body check similarly tightens. The change follows CLAUDE.md's explicit guidance ("use test.concurrent for independent subprocess-spawning tests") and REVIEW.md's note that slow test files get scrutinized for test.concurrent. No CODEOWNERS entry covers these paths, and there are no outstanding reviewer objections in the timeline.
…'s processes die
The runner kills a timed-out test's processes only when the test is not
concurrent, and `bun test` never runs `process.on("exit")` listeners. A
concurrent test that hung would have left its dev server and clients
alive for the rest of the file, and past it on lanes without
BUN_FEATURE_FLAG_NO_ORPHANS.
`run()` now races the socket connect, the test body and the graceful
exit against a deadline 5s (or 10%) before the runner's timeout. The
rejection returns through the disposers, which kill this test's
processes. An `afterAll` reaps whatever is still in `danglingProcesses`
when the file ends.
bundle.test.ts keeps the "commonjs forms" shape from main, minus its
debug console.log lines, so the file stays close to #39488.
The harness module is evaluated once for every file in a `bun test` run, so an `afterAll` registered at module scope only reached the first file. `testImpl` now registers it the first time a file adds a test. `stackTraceFileName` treated the letter before the first colon of a posix path as a drive letter and kept the line number in the result. The caller path is now the file alone on every platform.
There was a problem hiding this comment.
LGTM — the per-file afterAll reaper in d3d9ec7 addresses the concern I raised; each calling file now registers its own hook via Bun.jest(caller) inside testImpl, gated by filesWithReaper.
Checked: per-test dev.processes cleanup is scoped correctly and the disposer is registered before the test body runs; Deadline pre-attaches a .catch so its rejection is never unhandled and Promise.race keeps observing the abandoned body; the stackTraceFileName colon === 1 change stops mis-parsing POSIX path:line:col while still handling Windows drive letters; the two newly-awaited expectMessage calls and the status-based assertions strengthen tests without weakening any existing coverage.
Extended reasoning...
Overview
Test-infrastructure-only change across two files. test/bake/bake-harness.ts gains per-test process tracking (Dev.processes), a concurrent opt-in that routes to test.concurrent, a disposable Deadline that races the test body/connect/gracefulExit against a timeout slightly shorter than the runner's, per-test log prefixing for interleaved output, a tightened Windows drive-letter check in stackTraceFileName, and a per-file afterAll reaper (previously my one open concern — now registered inside testImpl gated on the caller path). test/bake/dev/bundle.test.ts opts all 21 cases into concurrent: true, awaits two previously-floating expectMessage calls, tightens two toBeInstanceOf(Response) assertions to check status/body, and removes debug console.logs.
Security risks
None. No production/runtime code touched; changes are confined to the bake test harness and one test file. No network egress, credentials, or user-facing surface.
Level of scrutiny
Moderate for a harness refactor. I focused on process-lifecycle correctness under concurrency (that the per-test disposer only kills dev.processes, that it's armed before any awaited call that could hang, that Deadline doesn't leak an unhandled rejection), the reaper registration scope (fixed in d3d9ec7 — jest = Bun.jest(caller) so jest.afterAll attaches to the calling file, and filesWithReaper prevents duplicates), and that no existing assertions were weakened (the two Response checks are strictly stronger; the removed console.logs were debug noise, not assertions).
Other factors
The prior inline concern about the module-cached afterAll was directly addressed by the latest commit. No CODEOWNERS cover these paths. The PR description documents verification across release and debug+ASAN, all 155 bake dev tests passing, and a probe confirming hung concurrent cases leave no processes. Exit reason was dry_streak.
Problem
test/bake/dev/bundle.test.tstakes 15 to 19s per CI lane (build #106056), 10.7s locally on release, 47.8s on debug+ASAN. Its 21 cases run in sequence, each with its own dev server and usually a happy-dom client. That fixed cost is the time.run()intest/bake/bake-harness.tsSIGKILLs the module-globaldanglingProcessesset when any case ends. Undertest.concurrent, the first case to finish kills every other case's server and clients.Fix
Dev.processes) and kills only those at teardown. A new opt-inconcurrentoption ondevTestregisters the case withtest.concurrent.bun testskipsprocess.on("exit")listeners. Sorun()gives up 5s (or 10%) before the runner's timeout, and its disposers kill that test's processes. AnafterAll, registered once per test file, reaps what is left at file end.bundle.test.tsopts all 21 cases in. TwoexpectMessagecalls are now awaited, the twotoBeInstanceOf(Response)checks assert the status, and "commonjs forms" loses its debug logs. No case is merged, skipped, or changed in what it tests.bun bd test test/bake/dev/bundle.test.ts47.8s before, 13.3 to 13.6s after (ASAN caps concurrency at 5). Release 10.7s before, 1.2 to 1.4s after. All 155 bake dev tests pass. A hung concurrent case leaves no process behind.Background
devTestgives each case its own temp directory and a dev server on port 0.dev.client()is a node process that loads the page in happy-dom and acks each build over IPC.test.concurrentlets async tests in one file overlap, up to--max-concurrency(20, or 5 in an ASAN build). In CI this file runs alone in the runner's serial phase.Notes
Per-phase timing of one case (local, release, before)
setTimeout(50)inwaitForLine)dev.client()page load and overlay checkdev.write()rebuild and client ackBun.sleep(50)inbatchChanges)gracefulExitDebug+ASAN: 330ms to the port line, 460ms page load, 110 to 190ms per write, 800 to 1100ms
gracefulExit.The two fixed waits in the harness (50ms after a matched line, 50ms to let watcher events coalesce) are left alone. They exist for every bake dev file, the coalescing one has a stated reason (several watcher events per write on Windows), and under concurrency they overlap with other cases anyway.
Why the harness and not the file
Two earlier PRs (#40082, #40315) merged cases onto shared dev servers and were closed: #39488 carries the same restructure of this file, and #40315's own review found that the serial harness was the real cost. This PR keeps the 21 cases as they are, so it stays small against #39488, and makes the harness able to run them concurrently. The
bundle.test.tsedits are oneconcurrent: trueline per case plus the assertion fixes #39488 also has (the awaitedexpectMessagecalls matter under concurrency: an unawaited rejection lands on whichever test is current).concurrentis opt-in so the other 18 dev files keep their sequential behavior until each one is checked.The deadline
src/runtime/test_runner/Execution.rs(kill_dangling_processes_on_timeout) kills a timed-out test's spawned processes only when the concurrent group has one sequence.src/runtime/cli/test_command.rs(skip_exit_listeners) skips a test file'sprocess.on("exit")listeners, so the harness's exit hook never ran underbun test, on main too. Before this PR the kill-all at the end of every case reaped a hung case's processes when the next case finished. The deadline replaces that: a case that hangs fails with "The test did not finish before the harness deadline" 5s (or 10% for short timeouts) before the runner's timeout, and its disposers SIGKILL its server and clients.Promise.racekeeps observing the abandoned body, so its later rejection is not unhandled.Probe (not committed): a
concurrent: truecase withtimeoutMultiplier: 0.1that awaits a promise that never settles, next to two live siblings and a final check withprocess.kill(pid, 0). Release: the case fails at 2.7s of its 3s timeout, both siblings pass, server and client are dead. Debug+ASAN: fails at 24.4s of 27s, same result. TheafterAllreaper fires at file end (checked with a temporary log, 0 processes left).Concurrency limits and memory
bun testruns at most--max-concurrencytests at once: 20 by default, 5 when the build has ASAN (src/options_types/context.rs). Peak RSS per case: about 40 MB for the dev server and 110 MB for the client on release, 400 MB and 120 MB on debug+ASAN. So 20 release cases need about 3 GB, 5 ASAN cases about 2.6 GB. The file is inexcludeFilesoftest/parallel-allowlist.json, so the CI runner runs it alone in its serial phase.Release, pinned to 4 cores (
taskset), 10 runs: 2.2 to 3.9s, all pass. Pinned to 2 cores, 5 runs: 4.0 to 4.2s, all pass. Debug+ASAN pinned to 8 cores, 4 runs: 13.4 to 13.5s, all pass.Other harness users
bun bd test test/bake/dev/ test/bake/dev-and-prod.test.ts: 155 pass, 0 fail (22 files, 425s debug+ASAN), run after every harness change. The interactive mode (bun test/bake/dev/bundle.test.ts "<filter>") still starts:test.concurrentandafterAllthrow outside the test runner the same waytestdoes, and the harness catches them.The harness module is evaluated once per
bun testprocess, so anafterAllat module scope reached only the first file of a multi-file run.testImplregisters it the first time a file adds a test.stackTraceFileNameused to keep the line number in the caller path on posix (it took thesof.tsbefore the first colon for a drive letter), so the per-file key needed that fixed too. Test names and temp directories are unchanged.Assertions
expect()calls: 44 before, 45 after. The final fetch in "removing 'use client' from a component with a pending resolution failure" returns 500 today through a runtime error page (#40113, fix in #40115) and a "Build Failed" page once that lands. The route has an unresolved import either way, so the status is the stable assertion.