Repository navigation
Conversation
WalkthroughThis PR updates fd ownership handling in ChangesWatcher fd lifecycle
Possibly related PRs
🚥 Pre-merge checks | ✅ 4✅ Passed checks (4 passed)
Comment |
|
Updated 8:32 AM PT - Jul 15th, 2026
❌ @robobun, your commit 8dc42b8 has 2 failures in
🧪 To try this PR locally: bunx bun-pr 33197That installs a local version of the PR into your bun-33197 --bun |
There was a problem hiding this comment.
Actionable comments posted: 1
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Inline comments:
In `@test/cli/hot/hot.test.ts`:
- Around line 856-857: The cleanup in hot.test.ts is letting the deliberate
runner.kill() abort the stdout pump and fail the test before the fd-count
assertion can be checked. Update the teardown around runner.kill() and the
Promise.all([pump, stderrText, runner.exited]) await so the stdout reader’s
abort from the kill is caught and swallowed during cleanup, using the relevant
pump/runner cleanup logic in this test helper instead of propagating the
stream-abort error.
🪄 Autofix (Beta)
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: Path: .coderabbit.yaml
Review profile: ASSERTIVE
Plan: Pro
Run ID: cecb20fa-368c-498c-a107-f0777c3c9868
📒 Files selected for processing (2)
src/watcher/Watcher.rstest/cli/hot/hot.test.ts
There was a problem hiding this comment.
I didn't find bugs in the diff itself, but test/bake/dev/hot.test.ts is failing across all 9 Linux CI configurations on this build — since the Linux branch of add_file now closes the previously-stored fd and bake's dev server also consumes the watcher via bundle_v2.rs, it's worth confirming those failures are pre-existing on main before merging.
Extended reasoning...
Overview
This PR fixes an fd leak in Watcher::add_file (src/watcher/Watcher.rs): when a file is already in the watchlist, the dedup branch now ensures exactly one descriptor stays open — on Linux the new fd replaces the stored one and the old one is closed; on kqueue platforms the new fd is closed since the kqueue registration lives on the stored one. The now-redundant per-call-site compensation in add_file_by_path_slow is removed. A Linux-only regression test in test/cli/hot/hot.test.ts counts /proc/<pid>/fd entries pointing at the entrypoint across 20 --hot reloads.
Security risks
None. This is internal fd bookkeeping in the file watcher; no auth, crypto, untrusted input parsing, or externally-reachable surface is touched.
Level of scrutiny
High. This changes the ownership contract of a public function (add_file) with callers in jsc_hooks.rs, RuntimeTranspilerStore.rs, AsyncModule.rs, hot_reloader.rs, and two sites in bundle_v2.rs. On Linux it now closes an fd that was previously left open in the watchlist; on macOS/FreeBSD it now closes an fd the caller just handed over. Per this repo's review history, fd/resource lifecycle in cross-thread native code is the most-blocked category — every caller and every reader of the stored fd (e.g. snapshot_fd_and_package_json, which copies the fd out under the mutex and then uses it after releasing) needs to be checked for use-after-close. The PR description itself scopes out the sibling directory-fd leak precisely because a naive close there risks use-after-close in bake's dev server; a human should confirm the file-fd path fixed here doesn't share that hazard.
Other factors
- The BuildKite run for be48afb shows
test/bake/dev/hot.test.tsfailing on all 9 Linux configurations and none elsewhere. Bake's dev server registers files viabundle_v2.rs → add_file, and this PR's only Linux-specific behavior change is closing the previously-stored fd — the correlation is strong enough that it should be ruled out (or confirmed as a pre-existing main flake) before merging.test/bake/dev/production.test.tsandtest-net-connect-memleak.jsfailures look unrelated. - The new test is well-constructed (awaits observable conditions, wires early-exit to reject, uses
Promise.allSettledin teardown per the resolved CodeRabbit thread), and the PR description demonstrates a clear root-cause understanding. - No prior claude[bot] reviews on this PR.
|
Confirmed: the The mechanism: on Linux, 16e2193 gates the close on |
There was a problem hiding this comment.
Actionable comments posted: 1
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Inline comments:
In `@src/watcher/Watcher.rs`:
- Around line 870-879: The watchlist ownership in Watcher::update is broken
because the slot is overwritten before the st_nlink check, so an unlinked
previous fd can be lost when close(previous) is skipped. Fix the fd swap logic
in the atomic watcher path so fds[index as usize] is only replaced when previous
is valid and will actually be closed; otherwise keep previous stored and close
the incoming fd instead. Re-audit the ownership flow around bun_sys::fstat,
bun_sys::close, and the previous/fd variables to ensure every acquired
descriptor remains owned until eviction or shutdown.
🪄 Autofix (Beta)
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: Path: .coderabbit.yaml
Review profile: ASSERTIVE
Plan: Pro
Run ID: 85fed77a-f62c-4ae5-a3f9-b023545d3a73
📒 Files selected for processing (2)
src/watcher/Watcher.rstest/cli/hot/hot.test.ts
There was a problem hiding this comment.
Additional findings (outside current diff — PR may have been updated during review):
-
🔴
src/watcher/Watcher.rs:873-880— Thest_nlink > 0guard reintroduces the entrypoint fd leak for atomic saves: when an editor does write-temp + rename-over (the default for VS Code/vim/emacs),fstat(previous).st_nlink == 0, soclose(previous)is skipped — butfds[index]was already overwritten two lines above, sopreviousis orphaned and leaks one fd per save. This is exactly the scenario the PR description cites as the reason the module loader re-opens the entrypoint, and the new test does not cover it (it onlywriteFileSyncsdep.tsin place, soentry.ts's inode never changes). Considerinotify_rm_watching the stale wd — or re-watch_pathing to migrate it to the new inode — before closingpreviousunconditionally, rather than trading the spuriousIN_DELETE_SELFfor a permanent leak.Extended reasoning...
What the guard does
Commit 16e2193 added this to the already-watched branch of
add_fileon Linux:fds[index as usize] = fd; // Releasing the last reference to an unlinked inode delivers // IN_DELETE_SELF on this item's still-registered inotify watch (a // spurious change event), so only close while the inode is linked. if previous.is_valid() && matches!(bun_sys::fstat(previous), Ok(st) if st.st_nlink > 0) { let _ = bun_sys::close(previous); }
When
st_nlink == 0thematches!is false andclose(previous)is skipped. Butfds[index]has already been overwritten with the new fd on the line above, sopreviousis no longer stored anywhere. The caller transferred ownership (add_fileis documented as owningfdon success;maybe_watch_fileinjsc_hooks.rssetsshould_close_input_file_fd = false), so nothing else will ever close it. It leaks permanently.When is
st_nlink == 0?Exactly on an atomic save of the entrypoint — write to a temp file, then
rename(tmp, entry.ts). The rename drops the old inode's link count to 0 while bun's stored fd keeps it alive. This is the default save behavior for VS Code, vim, emacs, and most editors, and it is the very scenario the PR description itself cites as the reason the module loader deliberately re-opens the entrypoint on every reload ("an atomic save can replace its inode, see the note aboveis_maininjsc_hooks.rs").Step-by-step trace
- Editor does
rename(tmp, entry.ts). Old inode I1 now hasnlink == 0; the path points at new inode I2. Bun's storedfd_A(on I1) keeps I1 alive. - The file watch on I1 (mask
EXCL_UNLINK | MOVE_SELF | DELETE_SELF | MOVED_TO | MODIFY, perINotifyWatcher.rs:134) receives no event: rename-over on the target inode does not emitMOVE_SELF, and the kernel defersIN_DELETE_SELFuntil the last reference to I1 is dropped — whichfd_Ais holding open. This deferral is precisely what the guard's own comment describes. - The parent-directory watch delivers
IN_MOVED_TO "entry.ts"→hot_reloadertriggers a reload. jsc_hooks.rsforcesfd = Noneforis_main, the loader opensentry.tsfresh asfd_Bon I2, and callsadd_file(fd_B, ...).index_of(hash)hits (the entry was never evicted — see step 2).previous = fd_A.fds[index] = fd_Boverwrites the slot.fstat(fd_A).st_nlink == 0→ close skipped.fd_Ais now referenced nowhere. Because it stays open, I1 is never freed, soIN_DELETE_SELFnever fires, soremove_at_index→flush_evictionsnever runs for this item. The leak is permanent.
Every subsequent atomic save of
entry.tsrepeats this: the previous fd always points at the just-unlinked prior inode, sost_nlinkis always 0 and the close is always skipped. One fd per save, unbounded → EMFILE, the exact failure mode the PR title claims to fix.Why the new test doesn't catch it
The added test only does
writeFileSync(depPath, ...)— an in-place write ofdep.ts.entry.ts's inode never changes, so whenadd_fileruns for the entrypoint,fstat(previous).st_nlinkstays ≥ 1 and the close fires. The two existing atomic-replacement tests inhot.test.ts("deleted and rewritten", "renamed() into place") bothrmSync()the target first, which deliversIN_DELETE_SELFimmediately and evicts the entry before the JS thread'sadd_fileruns, so they don't reach thest_nlink == 0branch either. And they only do 3 iterations without counting fds.Impact and fix
The guard was added to suppress a spurious
IN_DELETE_SELF→ extra reload after closing the old fd. But suppressing it by never closing the fd trades one benign extra reload per save for an unbounded fd leak in the common editor workflow — the wrong tradeoff for a leak-fix PR. This is not a regression vs.main(the atomic-save case leaked before too), but the guard was added in this PR's own final commit, and it leaves the primary real-world path unfixed while the PR title claims otherwise.Alternatives that avoid both the leak and the spurious event:
inotify_rm_watchthe stale wd (stored ineventlist_index) before closingprevious, then close unconditionally.- Re-
watch_paththe file path (which moves the wd to the new inode), updateeventlist_index, then closepreviousunconditionally. - Close
previousunconditionally and haveon_file_updateignoreIN_DELETE_SELFfor an item whose stored fd was just replaced this cycle.
An atomic-save variant of the new test (write
entry.ts.tmpthenrenameSyncoverentry.ts, without a precedingrmSync) would exercise this path and currently fail. - Editor does
|
Accurate, and it matches the first bullet under "Not fixed here" in the PR description: an entrypoint that is atomically renamed over on each save still leaks one descriptor per save on Linux. Two clarifications on the framing.
The one close this PR keeps is the one with a containment argument: the replaced descriptor is exactly what |
|
CI status for build 73264 (sha 8dc42b8, rebased onto current main), finished: 283 lanes passed, 3 failed. The 3 failed lanes are 2 distinct tests, both unrelated to this change:
Plus BuildKite's own Neither test spawns Bun with
Locally on this sha's ASAN debug build: the new test fails on stock Bun ( The diff is 10 Linux-only lines in |
There was a problem hiding this comment.
I didn't find any issues, but this touches fd ownership in the concurrent watcher path with subtle inotify-semantics reasoning (the st_nlink guard, and the containment argument that previous inherits flush_evictions' close authority), so it's worth a human confirming the ownership model holds.
Extended reasoning...
Overview
The PR fixes an fd leak in bun --hot: every reload re-opens the entrypoint and hands the fresh fd to Watcher::add_file, whose already-watched branch overwrote the stored fd without closing the old one. The fix is 10 lines in src/watcher/Watcher.rs — capture previous, overwrite the slot, then close previous iff it's valid, distinct from the incoming fd, and its inode is still linked (fstat().st_nlink > 0). A Linux-only regression test in test/cli/hot/hot.test.ts counts /proc/<pid>/fd entries resolving to the entrypoint across 20 reloads.
Security risks
None. This is internal fd bookkeeping; no user-controlled input reaches the changed code path beyond what already flowed through it.
Level of scrutiny
High. This is fd lifecycle management on a path shared between the JS thread (which calls add_file) and the watcher thread (which calls flush_evictions), inside the ATOMIC_FILE_WATCHER (Linux-only) branch. The correctness argument rests on two non-obvious claims: (1) closing previous is safe because it was fds[index], i.e. exactly the fd flush_evictions was already authorized to close for that slot — so this is a strict subset of existing close authority and cannot introduce a new double-close; and (2) the st_nlink > 0 guard is necessary and sufficient to avoid the spurious IN_DELETE_SELF that broke DEV:hot-9 in the first revision. Both arguments are well-made in the PR description and the resolved CodeRabbit thread, but they depend on invariants about who stores what in fds[index] on Linux that a maintainer familiar with the watcher/resolver/bundler fd-ownership boundary (cf. #33102) should confirm.
The previous != fd guard correctly prevents closing a descriptor the caller might still be using when the same fd is re-registered.
Other factors
- The first revision of this PR deterministically broke
test/bake/dev/hot.test.ts(DEV:hot-90/20), which is exactly the kind of subtle interaction that argues for human review even after the fix was narrowed. - All CodeRabbit threads are resolved; the second one (suggesting to also close the incoming
fdon the unlinked-inode arm) was correctly rejected with a clear ownership argument. - The "Not fixed here" section documents three remaining leaks (atomic-rename entrypoint on Linux, macOS/FreeBSD incoming-fd drop, and the resolver's
DirEntrystore_fd) with reasoning for why each is out of scope — a maintainer may want to weigh in on that scoping. - The new test is well-constructed: awaits actual reload signals, drains both pipes concurrently, uses
Promise.allSettledin teardown so the kill can't mask the fd-count assertion, and the PR description shows it fails on the unfixed build (after: 21vsafter: 1).
…file `Watcher::add_file` for an already-watched hash overwrote the stored fd with the caller's freshly-opened one without closing the previous one (and on non-Linux it dropped the new one instead). `bun --hot` re-opens the entrypoint on every rebuild, so each reload leaked one open descriptor of the entrypoint until the process ran out and everything started failing with EMFILE. Make `add_file` own the incoming fd: on the already-watched path exactly one descriptor stays open for the item and the other is closed. Remove the per-call-site compensation in `add_file_by_path_slow` this makes redundant.
…linked On Linux, releasing the last open reference to an unlinked inode is what makes the kernel deliver the deferred IN_DELETE_SELF for that inode's still-registered inotify watch. Closing the previous descriptor unconditionally in `add_file`'s already-watched branch therefore turned a rename-over of a watched file into a spurious extra change event, which `test/bake/dev/hot.test.ts` caught as a duplicate HMR reload. (Eviction is the only place that close was already safe: `flush_evictions` removes the item before the event is read back.) Gate the close on `fstat(previous).st_nlink > 0`. The `--hot` entrypoint leak this PR fixes re-opens the same, still-linked inode on every reload, so that descriptor is always closed. A renamed-over inode keeps deferring its descriptor to eviction, exactly as before this PR.
Revert the non-Linux arm of add_file's already-watched branch (and the add_file_by_path_slow cleanup that depended on it) back to main. Both closed the INCOMING descriptor, which the bundler can hand in borrowed from the resolver's entry cache (#33102), so that close has no ownership proof. The Linux branch now closes only the descriptor it just REPLACED in the watchlist slot. That is exactly the descriptor flush_evictions was already authorized to close for this item an instant earlier, with the st_nlink guard added on top, so it is a strict subset of the existing close authority and cannot introduce a new double-close.
18d4178 to
8dc42b8
Compare
There was a problem hiding this comment.
I didn't find any bugs, but this is fd-lifetime management in the multi-threaded watcher — the PR's own first revision showed how easily a plausible close breaks a consumer — so it's worth a human look at the ownership argument and the st_nlink guard.
Checked: the previous != fd guard prevents closing a re-passed borrowed fd; the close authority is a strict subset of what flush_evictions already exercises on the same slot; self.mutex is held across the read/replace/close so there's no race with the watcher thread's fd read. The test's waitForReload wires runner.exited to reject so an early child death can't hang, and the realpathSync on the entrypoint means the /proc readlink comparison is exact.
Extended reasoning...
Overview
Two files: a 10-line addition to src/watcher/Watcher.rs in add_file's already-watched branch (Linux-only, gated on feature_flags::ATOMIC_FILE_WATCHER), and an ~85-line Linux-only regression test in test/cli/hot/hot.test.ts. The Rust change captures the previous fds[index] before overwriting it and closes it iff it is valid, distinct from the incoming fd, and fstat(previous).st_nlink > 0.
Security risks
None. This is local fd bookkeeping; no untrusted input, auth, crypto, or network surface is touched.
Level of scrutiny
High. File-descriptor ownership across the watcher thread, the JS-thread transpiler, the resolver's entry cache, and the bake dev server is exactly the "every allocation has one named owner" territory CLAUDE.md flags as the most-blocked review category. The PR's own history validates that: the first revision (unconditional close) deterministically broke test/bake/dev/hot.test.ts DEV:hot-9 via a deferred IN_DELETE_SELF, and CodeRabbit's suggested alternative would have closed a potentially-borrowed incoming fd. The final shape is well-argued — closing only what flush_evictions was already authorized to close, further narrowed by the link-count guard — but the correctness of that containment argument depends on whether every path that populated fds[index] handed over ownership. That's a claim a maintainer familiar with #33102 and the resolver's store_fd semantics should confirm.
Other factors
The PR description is unusually thorough (mechanism, verification matrix, three explicit "not fixed here" follow-ups). All CodeRabbit threads are resolved with reasoned responses. The new test awaits the observable condition (RELOAD n on stdout) rather than sleeping, drains both pipes concurrently, wires early-exit to reject, and uses Promise.allSettled in teardown so the kill can't mask the fd-count assertion. CI on the final sha shows the two directly-affected suites (test/cli/hot/hot.test.ts, test/bake/dev/hot.test.ts) passing on every lane; the remaining failures are documented as pre-existing flakes on unrelated files. Given the subtlety already demonstrated in this PR's iteration and the explicit remaining leaks called out for follow-up, a human sign-off is the right bar.
|
Superseded by #36675, which combines this with the other |
What
Every incremental rebuild under
bun --hotleaked one open file descriptor of the entrypoint. A long editing session eventually exhausts the fd limit and every lateropen/socket/acceptin the process starts failing withEMFILE.Repro
On 1.4.0 the count of open descriptors pointing at
entry.tsgrows by exactly one per reload (201 after 200 saves).Cause
A
--hotreload re-transpiles the entrypoint. The module loader deliberately does not reuse the watcher's cached descriptor for the entrypoint (an atomic save can replace its inode, see the note aboveis_maininjsc_hooks.rs), so it opens the file again and hands the fresh descriptor toWatcher::add_file. The entrypoint is already in the watchlist, and the dedup branch there did this:The stored descriptor is overwritten and the previous one is never closed. Once overwritten it has no owner, so
flush_evictions/shutdowncan never close it either: one descriptor per reload, forever.Fix
On Linux,
add_file's already-watched branch closes the descriptor it just replaced, guarded byfstat(previous).st_nlink > 0. The wholesrc/watcher/Watcher.rsdiff is those 10 lines.Two things shaped that exact form:
Why the
st_nlinkguard. A first revision of this PR closedpreviousunconditionally and deterministically broketest/bake/dev/hot.test.ts(DEV:hot-9). The leaked descriptor was load bearing:IN_DELETE_SELFfor a per-file inotify watch is deferred until the watched inode is actually destroyed, and the watcher's descriptor was the last open reference keeping a renamed-over inode alive. Closing it made the kernel deliver that deferred event for the item's still-registered watch, and the dev server turned it into a duplicate HMR reload. (flush_evictionsonly gets away with the same close because itswap_removes the item before the event is read back.)st_nlink > 0is exactly the condition under which the close cannot generate an fsnotify event, and it is the one the--hotentrypoint case always satisfies: every reload re-opens the same, still-linked inode.Why only the replaced descriptor, and why only Linux. Callers can hand
add_filea descriptor they do not exclusively own: the bundler passes a descriptor that may be borrowed from the resolver's entry cache (the ownership bundler: don't close a borrowed resolver-cache fd after parsing #33102 fixed), so closing the incomingfdhas no proof it is not someone else's. Closing the replaced one does: at that instant it isfds[index], i.e. exactly the descriptorflush_evictionswas already authorized to close for this item, and thest_nlinkguard only narrows that further. An earlier revision also closed the incomingfdon macOS/FreeBSD (where the dedup branch just drops it); that is reverted and left as-is, for the same "no ownership proof" reason.Verification
New test in
test/cli/hot/hot.test.tsrunsbun --hot, triggers 20 reloads, and counts the/proc/<pid>/fdentries that resolve to the entrypoint.Without the fix:
With the fix it passes, along with the rest of
test/cli/hot/hot.test.ts(13 pass) and all oftest/bake/dev/hot.test.ts(9 pass) on a debug (ASAN) build.The
DEV:hot-9regression the first revision introduced was verified directly, all on the same ASAN debug configuration:DEV:hot-9st_nlink > 0cargo check -p bun_watcherpasses forx86_64-apple-darwin,x86_64-unknown-freebsd,x86_64-pc-windows-msvc, andx86_64-unknown-linux-gnu.Not fixed here
rename(tmp, target)) still leaks one descriptor per save on Linux, for theIN_DELETE_SELFreason above. A proper fix has to also re-register the item's inotify watch on the new inode so the old descriptor can be released without the spurious event; that interacts with the existing directory-watch recovery path and is deliberately out of scope.--hotentrypoint reload there too). Closing it needs a proof that the caller owned it exclusively, which the bundler call sites do not provide.bun --hotalso leaks a second descriptor per reload: theO_DIRECTORYhandle the resolver keeps in the cachedDirEntryunderstore_fd, orphaned becausebust_entries_cacheremoves only the hash-to-index mapping. Its raw value is also copied into queued bundlerParseTasks, the bake dev server's directory watch store, and sometimes the watcher's own watchlist, so a plain close at eviction risks a use-after-close on another thread. fs: reuse DirEntry in bustEntriesCache instead of orphaning it #29919 already reworksbust_entries_cacheto reuse theDirEntryin place instead of orphaning it, which is the right shape for that half.no test proof · iteration 4 · Platform-specific test(s) that do not run on this machine. Deferring to CI, which covers all platforms: test/cli/hot/hot.test.ts