Repository navigation
Conversation
read_directory_with_iterator and entries_at_locked re-read a cached directory in place when its generation is older than the current bundle generation. When that re-read fails (open_dir or readdir returns an error) the old code routed through read_directory_error, which overwrote the BSSMap slot with an Err variant or re-pointed the index at NOT_FOUND. The slot previously held Entries(&'static mut DirEntry) and the reference pointed at a leaked Box<DirEntry>; overwriting the slot orphaned that Box along with its entry map. The cached .fd cannot be closed on this path either (copies live in Watcher.watchlist, ParseTask.contents_or_fd, and DirectoryWatchStore), so once the Err was cached the fd was unreachable as well. The cached Err also never retries across generations, so a directory that was recreated after being replaced stayed unresolvable for the rest of the process. Pass a refreshing_in_place flag through read_directory_error and, when set, hand back a threadlocal Err without touching the slot. The existing Entries stays reachable, the Box and fd are not orphaned, and the next lookup re-tries the open so a recreated directory resolves again.
|
Updated 4:01 AM PT - Aug 4th, 2026
✅ @robobun, your commit a777681f71cc99320273f5894453e77ab885335a passed in 🧪 To try this PR locally: bunx bun-pr 36874That installs a local version of the PR into your bun-36874 --bun |
WalkthroughThe resolver preserves existing directory-cache entries when in-place refreshes fail. A regression test covers repeated ChangesDirectory cache refresh handling
Possibly related issues
Possibly related PRs
🚥 Pre-merge checks | ✅ 4✅ Passed checks (4 passed)
Comment |
There was a problem hiding this comment.
Actionable comments posted: 1
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Inline comments:
In `@src/resolver/lib.rs`:
- Line 1278: Update the in-place refresh flow around read_directory_error and
its corresponding branches at the other reported call sites so readdir results
are staged in temporary state rather than clearing (*existing).data or
(*e_ptr).data beforehand; only replace the live DirEntry after readdir succeeds,
preserving the cached map on failure. Add coverage verifying a failed refresh
retains the existing map and a subsequent retry still works.
🪄 Autofix (Beta)
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: Path: .coderabbit.yaml
Review profile: ASSERTIVE
Plan: Pro
Run ID: f9279f57-7733-4ab3-b4ff-95b37a9356cc
📒 Files selected for processing (2)
src/resolver/lib.rstest/bundler/bun-build-api.test.ts
There was a problem hiding this comment.
The fix looks correct and well-reasoned, but this is process-lifetime resolver cache state with unsafe raw-pointer/leaked-Box management, so it's worth a maintainer glance.
What was checked:
- All four
read_directory_errorcall sites: the two inentries_at_lockedare always inside the stale-generation refresh arm (literaltrueis correct); the two inread_directory_with_iteratorcorrectly key onin_place.is_some(). - Callers that now receive a threadlocal
Errinstead of a BSSMap-backed one:dir_info.rsget_entries*immediately mapErr → None, andresolver.rs:5660readsoriginal_errthen bails — none stash the pointer past the call. Generation-0 callers (PackageManager, glob accessor) never hitin_place, so unchanged. - After a failed refresh the slot keeps its old
generation, so the next lookup at ≥ current generation retries — no permanent stale-empty state.
Extended reasoning...
Overview
Adds a refreshing_in_place: bool parameter to read_directory_error in src/resolver/lib.rs and threads it through four call sites. When set, the function skips the branch that overwrites the BSSMap slot with EntriesOption::Err (or re-points the index at NOT_FOUND) and instead returns a threadlocal Err, leaving the existing Entries(&'static mut DirEntry) slot — and the leaked Box<DirEntry> plus its cached fd behind it — untouched. A new subprocess-isolated regression test in test/bundler/bun-build-api.test.ts drives three in-process Bun.build() calls (dir → file → dir again) and asserts the third build resolves.
Security risks
None. This is internal resolver cache bookkeeping; no user-controlled input reaches new parsing or allocation logic.
Level of scrutiny
High. This is the resolver's process-lifetime directory cache — shared by the bundler, runtime module loader, dev server, and package manager — and the touched code is dense with unsafe raw-pointer dereferences, heap::into_raw-leaked boxes, and fd-sharing invariants (the doc comment names Watcher.watchlist, ParseTask.contents_or_fd, DirectoryWatchStore). REVIEW.md flags memory safety as the most-blocked category, and "fix at the layer that owns the invariant" applies here: the fix intentionally leaves a stale-generation DirEntry (with cleared .data) in the slot rather than reclaiming it, which is the right call given the shared fd, but a maintainer who owns this cache should confirm that's the intended long-term shape (vs. e.g. giving Err a generation so it can be retried without keeping the old DirEntry alive).
Other factors
I traced every caller of entries_at/entries_at_locked/read_directory for whether the newly-threadlocal Err return could outlive the call — none stash it; they all match-and-discard synchronously. The in_place.is_some() gating in read_directory_with_iterator is correct: in_place is only Some when a cached Entries at an older generation was found, exactly the refresh case. Generation-0 callers can never trip in_place (u16 < 0 is always false). The test follows harness conventions (tempDir, bunEnv, concurrent pipe drain, exit-code asserted last). The one open finding is a nit about documenting the Windows skip. Deferring rather than approving because the change, while small, alters ownership/lifetime semantics of a shared unsafe cache slot.
The slot now survives a failed refresh, so clearing .data on the error path would drop the prev_map the next retry uses to reuse Entry cells (and orphan them in the process-lifetime EntryStore). Leave the map in place; readdir only reads from prev_map, so a partial iteration leaves it consistent. Also drop the Windows skip: Bun.build runs with store_fd=false, so the resolver does not hold an open handle to the directory and rmSync succeeds. Verified against the Windows canary build.
bb7d54a to
3d7eab4
Compare
There was a problem hiding this comment.
Actionable comments posted: 3
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Inline comments:
In `@src/resolver/lib.rs`:
- Line 1604: Update the stale-generation documentation near the directory lookup
logic to state that the existing DirEntry and data map are preserved when
refresh fails, and replaced only after a successful refresh. Align the comment
with the behavior of the error branches around read_directory_error.
- Line 1269: Update the readdir error branch in the directory-reading flow to
close the newly opened handle when store_fd is enabled and need_to_close_files()
is false, before calling read_directory_error. Preserve the existing error
propagation and avoid closing handles that are already managed by the normal
cleanup path.
In `@test/bundler/bun-build-api.test.ts`:
- Line 504: Strengthen the child-driver assertions for the second build in the
“rebuilding after a cached directory is replaced with a file” test: after
checking second.success is false, verify the reported logs identify sub/mod.js
and contain the expected directory-resolution error. Keep the existing
success/failure flow unchanged while ensuring unrelated failures cannot pass the
regression test.
🪄 Autofix (Beta)
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: Path: .coderabbit.yaml
Review profile: ASSERTIVE
Plan: Pro
Run ID: e12b19b8-4121-4ae5-b83f-2a1b78382d24
📒 Files selected for processing (2)
src/resolver/lib.rstest/bundler/bun-build-api.test.ts
There was a problem hiding this comment.
Caution
Some comments are outside the diff and can’t be posted inline due to platform limitations.
⚠️ Outside diff range comments (2)
src/resolver/lib.rs (2)
1265-1265:⚠️ Potential issue | 🟠 MajorClose the newly opened handle before returning the refresh error.
When
store_fdis true andneed_to_close_files()is false,should_close_handleis false. Thereaddirerror at Line [1265] returns beforeentries.fd = handleat Line [1281]. No owner closes the newly opened descriptor. Repeated failed refreshes can exhaust process descriptors. Keep the guard armed until successful publication transfers ownership toentries.fd.As per coding guidelines, every resource acquisition must have a release on all error paths.
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@src/resolver/lib.rs` at line 1265, Ensure the readdir error path in the surrounding refresh logic closes the newly opened handle when ownership has not yet been transferred to entries.fd. Keep should_close_handle armed through the read_directory_error return and only disarm it after successful publication of the handle, preventing descriptor leaks on failed refreshes.Source: Coding guidelines
1115-1122: 🩺 Stability & Availability | 🟡 Minor | ⚡ Quick winDocument the lifetime of in-place refresh errors.
An in-place refresh failure returns the thread-local
TEMP_ENTRIES_OPTIONeven when entry caching is enabled. Update the public contract and ensure callers consume this result before another lookup, or return a correctly scoped value.🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@src/resolver/lib.rs` around lines 1115 - 1122, Update the contract documented for read_directory_error to state that in-place refresh failures may return the thread-local TEMP_ENTRIES_OPTION even when entry caching is enabled. Audit its callers and ensure they consume the returned result before performing another lookup, or change the return handling to use a correctly scoped value.
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Outside diff comments:
In `@src/resolver/lib.rs`:
- Line 1265: Ensure the readdir error path in the surrounding refresh logic
closes the newly opened handle when ownership has not yet been transferred to
entries.fd. Keep should_close_handle armed through the read_directory_error
return and only disarm it after successful publication of the handle, preventing
descriptor leaks on failed refreshes.
- Around line 1115-1122: Update the contract documented for read_directory_error
to state that in-place refresh failures may return the thread-local
TEMP_ENTRIES_OPTION even when entry caching is enabled. Audit its callers and
ensure they consume the returned result before performing another lookup, or
change the return handling to use a correctly scoped value.
ℹ️ Review info
⚙️ Run configuration
Configuration used: Path: .coderabbit.yaml
Review profile: ASSERTIVE
Plan: Pro
Run ID: e4c480d0-1443-4c41-b3a5-e931319a2ebe
📒 Files selected for processing (1)
src/resolver/lib.rs
There was a problem hiding this comment.
My earlier inline note about the doc-comment length is stale — it was already trimmed to one line in 3d7eab4, so that concern is resolved. I didn't find any bugs in the current revision, but this touches the resolver's process-static DirEntry cache (leaked Boxes, raw-pointer slot writes, fd aliasing across watchers), and the bool-flag-vs-generation-on-Err design choice is one a maintainer should sign off on.
What was reviewed:
read_directory_error(..., refreshing_in_place)— verified the new flag only skips the cache write; the fresh-slot path is byte-identical.- Confirmed
readdirbuilds into a localDirEntryand only readsprev_map, so removing the three error-path.data.clear()calls cannot leave the live map partially mutated. - All four call sites pass the flag consistently (
in_place.is_some()inread_directory_with_iterator, literaltrueinentries_at_locked). - Test drains stdout/stderr concurrently, asserts the second build fails specifically on
./sub/mod.js, and asserts the third recovers.
Extended reasoning...
Overview
The PR changes ~30 net lines in src/resolver/lib.rs and adds one regression test in test/bundler/bun-build-api.test.ts. It adds a refreshing_in_place: bool parameter to read_directory_error; when set, the function returns a threadlocal Err instead of overwriting the BSSMap slot, so the existing EntriesOption::Entries(&'static mut DirEntry) (a leaked Box) stays reachable and its stale generation triggers a retry on the next lookup. Three error-path .data.clear() calls are removed so a failed refresh leaves the cached entry map intact.
Security risks
None identified. This is internal resolver caching state; no untrusted input parsing, auth, or network surface is touched.
Level of scrutiny
High. The diff is small but sits inside a block of unsafe raw-pointer manipulation of a process-static singleton (entries_option_map()). Correctness depends on non-local invariants: that readdir never mutates prev_map (verified — it builds into a fresh local DirEntry and only reads from prev_map via add_entry_with_store), that the leaked Box<DirEntry> behind the slot must never be orphaned, and that the cached .fd is aliased elsewhere and cannot be closed here. The PR description and prior review threads walk through all of these, but they are exactly the kind of invariants a maintainer familiar with the resolver's ownership model should confirm.
Other factors
- All prior review threads (CodeRabbit's data-map-clearing concern, comment-cop's doc-length flags, my Windows-skip note, the second-build assertion) are addressed and resolved. My own inline comment on line 1123 was posted against a stale revision — the doc comment had already been trimmed to one line before I posted, so that thread should be considered resolved.
- CodeRabbit raised a pre-existing fd-leak on the
store_fd && !need_to_close_files()path when iteration fails after a successful open; the author correctly scoped that out as independent of this change. - There is a legitimate structural alternative (give
EntriesOption::Erra generation so it participates in the same stale-retry logic, eliminating the boolean side-channel). The author chose the minimal fix. That's reasonable, but it's the kind of design call REVIEW.md asks a maintainer to weigh ("Use the simplest mechanism the invariants allow" vs. "when a heuristic keeps sprouting counterexamples, redesign structurally"). - The test follows harness conventions (
tempDir,bunEnv, concurrent pipe drain, combined stdout/stderr assertion before exit code) and now runs on Windows.
BundleThread bumps its generation only after the work queue drains, which can be after the previous Bun.build() promise has already resolved. When the race hits, the second build reuses the stale sub/ listing and fails reading the file (ENOTDIR) instead of refreshing the cache. Re-build until a resolve-level error is observed so the test always exercises the refresh path.
There was a problem hiding this comment.
No bugs found, but this touches resolver directory-cache slot ownership (leaked Box<DirEntry> behind &'static mut, threadlocal-vs-BSSMap error stashing) — worth a human look given how central this path is.
What was reviewed:
- Verified
readdironly reads fromprev_map(get_hashedat fs.rs:467), so a mid-iteration failure leaves the in-place map and its EntryStore-backed entries valid after the.data.clear()removals. - Confirmed all four
read_directory_errorcall sites passrefreshing_in_placecorrectly:in_place.is_some()inread_directory_with_iterator, hard-codedtrueinside thegeneration < generationarm ofentries_at_locked. - Checked that skipping the cache write also skips
mark_not_found, so an ENOENT during refresh no longer re-points the index at NOT_FOUND either. - Test driver's bounded poll loop (100×5ms) waits on the observable "Could not resolve" signal rather than a raw sleep.
Extended reasoning...
Overview
The PR changes read_directory_error in src/resolver/lib.rs to take a refreshing_in_place flag. When set, it returns a threadlocal EntriesOption::Err instead of writing into the BSSMap slot (or re-pointing the hash index at NOT_FOUND). Four call sites are updated: two in read_directory_with_iterator (open failure and readdir failure, gated on in_place.is_some()) and two in entries_at_locked (both hard-coded true since they only run inside the stale-generation arm). Three error-path .data.clear() calls are removed so the existing entry map survives a failed refresh. A subprocess-driven regression test in test/bundler/bun-build-api.test.ts exercises the dir→file→dir sequence across three in-process Bun.build() calls.
Security risks
None. This is internal resolver cache-lifetime management with no user-controlled input reaching the changed branches beyond filesystem state. No auth, crypto, or permission surface.
Level of scrutiny
High. The resolver's directory cache sits under every module resolution, and the code being changed manipulates raw *mut DirEntry pointers, &'static mut reborrows into a process-static BSSMap, and a leaked-Box ownership model under entries_mutex. The fix itself is small and I traced it as correct — add_entry_with_store only reads prev_map via get_hashed, so dropping the local DirEntry on readdir error does not invalidate anything the preserved map points at (entries are EntryStore-owned). But the surrounding invariants (fd aliasing across Watcher/ParseTask/DirectoryWatchStore, the deferred structural alternative of giving EntriesOption::Err a generation, the pre-existing fd-leak the author scoped out) are the kind a maintainer who owns this module should sign off on.
Other factors
All prior review threads (CodeRabbit's data.clear concern, comment-cop's doc-length flags, my own skipIf/doc-comment nits) are resolved in the current head. The entries_at doc comment at lib.rs:1545-1548 still says the stale branch "drops the existing DirEntry"; that's accurate for the success path but no longer describes the error path — minor. CI build #88731 was still running at review time. The test's bounded poll for "Could not resolve" is the right shape per REVIEW.md but couples to a specific error-message prefix; if that wording ever changes the loop falls through to the i === 100 throw with a diagnosable message, so it fails loud rather than silently.
|
CI build #88731 is green for this change: The six tests that tripped retries are all unrelated flakes (install migration, |
|
Closing this PR because it has been inactive for more than 90 days. |
Repro
Cause
read_directory_with_iteratorandentries_at_lockedre-read a cached directory in place when its stored generation is older than the current bundle generation (BundleThreadbumps the generation betweenBun.build()calls). When that re-read fails, both routed throughread_directory_error, which either writesEntriesOption::Errinto the BSSMap slot or re-points the hash index atNOT_FOUND.The slot previously held
Entries(&'static mut DirEntry)where the reference points at aBox<DirEntry>leaked viaheap::into_raw. Overwriting the slot drops the reference and orphans the Box together with itsdatamap. The directory's cached.fdcannot safely be closed here either: copies of the descriptor live inWatcher.watchlist,ParseTask.contents_or_fd, and the bakeDirectoryWatchStore, so closing it would turn into a use-after-close once the fd value is reused.Separately, the cached
Errcarries no generation, so theEntries(e) if e.generation < generationarm never matches it and the directory stays unresolvable for the rest of the process even after it reappears.Fix
read_directory_errornow takes arefreshing_in_placeflag. When set, it returns a threadlocalErrand leaves the slot alone. The existingEntriesvalue (and the Box and fd behind it) stay reachable. The next lookup at the same or a later generation seese.generation < generation, retries the open, and either succeeds (directory is back) or returns another transient error.The fresh-slot path (
refreshing_in_place == false) is unchanged: a directory that has never been listed still caches its error.Verification
New test
rebuilding after a cached directory is replaced with a fileintest/bundler/bun-build-api.test.tsdrives three in-processBun.build()calls as above and asserts the third succeeds. It fails on main withCould not resolve: "./sub/mod.js"and passes with this change.no test proof · iteration 1 · Platform-specific test(s) that do not run on this machine. Deferring to CI, which covers all platforms: test/bundler/bun-build-api.test.ts