Repository navigation
dev server: continue the deleted-file walk through importers that are gone - #47016
Conversation
…not resolve The dev server learns that a file is gone when a build fails to open it. It then builds the importers of the file again, so that each one reports its failed import. An importer that is gone too, for example another file of a removed package, does not resolve as an entry point. The walk dropped it, so a page above it kept its old modules, and resolve_entry_point logged the miss as an error of the build. - on_file_deleted is now a walk. It handles an importer that the resolver does not find like the deleted file: it disconnects its imports, marks it stale and builds its importers. The bundle gets the file as a missing source, so missing_sources::report fails an import that is bound to it. - The importers of a client component are in the server graph. The walk continues there. - The resolver gets one more look at a directory before a file in it is handled as gone, one time per directory per walk. - DirectoryWatchStore does not leave a directory that is gone cached as missing. A package specifier never asks the resolver again, so a package that came back stayed unresolved.
|
Updated 10:42 AM PT - Oct 27th, 2026
✅ @robobun, your commit be8a64c2b6eed68ceecc03688355df8660ac7e6d passed in 🧪 To try this PR locally: bunx bun-pr 47016That installs a local version of the PR into your bun-47016 --bun |
|
Status
|
There was a problem hiding this comment.
Nothing blocking. The comments below are optional suggestions. There is no need to push a fix for them before merging.
Beyond the inline findings, I also checked that moving the bust_dir_cache call into DevServer::on_file_deleted is behavior-preserving (in_graph is exactly the old "index found" condition), that every caller of on_file_deleted / enqueue_file_from_dev_server_incremental_graph_invalidation and every match on MissingSource was updated, and that the one-more-look retry after a package reinstall and a plugin onResolve returning an absolute path with no file on disk do not misclassify an existing importer as deleted.
Extended reasoning...
The change reworks the Bake dev server's deleted-file walk across bundle_v2.rs, missing_sources.rs, transpiler.rs, directory_watch_store.rs, failures.rs and incremental_graph.rs, plus 8 new tests; it touches no auth, crypto or injection surface. Four confirmed findings are posted inline, so a human review is already signalled; this note only records the additional refactor-equivalence, caller-audit and ruled-out checks.
| InvalidationOutcome::Unresolved => { | ||
| bv2.note_missing_file(key); | ||
| walk.unresolved.put(key, ())?; | ||
| // The other target has the same file. | ||
| break; |
There was a problem hiding this comment.
🟡 (optional) Developers who replace a package under a running dev server with the isolated linker (link retargeted to a new store directory) now get 500 on every page above it until they restart; on the base those pages keep answering 200. The walk at incremental_graph.rs:1708-1712 marks an importer whose realpath key no longer resolves as gone and rebuilds every page above it, and the resolver's cached link target still points at the removed store, so the rebuild fails for good. Fix: when an importer under a store directory is handled as gone, also drop the cached listing of the directory that holds the link (or re-resolve the page's specifier rather than the realpath key) so a retargeted link is picked up at the next build.
Why this was flagged
Trigger: isolated-linker project, dev server running; bun add lib@ 2 removes node_modules/.store/lib@ 1 and retargets node_modules/lib to .store/lib@ 2. rebuild_importer_of_deleted_file calls enqueue_file_from_dev_server_incremental_graph_invalidation with the importer's realpath key, which returns Unresolved (bundle_v2.rs:3091); the retry busts only dirname(key) inside the old store (incremental_graph.rs:1697-1699), then incremental_graph.rs:1709-1712 notes the file missing and the loop at failures.rs:414-419 rebuilds every page above. Those pages resolve lib/outer.js through the resolver's cached entry for node_modules/lib, whose link target is still the removed store, so missing_sources::report fails them on every build until restart. Base: Err(_) => return Ok(()) at the old bundle_v2.rs:3080 stopped the walk, so pages two or more imports above kept serving the previous bundle. The DirectoryWatchStore::insert bust (directory_watch_store.rs:312-315) only drops the gone store directory, not the directory holding the link, so it does not recover this case.
Verification: Nothing busts node_modules/ or node_modules/<pkg>, whose cached DirInfo.abs_real_path is the old store target (resolver.rs:6382, 6413), so each rebuilt page resolves its import back to the removed realpath and the route answers 500 on every build until restart. On the base the same importer hit Err(_) => return Ok(()) (old bundle_v2.rs:3080) and pages above kept serving the old lib@ 1 bundle (200).
There was a problem hiding this comment.
This is the second item under Downsides, and it stays in this PR.
Measured on release builds with the link retargeted to a new store directory: #46983 answers 500 for the page whose build found the removal and 200 with the old modules for the page above. This PR answers 500 for both, until restart.
- The walk cannot find the directory that holds the link. The graph key of the importer is the real path, and the resolver keeps the link target in the listing of the parent of the link.
- A prototype of this change dropped every cached directory whose real path is the removed one. It did not stay: a workspace link that was restored then answered
Could not resolveuntil restart, and it scanned the whole directory cache for each deleted file. - The resolver side has open changes: hot reloader: follow a retargeted symlink on the import path #41534 (a retargeted symlink on the import path) and resolver: bust the real-path alias of a symlinked directory with the dir cache #40587 (the real-path alias of a symlinked directory).
A maintainer decides if 500 until restart is acceptable before those land. The alternative is what the base does: the pages above a removed store directory keep the old modules.
There was a problem hiding this comment.
| let in_graph = match graph { | ||
| bake::Graph::Server | bake::Graph::Ssr => self | ||
| .server_graph | ||
| .on_file_deleted(abs_path, bv2, &mut walk)?, |
There was a problem hiding this comment.
🟣 pre-existing, not blocking: Users who delete a file shared by server and client code keep getting the old client bundle: the client importers are never rebuilt when the deletion is reported through the server graph. failures.rs:391-394 walks only the server graph for Graph::Server/Ssr, while the Client arm at failures.rs:395-406 adds a hop to the server graph. The new hop is one-directional, so a file present in both graphs whose client parse task is cached (no ENOENT on the client side this build) leaves the client graph with dangling edges and stale modules. Fix: when the deleted file is in both bundled_files maps, run on_file_deleted on both graphs regardless of which graph reported it.
A small fix can ride a push you are already making; otherwise a short reply is enough.
Why this was flagged
Trigger: a util file imported by a server page and by client code is deleted. The server parse task fails first, handle_parse_task_failure (src/runtime/bake/dev_server/failures.rs:339) calls on_file_deleted with Graph::Server; failures.rs:391-394 runs only server_graph.on_file_deleted. If the client side of that file was not a parse task in this build (the client page files are only appended when stale, bundle.rs:53-59), no client walk ever happens; client importers keep their edges to the gone file and the browser receives the old module. The PR adds the reverse hop (failures.rs:398-406) for is_hmr_root only, so the walk added by this change is incomplete for the sibling direction; REVIEW.md's "fix the whole class" applies. The base had the same single-graph walk, but the PR's stated purpose is that pages above a removed file stop serving old modules, and this population still does. Remedy: check membership in both graphs and walk each one that has the file.
Verification: A file present in both the server and client graphs is reported missing by a Server/Ssr-target parse task. src/runtime/bake/dev_server/failures.rs:391-394 runs only self.server_graph.on_file_deleted(...) for Graph::Server | Ssr. So client-graph importers of the deleted file keep their edges. Severity is pre-existing because the base already fails identically by the same route.
There was a problem hiding this comment.
Done in 6892cd0. The walk now handles the file in the other graph too, unless this build read the file for that graph.
- The exception keeps a case that dev server: fail the importers of a file that was not there to read #46983 tests: a plugin gives one graph the source of a file that the other graph does not find (
a barrel that only the client graph of a "use client" module does not find fails that graph). With no exception, that test reportsCompA.tstwice. - This replaces the hop for client components.
- New test:
a removed package that the server graph finds gone fails a client component that imports it. On dev server: fail the importers of a file that was not there to read #46983 and on the first commit of this PR the route answers 200 with its old modules.
There was a problem hiding this comment.
42ca0b1 takes the change of 6892cd0 back, so this thread is open again.
Why the rule of 6892cd0 is gone:
- It cut the import edges of the file in the other graph and failed no import there. With two files in one save, a route answered 200 with a client bundle that did not have a module of its own code.
- It changed two flows of watched files with no test: a server route whose css import or shared import is deleted answered 500 where dev server: fail the importers of a file that was not there to read #46983 answers 200.
What 42ca0b1 does:
- The file whose read failed is handled in the graph of that read, as on main.
- One step crosses the graphs: a client component that the client graph does not find is also handled in the server graph, because its importers are there. The test is
a removed package with "use client" modules fails the routes that import the modules, with and without a separate SSR graph. - An importer whose file is gone too is handled in both graphs.
The direction of this finding is not fixed here: the server graph finds a file gone, and the client graph has importers of it. It answers as on main and on #46983 (200 with old modules). The body lists it under "Not fixed here".
…ng ancestor of its directory - A file that one graph does not find is gone for the other graph too, unless this build read it there: a plugin can give one graph the source. The walk handles the file in both graphs. This replaces the hop for client components, and it builds the client importers again when the server graph finds the file gone. - DirectoryWatchStore drops every missing ancestor of a directory that gets no watch. A lookup of node_modules/lib/sub caches node_modules/lib as missing, so a page that imports a file of a sub directory stayed unresolved after the package was back.
|
Review follow-up in 6892cd0. Each thread has the details.
|
There was a problem hiding this comment.
I re-reviewed the second push (6892cd0) and found no new bugs; it addresses the one-directional graph hop and the ancestor busting from my earlier comments. Since the change still rewires the deleted-file walk across both incremental graphs and the resolver's directory cache, a human look remains worthwhile.
What was reviewed:
DevServer::on_file_deleted: thehas_source_ofgate and thewalk.unresolvedloop (keys are ownedBox<[u8]>, cloned per step while the map grows) are sound.DirectoryWatchStore::insertancestor loop: bounded by the root check; a parent whose read errors (not just missing) is also dropped, which only costs a re-listing.MissingSource::Fileinmissing_sources::report: the single match on the enum is updated andBOTH_SIDESmatches theside()bits.- The eight new tests: no sleeps, exact error strings,
await usingcleanup.
Extended reasoning...
The change touches the bundler's dev-server entry (bundle_v2.rs, transpiler.rs, missing_sources.rs) and the bake dev server's incremental graph, failure handling and directory watch store, replacing the per-graph deleted-file handling with a walk that treats unresolvable importers as deleted and busts resolver directory caches. It touches no security-sensitive surface. Deferred rather than approved because the walk mutates incremental-graph edges and resolver cache state across both graphs, and two of my earlier inline threads (the resolver error-kind mapping at bundle_v2.rs:3098 and the retargeted-link case at incremental_graph.rs:1712) were not changed by this push.
Still open from earlier reviews (1):
- Unresolved: 1 minor or pre-existing.
The walk over the importers of a deleted file asked the resolver, and the resolver can answer from a directory listing older than the removal. The walk now checks the file of each importer. An importer that is gone is handled like the deleted file, for both graphs. `enqueue_file_from_dev_server_incremental_graph_invalidation` and `resolve_entry_point` are as before. The file whose read failed is handled in the graph of that read. A client component that the client graph does not find is also handled in the server graph, where its importers are. The server graph keeps the flag for that: the flag of the client graph is lost when a build with no server importer has the file again. The rule of the last commit for the other graph is gone. `missing_sources::report` ends with a drop of each directory of a missing file that the resolver has as not found. `DirectoryWatchStore::insert` is as before.
| bake::Graph::Server | bake::Graph::Ssr => self | ||
| .server_graph | ||
| .on_file_deleted(abs_path, bv2, &mut gone)?, |
There was a problem hiding this comment.
🟣 pre-existing, not blocking: Pre-existing, and the earlier review's fix (the failures.rs:394 entry, applied in 6892cd0) was removed again in 42ca0b1: when the server graph finds a shared file gone, client importers keep serving the old module. failures.rs:339-341 walks only server_graph for Graph::Server | Ssr, so the client graph's copy keeps its edges and nothing rebuilds its importers. The PR text still claims "a file that is gone for one graph is gone for the other", and the test for that direction was deleted with the code. Fix: for every deleted file walk each graph that has it, in both directions, skipping a graph only when this build re-read the file there.
A small fix can ride a push you are already making; otherwise a short reply is enough.
Why this was flagged
Setup from the removed test: Local.ts ("use client") imports '#lib' whose browser condition maps to 'lib'; routes/local.ts imports Local.ts; routes/barrel.ts imports 'lib' on the server. After /local is built the package is removed and /barrel is fetched. The server parse of node_modules/lib/index.js fails with ENOENT and reaches handle_parse_task_failure with Graph::Server; failures.rs:339-341 calls only self.server_graph.on_file_deleted, and index.js is not in the server graph, so the walk returns at incremental_graph.rs:1622-1623. The client graph still holds the edge Local.ts -> index.js, Local.ts is not stale, and the next /local answers 200 with the old bundle instead of the "Could not resolve: "#lib"" error. The base branch behaves the same, so this is pre-existing; commit 6892cd0 fixed it with has_source_of and the test "a removed package that the server graph finds gone fails a client component that imports it", and 42ca0b1 deleted both.
Verification: pre-existing (base c6c92f5 failures.rs:339 already runs only self.server_graph.on_file_deleted for Server/Ssr; same route). Its fix landed in 6892cd0 and 42ca0b1 removed it again. failures.rs:339-341 is the only walk for a server-side ENOENT; incremental_graph.rs:1622-1624 returns when the file is not in the server graph, so gone stays empty and the worklist at failures.rs:372-377 does nothing.
|
Reply to the review of 42ca0b1. The finding on Why the rule of 6892cd0 is not back. The proposed fix (handle the file in each graph that has it, unless this build read it there) is that rule. It had two defects:
What a complete fix needs.
That is a second change, with its own behaviour for watched files. I can add it here or in a follow-up PR. A maintainer can say which. What 42ca0b1 has. One step crosses the graphs: a client component that the client graph does not find is also handled in the server graph ( "A file that is gone for one graph is gone for the other". That sentence is in the message of the earlier commit 6892cd0. The PR body does not have it, and the body is the text of the squash commit. r4382669456 (retargeted link). No change: each page above answers 500 until restart. It is in the first Downsides bullet. #41534 and #40587 are the open changes for it. |
|
The base branch changed. #46983 is now at 886f53e.
|
Stacked on #46983.
Problem
on_file_deleted(src/runtime/bake/dev_server/incremental_graph.rs:1615) builds each importer of a deleted file again. A removed importer does not resolve:Err(_) => return Ok(())(src/bundler/bundle_v2.rs:3083) drops it and logsModuleNotFound resolving "<path>" (entry point).Fix
on_file_deletedasks the file system for each importer. A gone importer is handled as deleted, in both graphs, andmissing_sourcesfails each import bound to it.reportalso drops the directories that the resolver has as not found. A package away for one build then builds at the next save.test/bake/dev/bundle.test.ts(8 new tests, all fail on dev server: fail the importers of a file that was not there to read #46983), rest oftest/bake/. Self-reviewed: 8 concerns raised, 6 addressed (Notes).Background
node_modules. A failed open tells it that a file is gone.Downsides
.text+2,560 B. A deleted file: oneaccesscall per importer.Notes
Stack. The base branch is the branch of #46983, so this diff is only the walk: 108 source lines in 4 files (+91, -17). #46983 has the pass that fails an import bound to a missing source (
missing_sources::report). This PR gives that pass one more kind of entry and one last step.Origin. Found by a read of
on_file_deleted, not by a user. It needs a package that is removed while the server runs, and a build that opens a removed file: a build that imports asideEffects: falsebarrel with an unused export, or a request for a route that has a build error.Answers. Release builds of main 449877f, #46983 (c6c92f5) and this PR. The barrel page imports the package entry and is built after the removal. "Old modules" is a complete bundle from before the removal.
node_modules/libis a symlink), one import aboveThe same as #46983: a file that is back inside the bundle (200), a handled import, a second copy of the package higher up, an html route whose script was removed before its first build, a deleted file of a linked workspace package, a page that imports only a leaf file.
When the package is back (release builds). The package is away for one save of the barrel page. Then it is back and each page is saved.
BUN_ASSUME_PERFECT_INCREMENTAL0 and 1 give the same rows 1 to 3.Failed to load bundled module 'node_modules/lib/index.js'node_modulesis away)missing_sources::reportdrops the directories that the resolver has as not found: dev server: fail the importers of a file that was not there to read #46983 gives the directory watch the path of the missing file, and that lookup leaves the directory as not found. A page that the walk builds again does the same with its failed import.The client component step. A client component has its importers in the server graph. When the client graph is the one that fails to read it, the server graph handles the file too. The test for it is the flag of the server graph (
is_client_component_boundary). The flag of the client graph (is_hmr_root) is lost when a build with no server importer has the file again: with it, the route stays at 200 without a separate SSR graph.Work of the walk. The bundle that finds the removal. The page is N imports above the barrel, all in the package.
handle_parse_task_failure(debug, scoped logs)openatin the package,access(release, ptrace counter)openat,accessopenat,access(entry point)errors indev.log: 0 (dev server: fail the importers of a file that was not there to read #46983: 1).openat, and it asked anonLoadplugin to load a file that was gone.openat, 29getdents64and 6accesson both (ptrace counter, release builds run back to back, 2 runs each).openatand 1accessbehind a link, constant over 6 requests. Hoisted: 1 failedopenaton the first request, then 0.Size. Release builds,
sizeandnm..text: 107,560,880 to 107,563,440 B (+2,560).handle_parse_task_failure: 3,125 to 1,175 B.IncrementalGraph::on_file_deletedis out of it: 1,624 B (client) and 1,659 B (server).missing_sources::report: 4,286 to 5,325 B.note_file: 149 B.enqueue_file_from_dev_server_incremental_graph_invalidation,resolve_entry_point,resolve_import_records,on_parse_task_complete,finish_from_bake_dev_server,enqueue_parse_task,is_file_cached,finalize_bundle, andload_as_file,load_node_modules,dir_info_cached_maybe_log,finalize_result,bust_dir_cacheof the resolver.track_resolution_failurehas the same size.BundleV2orDevServer.MissingSourceis 16 B (was 8). The walk allocates only for an importer that is gone.Tests.
report, and its walk to the parent directory.bundle.test.ts,css.test.ts,html.test.ts125 of 125. 28 other files oftest/bake/,bun-serve-html*andcache-invalidation.test.tspass. Exceptions, all 5 s timeouts that a debug build of dev server: fail the importers of a file that was not there to read #46983 also has here:production.test.ts(7 to 9 tests, dev server: fail the importers of a file that was not there to read #46983: 9) and thebun bake hydratestests ofreact-spa.test.ts(1 to 2, dev server: fail the importers of a file that was not there to read #46983: 2).cargo clippy -p bun_bundler -p bun_runtime: no warning in the changed lines. Nothing ran on Windows or macOS here. The tests remove and rename directories undernode_modulesand have no platform skip.Not fixed here. No issue is filed for the lines that name no PR.
Order with open PRs.
DirectoryWatchStore::insertlooks the directory up, and only macOS and FreeBSD read the result. dev server: release the directory watch of a failed import by hash #46916 deletes that lookup. With it gone, dev server: fail the importers of a file that was not there to read #46983 alone answers like main in those rows (a patched binary, not a compiled change), and this PR still needs its last step for the pages that the walk builds again.git merge-treeof this branch with dev server: release the directory watch of a failed import by hash #46916, dev server: read a directory again when a file that no build bundled fails to open #46846, Resolver: run a missed lookup once more at a fresh Generation, with DirInfo and not-found marks under Generation #40279, dev server: keep a barrel's requests across builds, or never defer it again #46648, dev server: invalidate clientPages route scripts for a save below the browser copy of a page or layout #47017 and draft resolver: bust a directory under its real path and under each link to it #47046: no conflict insrc/.test/bake/dev/html.test.tsconflicts with dev server: release the directory watch of a failed import by hash #46916 andbundle.test.tswith resolver: bust a directory under its real path and under each link to it #47046, the same as for dev server: fail the importers of a file that was not there to read #46983 alone. None of these was built together with this branch.reportcan go.git merge-treewith main is clean. A rebase follows the base branch.Self-review. Three rounds.
DirectoryWatchStore::insertfor the end ofreport. The test withBUN_ASSUME_PERFECT_INCREMENTAL=1now makes a request while the package is away. The file-system check replaced the resolver as the test for a gone importer.DirectoryWatchStore::insertthere, and a move of the last step ofreportinto it.