Skip to content

dev server: continue the deleted-file walk through importers that are gone - #47016

Open
robobun wants to merge 4 commits into
robobun/2d8d0266/dev-server-missing-file-importersfrom
robobun/dd8c019a/dev-server-walk-unresolvable-importers
Open

robobun wants to merge 4 commits into
robobun/2d8d0266/dev-server-missing-file-importersfrom
robobun/dd8c019a/dev-server-walk-unresolvable-importers

Conversation

@robobun

@robobun robobun commented Oct 26, 2026 •

Copy link
Copy Markdown
Collaborator

Stacked on #46983.

Problem

  • A package is removed under a running dev server. A page two imports above a removed file answers 200 with old modules.
  • on_file_deleted (src/runtime/bake/dev_server/incremental_graph.rs:1615) builds each importer of a deleted file again. A removed importer does not resolve: Err(_) => return Ok(()) (src/bundler/bundle_v2.rs:3083) drops it and logs ModuleNotFound resolving "<path>" (entry point).

Fix

  • on_file_deleted asks the file system for each importer. A gone importer is handled as deleted, in both graphs, and missing_sources fails each import bound to it.
  • Correct: a gone file cannot be built, so its importers report it. For a client component those are in the server graph, and the walk goes there.
  • report also drops the directories that the resolver has as not found. A package away for one build then builds at the next save.
  • Verified: test/bake/dev/bundle.test.ts (8 new tests, all fail on dev server: fail the importers of a file that was not there to read #46983), rest of test/bake/. Self-reviewed: 8 concerns raised, 6 addressed (Notes).

Background

Downsides

Notes

Stack. The base branch is the branch of #46983, so this diff is only the walk: 108 source lines in 4 files (+91, -17). #46983 has the pass that fails an import bound to a missing source (missing_sources::report). This PR gives that pass one more kind of entry and one last step.

Origin. Found by a read of on_file_deleted, not by a user. It needs a package that is removed while the server runs, and a build that opens a removed file: a build that imports a sideEffects: false barrel with an unused export, or a request for a route that has a build error.

Answers. Release builds of main 449877f, #46983 (c6c92f5) and this PR. The barrel page imports the package entry and is built after the removal. "Old modules" is a complete bundle from before the removal.

Case main #46983 this PR
hoisted, a page one import above the barrel 500 500 500
hoisted, a page two or three imports above 200, old modules 200, old modules 500
linked (node_modules/lib is a symlink), one import above 200, undefined import 200, undefined import 500
linked, two or three imports above 200, old modules 200, old modules 500
two links to one store directory and a sub directory, three pages above 200, old modules 200, old modules 500
both pages saved into one bundle, both 500 (10 runs) 4 3 10
link retargeted to a new store directory, page above 200, old modules 200, old modules 500 until restart
a route imports a client component of the package that imports the barrel, the client graph finds the barrel gone 200, undefined import 200, undefined import 500
a route imports a client component that is a barrel, the client graph finds it gone (with and without a separate SSR graph) 200, the client bundle lacks a module, or old modules the same 500
a page imports a module that a plugin serves with no file, and that module imports the package 200 200 500
only the browser copy of a client component imports the package, the server graph finds the removal 200, old modules 200, old modules 200, old modules

The same as #46983: a file that is back inside the bundle (200), a handled import, a second copy of the package higher up, an html route whose script was removed before its first build, a deleted file of a linked workspace package, a page that imports only a leaf file.

When the package is back (release builds). The package is away for one save of the barrel page. Then it is back and each page is saved. BUN_ASSUME_PERFECT_INCREMENTAL 0 and 1 give the same rows 1 to 3.

Layout main #46983 this PR
a page two imports above the barrel 200, the page throws Failed to load bundled module 'node_modules/lib/index.js' both pages 500 both pages 200 and run
the page imports a file of a package sub directory both pages 500 both pages 500 both 200 and run
the page reaches the barrel through a second package (that stays, that is away too, or all of node_modules is away) 200, the page throws barrel page 500, the page throws both 200 and run
one page that imports the package entry 200 500 200
the same, and one request for the page while the package is away 200 500 until restart 500 until restart
the barrel page is saved two times while the package is away 500 until restart 500 until restart 500 until restart

The client component step. A client component has its importers in the server graph. When the client graph is the one that fails to read it, the server graph handles the file too. The test for it is the flag of the server graph (is_client_component_boundary). The flag of the client graph (is_hmr_root) is lost when a build with no server importer has the file again: with it, the route stays at 200 without a separate SSR graph.

Work of the walk. The bundle that finds the removal. The page is N imports above the barrel, all in the package.

Package #46983 this PR
50 files, one directory hand-offs to handle_parse_task_failure (debug, scoped logs) 2 1
listing drops that remove an entry 3 2
failed openat in the package, access (release, ptrace counter) 8, 0 4, 50
50 files, 5 directories hand-offs 3 1
listing drops that remove an entry 4 14
failed openat, access 9, 0 11, 50
2000 files, 20 directories hand-offs 3 1
listing drops that remove an entry 4 44
failed openat, access 9, 0 26, 2000
  • (entry point) errors in dev.log: 0 (dev server: fail the importers of a file that was not there to read #46983: 1).
  • An earlier form of this PR asked the resolver for each importer. For 2000 files in 20 directories it had 41 hand-offs, 62 listing drops and 2,064 failed openat, and it asked an onLoad plugin to load a file that was gone.
  • No removal, 2 routes and 20 saves: 0 hand-offs and 0 listing drops on both builds. 85 or 86 openat, 29 getdents64 and 6 access on both (ptrace counter, release builds run back to back, 2 runs each).
  • A request for a failing route, release: 3 failed openat and 1 access behind a link, constant over 6 requests. Hoisted: 1 failed openat on the first request, then 0.

Size. Release builds, size and nm.

  • .text: 107,560,880 to 107,563,440 B (+2,560).
  • handle_parse_task_failure: 3,125 to 1,175 B. IncrementalGraph::on_file_deleted is out of it: 1,624 B (client) and 1,659 B (server).
  • missing_sources::report: 4,286 to 5,325 B. note_file: 149 B.
  • Same instruction lists as dev server: fail the importers of a file that was not there to read #46983: enqueue_file_from_dev_server_incremental_graph_invalidation, resolve_entry_point, resolve_import_records, on_parse_task_complete, finish_from_bake_dev_server, enqueue_parse_task, is_file_cached, finalize_bundle, and load_as_file, load_node_modules, dir_info_cached_maybe_log, finalize_result, bust_dir_cache of the resolver. track_resolution_failure has the same size.
  • No new field in BundleV2 or DevServer. MissingSource is 16 B (was 8). The walk allocates only for an importer that is gone.

Tests.

Not fixed here. No issue is filed for the lines that name no PR.

Order with open PRs.

Self-review. Three rounds.

…not resolve

The dev server learns that a file is gone when a build fails to open it.
It then builds the importers of the file again, so that each one reports
its failed import. An importer that is gone too, for example another file
of a removed package, does not resolve as an entry point. The walk dropped
it, so a page above it kept its old modules, and resolve_entry_point
logged the miss as an error of the build.

- on_file_deleted is now a walk. It handles an importer that the resolver
  does not find like the deleted file: it disconnects its imports, marks
  it stale and builds its importers. The bundle gets the file as a missing
  source, so missing_sources::report fails an import that is bound to it.
- The importers of a client component are in the server graph. The walk
  continues there.
- The resolver gets one more look at a directory before a file in it is
  handled as gone, one time per directory per walk.
- DirectoryWatchStore does not leave a directory that is gone cached as
  missing. A package specifier never asks the resolver again, so a package
  that came back stayed unresolved.
@robobun

robobun commented Oct 26, 2026 •

Copy link
Copy Markdown
Collaborator Author
Updated 10:42 AM PT - Oct 27th, 2026

✅ @robobun, your commit be8a64c2b6eed68ceecc03688355df8660ac7e6d passed in Build #133757! 🎉


🧪   To try this PR locally:

bunx bun-pr 47016

That installs a local version of the PR into your bun-47016 executable, so you can run:

bun-47016 --bun

@robobun

robobun commented Oct 26, 2026 •

Copy link
Copy Markdown
Collaborator Author

Status

@claude claude Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nothing blocking. The comments below are optional suggestions. There is no need to push a fix for them before merging.

Beyond the inline findings, I also checked that moving the bust_dir_cache call into DevServer::on_file_deleted is behavior-preserving (in_graph is exactly the old "index found" condition), that every caller of on_file_deleted / enqueue_file_from_dev_server_incremental_graph_invalidation and every match on MissingSource was updated, and that the one-more-look retry after a package reinstall and a plugin onResolve returning an absolute path with no file on disk do not misclassify an existing importer as deleted.

Extended reasoning...

The change reworks the Bake dev server's deleted-file walk across bundle_v2.rs, missing_sources.rs, transpiler.rs, directory_watch_store.rs, failures.rs and incremental_graph.rs, plus 8 new tests; it touches no auth, crypto or injection surface. Four confirmed findings are posted inline, so a human review is already signalled; this note only records the additional refactor-equivalence, caller-audit and ruled-out checks.

Comment on lines +1708 to +1712
InvalidationOutcome::Unresolved => {
bv2.note_missing_file(key);
walk.unresolved.put(key, ())?;
// The other target has the same file.
break;

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟡 (optional) Developers who replace a package under a running dev server with the isolated linker (link retargeted to a new store directory) now get 500 on every page above it until they restart; on the base those pages keep answering 200. The walk at incremental_graph.rs:1708-1712 marks an importer whose realpath key no longer resolves as gone and rebuilds every page above it, and the resolver's cached link target still points at the removed store, so the rebuild fails for good. Fix: when an importer under a store directory is handled as gone, also drop the cached listing of the directory that holds the link (or re-resolve the page's specifier rather than the realpath key) so a retargeted link is picked up at the next build.

Why this was flagged

Trigger: isolated-linker project, dev server running; bun add lib@ 2 removes node_modules/.store/lib@ 1 and retargets node_modules/lib to .store/lib@ 2. rebuild_importer_of_deleted_file calls enqueue_file_from_dev_server_incremental_graph_invalidation with the importer's realpath key, which returns Unresolved (bundle_v2.rs:3091); the retry busts only dirname(key) inside the old store (incremental_graph.rs:1697-1699), then incremental_graph.rs:1709-1712 notes the file missing and the loop at failures.rs:414-419 rebuilds every page above. Those pages resolve lib/outer.js through the resolver's cached entry for node_modules/lib, whose link target is still the removed store, so missing_sources::report fails them on every build until restart. Base: Err(_) => return Ok(()) at the old bundle_v2.rs:3080 stopped the walk, so pages two or more imports above kept serving the previous bundle. The DirectoryWatchStore::insert bust (directory_watch_store.rs:312-315) only drops the gone store directory, not the directory holding the link, so it does not recover this case.

Verification: Nothing busts node_modules/ or node_modules/<pkg>, whose cached DirInfo.abs_real_path is the old store target (resolver.rs:6382, 6413), so each rebuilt page resolves its import back to the removed realpath and the route answers 500 on every build until restart. On the base the same importer hit Err(_) => return Ok(()) (old bundle_v2.rs:3080) and pages above kept serving the old lib@ 1 bundle (200).

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is the second item under Downsides, and it stays in this PR.

Measured on release builds with the link retargeted to a new store directory: #46983 answers 500 for the page whose build found the removal and 200 with the old modules for the page above. This PR answers 500 for both, until restart.

A maintainer decides if 500 until restart is acceptable before those land. The alternative is what the base does: the pages above a removed store directory keep the old modules.

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Follow-up for 42ca0b1: the answer is the same. A link that points to a new store directory leaves each page above at 500 until restart. It is in the first Downsides bullet and in row 7 of the Answers table now. #41534 and #40587 are the open changes for the listing under the path of a link.

Comment thread src/bundler/bundle_v2.rs Resolved
Comment thread src/runtime/bake/dev_server/failures.rs Outdated
Comment on lines +391 to +394
let in_graph = match graph {
bake::Graph::Server | bake::Graph::Ssr => self
.server_graph
.on_file_deleted(abs_path, bv2, &mut walk)?,

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟣 pre-existing, not blocking: Users who delete a file shared by server and client code keep getting the old client bundle: the client importers are never rebuilt when the deletion is reported through the server graph. failures.rs:391-394 walks only the server graph for Graph::Server/Ssr, while the Client arm at failures.rs:395-406 adds a hop to the server graph. The new hop is one-directional, so a file present in both graphs whose client parse task is cached (no ENOENT on the client side this build) leaves the client graph with dangling edges and stale modules. Fix: when the deleted file is in both bundled_files maps, run on_file_deleted on both graphs regardless of which graph reported it.
A small fix can ride a push you are already making; otherwise a short reply is enough.

Why this was flagged

Trigger: a util file imported by a server page and by client code is deleted. The server parse task fails first, handle_parse_task_failure (src/runtime/bake/dev_server/failures.rs:339) calls on_file_deleted with Graph::Server; failures.rs:391-394 runs only server_graph.on_file_deleted. If the client side of that file was not a parse task in this build (the client page files are only appended when stale, bundle.rs:53-59), no client walk ever happens; client importers keep their edges to the gone file and the browser receives the old module. The PR adds the reverse hop (failures.rs:398-406) for is_hmr_root only, so the walk added by this change is incomplete for the sibling direction; REVIEW.md's "fix the whole class" applies. The base had the same single-graph walk, but the PR's stated purpose is that pages above a removed file stop serving old modules, and this population still does. Remedy: check membership in both graphs and walk each one that has the file.

Verification: A file present in both the server and client graphs is reported missing by a Server/Ssr-target parse task. src/runtime/bake/dev_server/failures.rs:391-394 runs only self.server_graph.on_file_deleted(...) for Graph::Server | Ssr. So client-graph importers of the deleted file keep their edges. Severity is pre-existing because the base already fails identically by the same route.

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done in 6892cd0. The walk now handles the file in the other graph too, unless this build read the file for that graph.

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

42ca0b1 takes the change of 6892cd0 back, so this thread is open again.

Why the rule of 6892cd0 is gone:

  • It cut the import edges of the file in the other graph and failed no import there. With two files in one save, a route answered 200 with a client bundle that did not have a module of its own code.
  • It changed two flows of watched files with no test: a server route whose css import or shared import is deleted answered 500 where dev server: fail the importers of a file that was not there to read #46983 answers 200.

What 42ca0b1 does:

  • The file whose read failed is handled in the graph of that read, as on main.
  • One step crosses the graphs: a client component that the client graph does not find is also handled in the server graph, because its importers are there. The test is a removed package with "use client" modules fails the routes that import the modules, with and without a separate SSR graph.
  • An importer whose file is gone too is handled in both graphs.

The direction of this finding is not fixed here: the server graph finds a file gone, and the client graph has importers of it. It answers as on main and on #46983 (200 with old modules). The body lists it under "Not fixed here".

…ng ancestor of its directory

- A file that one graph does not find is gone for the other graph too,
  unless this build read it there: a plugin can give one graph the source.
  The walk handles the file in both graphs. This replaces the hop for
  client components, and it builds the client importers again when the
  server graph finds the file gone.
- DirectoryWatchStore drops every missing ancestor of a directory that
  gets no watch. A lookup of node_modules/lib/sub caches node_modules/lib
  as missing, so a page that imports a file of a sub directory stayed
  unresolved after the package was back.
@robobun

robobun commented Oct 26, 2026

Copy link
Copy Markdown
Collaborator Author

Review follow-up in 6892cd0. Each thread has the details.

  • The file is now handled in the other graph too, unless this build read it there (failures.rs). Fixed, with a new test.
  • The drop in DirectoryWatchStore::insert now covers each missing ancestor (directory_watch_store.rs). Fixed. The return test now imports a file of a sub directory.
  • Resolver errors other than not found (bundle_v2.rs): no change. The resolver folds a failed directory open into ModuleNotFound, so the walk gets no other error in those cases.
  • A retargeted link (incremental_graph.rs): no change. It is the second item under Downsides. The thread stays open for a maintainer.

@claude claude Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I re-reviewed the second push (6892cd0) and found no new bugs; it addresses the one-directional graph hop and the ancestor busting from my earlier comments. Since the change still rewires the deleted-file walk across both incremental graphs and the resolver's directory cache, a human look remains worthwhile.

What was reviewed:

  • DevServer::on_file_deleted: the has_source_of gate and the walk.unresolved loop (keys are owned Box<[u8]>, cloned per step while the map grows) are sound.
  • DirectoryWatchStore::insert ancestor loop: bounded by the root check; a parent whose read errors (not just missing) is also dropped, which only costs a re-listing.
  • MissingSource::File in missing_sources::report: the single match on the enum is updated and BOTH_SIDES matches the side() bits.
  • The eight new tests: no sleeps, exact error strings, await using cleanup.
Extended reasoning...

The change touches the bundler's dev-server entry (bundle_v2.rs, transpiler.rs, missing_sources.rs) and the bake dev server's incremental graph, failure handling and directory watch store, replacing the per-graph deleted-file handling with a walk that treats unresolvable importers as deleted and busts resolver directory caches. It touches no security-sensitive surface. Deferred rather than approved because the walk mutates incremental-graph edges and resolver cache state across both graphs, and two of my earlier inline threads (the resolver error-kind mapping at bundle_v2.rs:3098 and the retargeted-link case at incremental_graph.rs:1712) were not changed by this push.

Still open from earlier reviews (1):

  • Unresolved: 1 minor or pre-existing.

The walk over the importers of a deleted file asked the resolver, and the
resolver can answer from a directory listing older than the removal. The
walk now checks the file of each importer. An importer that is gone is
handled like the deleted file, for both graphs.
`enqueue_file_from_dev_server_incremental_graph_invalidation` and
`resolve_entry_point` are as before.

The file whose read failed is handled in the graph of that read. A client
component that the client graph does not find is also handled in the
server graph, where its importers are. The server graph keeps the flag
for that: the flag of the client graph is lost when a build with no
server importer has the file again. The rule of the last commit for the
other graph is gone.

`missing_sources::report` ends with a drop of each directory of a missing
file that the resolver has as not found. `DirectoryWatchStore::insert` is
as before.
@robobun robobun changed the title dev server: continue the deleted-file walk through importers that do not resolve dev server: continue the deleted-file walk through importers that are gone Oct 27, 2026

@claude claude Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nothing blocking. The comments below are optional suggestions. There is no need to push a fix for them before merging.

Still open from earlier reviews (2):

  • Unresolved: 2 minor or pre-existing.

Comment on lines +339 to +341
bake::Graph::Server | bake::Graph::Ssr => self
.server_graph
.on_file_deleted(abs_path, bv2, &mut gone)?,

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟣 pre-existing, not blocking: Pre-existing, and the earlier review's fix (the failures.rs:394 entry, applied in 6892cd0) was removed again in 42ca0b1: when the server graph finds a shared file gone, client importers keep serving the old module. failures.rs:339-341 walks only server_graph for Graph::Server | Ssr, so the client graph's copy keeps its edges and nothing rebuilds its importers. The PR text still claims "a file that is gone for one graph is gone for the other", and the test for that direction was deleted with the code. Fix: for every deleted file walk each graph that has it, in both directions, skipping a graph only when this build re-read the file there.
A small fix can ride a push you are already making; otherwise a short reply is enough.

Why this was flagged

Setup from the removed test: Local.ts ("use client") imports '#lib' whose browser condition maps to 'lib'; routes/local.ts imports Local.ts; routes/barrel.ts imports 'lib' on the server. After /local is built the package is removed and /barrel is fetched. The server parse of node_modules/lib/index.js fails with ENOENT and reaches handle_parse_task_failure with Graph::Server; failures.rs:339-341 calls only self.server_graph.on_file_deleted, and index.js is not in the server graph, so the walk returns at incremental_graph.rs:1622-1623. The client graph still holds the edge Local.ts -> index.js, Local.ts is not stale, and the next /local answers 200 with the old bundle instead of the "Could not resolve: "#lib"" error. The base branch behaves the same, so this is pre-existing; commit 6892cd0 fixed it with has_source_of and the test "a removed package that the server graph finds gone fails a client component that imports it", and 42ca0b1 deleted both.

Verification: pre-existing (base c6c92f5 failures.rs:339 already runs only self.server_graph.on_file_deleted for Server/Ssr; same route). Its fix landed in 6892cd0 and 42ca0b1 removed it again. failures.rs:339-341 is the only walk for a server-side ENOENT; incremental_graph.rs:1622-1624 returns when the file is not in the server graph, so gone stays empty and the worklist at failures.rs:372-377 does nothing.

@robobun

robobun commented Oct 27, 2026

Copy link
Copy Markdown
Collaborator Author

Reply to the review of 42ca0b1.

The finding on failures.rs:341. It is correct, and this PR does not fix it. When the server graph finds a file gone and the client graph has importers of it, the answer is the same as on main and on #46983: 200 with old modules. The body lists it under "Not fixed here" (last row of the first table).

Why the rule of 6892cd0 is not back. The proposed fix (handle the file in each graph that has it, unless this build read it there) is that rule. It had two defects:

What a complete fix needs.

  • The file is also recorded as missing for the other graph, so that missing_sources::report fails the imports that the build already bound there.
  • The check for "this build read it there" does not count the source whose read just failed.
  • The two flows of watched files get tests and a line in the body.

That is a second change, with its own behaviour for watched files. I can add it here or in a follow-up PR. A maintainer can say which.

What 42ca0b1 has. One step crosses the graphs: a client component that the client graph does not find is also handled in the server graph (is_client_component_boundary), because its importers are there. An importer whose file is gone too is handled in both graphs.

"A file that is gone for one graph is gone for the other". That sentence is in the message of the earlier commit 6892cd0. The PR body does not have it, and the body is the text of the squash commit.

r4382669456 (retargeted link). No change: each page above answers 500 until restart. It is in the first Downsides bullet. #41534 and #40587 are the open changes for it.

@robobun

robobun commented Oct 30, 2026

Copy link
Copy Markdown
Collaborator Author

The base branch changed. #46983 is now at 886f53e.

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant