Repository navigation
Conversation
StatusReproduced on main with a release build and the hoisted linker:
On main, With this PR, PR: #46153 |
|
Updated 6:56 AM PT - Oct 22nd, 2026
✅ @robobun, your commit fea0ca180b0d30e57ab0450b00e02da99ab7f2f4 passed in 🧪 To try this PR locally: bunx bun-pr 46153That installs a local version of the PR into your bun-46153 --bun |
|
No actionable comments were generated in the recent review. 🎉 ℹ️ Recent review info
WalkthroughInstallation now uses prior lockfile source information and ChangesPackage tag validation
Suggested reviewers: Priority: ➖ Normal Merge Risk: 🔵 Low · up to
|
There was a problem hiding this comment.
Nothing blocking. The comments below are optional suggestions. There is no need to push a fix for them before merging.
Beyond the inline findings, I also checked the isolated linker for the same name+version conflation: its store keys (fmt_store_key in src/install/isolated_install/Store.rs) include the full resolution (tarball URL, git commit), so a tarball and a registry package of the same version land in separate store entries and that path is not affected. The None passed from security_scanner.rs and the "unreadable .bun-tag counts as other" branch in PackageInstall::has_bun_tag were also examined and ruled out as regressions on the paths they gate.
Extended reasoning...
The change adds .bun-tag-based detection to the hoisted installer and bun prune so a tarball/git folder is not accepted as the registry package of the same name and version, plus a memo refactor in prune.rs and a new non-creating cache opener; it touches no auth, crypto or injection surface. Six verified findings are posted inline (false warnings when the cache entry is gone, nested extraneous entries no longer pruned under an OtherSource folder, empty-tag reinstall loops, and pre-existing gaps for the no-diff pull and file: cases), so a human should weigh those before merging.
Additional findings (outside the current diff — GitHub can't attach inline comments there):
-
🟡
src/install/prune.rs— Users runningbun prunewith a stale tarball or git folder now silently keep every extraneous entry under that folder's nested node_modules, where the base branch removed them. At src/install/prune.rs:1381is_installedreturns false for the newInstalled::OtherSourceverdict, soscan_folderkeeps everything, and onlyremovablesetskept_other_source, so no warn and no note are printed. The same silent keep applies to the folder's whole subtree throughowners_installedat src/install/prune.rs:1402. Fix: whenis_installedorowners_installedsee OtherSource, setkept_other_sourceand emit the same warn/note thatremovableprints, so prune never degrades to a silent no-op for the population this PR targets.Why this was flagged
The trigger is a hoisted project whose node_modules/dep-c is a tarball or git checkout with the same name and version as the registry dep-c that bun.lock now expects, and whose node_modules/dep-c/node_modules holds entries bun.lock does not list; the user runs
bun prune. On the base branchinstalledat the old src/install/prune.rs Npm arm returned Matches on name+version, sois_installedwas true and those nested entries were pruned. After the change src/install/prune.rs:1155-1158 returnsInstalled::OtherSource,is_installedat src/install/prune.rs:1065-1070 compares it to Matches and returns false, and the closure at src/install/prune.rs:1381 keeps every entry.kept_other_sourceis only set insideremovableat src/install/prune.rs:1036-1038, which this path never calls for the tarball folder itself because it is an expected entry, so the user sees exit 0, nowarn:and nonote:while prune has skipped the subtree. The PR text claims prune keeps and warns; here it keeps and stays silent.Verification: In src/install/prune.rs, the Npm arm of
installed(lines 1149-1158) returnsInstalled::OtherSource.is_installed(lines 1065-1070) compares the verdict== Installed::Matches, so it is false for OtherSource. At lines 1375-1386!is_installedshort-circuits, soremovableis never reached. Onlyremovable(lines 1035-1053) setskept_other_sourceand prints the warn, so nothing is printed. -
🟡
src/install/PackageManager/install_with_manager.rs— nit, pre-existing: after this changebun remove dep-cin the tarball-to-registry case prints1 package installedand no- dep-cline, while main prints- dep-cand1 package removed. The replacement ofnode_modules/dep-cmakesinstall_summary.success1, andprint_removedat src/install/PackageManager/install_with_manager.rs:1285-1287 requiresinstall_summary.success == 0. Fix: forSubcommand::Remove, print the removed rows wheneverrequests_removed_from_lockfile > 0, independent of how many packages the same run installed, and keep the installed count line after them.Why this was flagged
The root depends on a tarball
dep-c@ 1.0.1andpdepends on the registrydep-c@^1.0.0; the user runsbun remove dep-c. With this PRhas_other_tagat src/install/PackageInstaller.rs:2019-2021 makes the installer replacenode_modules/dep-c, soinstall_summary.successis 1.print_removedat src/install/PackageManager/install_with_manager.rs:1285-1287 isRemove && summary.remove > 0 && install_summary.success == 0, soprint_removed_rowsis skipped and theelse if this.summary.remove > 0branch at line 1305 is never reached; the summary says1 package installed. On main the folder is skipped,successis 0, and the user sees- dep-cand1 package removed. The PR description lists this output for sign-off; the condition is pre-existing code, but this change is what newly routes a plainbun removethrough it in this scenario, so a user who asked to remove a package is told one was installed.Verification: Trigger:
bun remove <name>where the removed dependency was a tarball/git package and another package needs the registry package. src/install/PackageManager/install_with_manager.rs:1285-1287 requiresinstall_summary.success == 0; on the base nothing is reinstalled, soprint_removed_rowsprints- dep-c. The new test at test/cli/install/bun-prune.test.ts:4338-4368 does not assert stdout.
14c6379 to
eb4ad72
Compare
|
Replies to the two findings outside the diff. The four inline threads have their own replies. Entries inside a folder of another source (
|
A tarball or git dependency can have the name and version of a registry package. The hoisted linker compares only the name and version in package.json, so when the name moved to the registry (`bun remove`, `bun add`, a package.json edit) it kept the old folder and reported no changes. The same run deleted the registry copy that a dependent had in its own node_modules, and the dependent loaded the old files. `bun update` has done this since #38333. `bun remove`, `bun add` and `bun install` do it since #44968. The install now replaces the folder when its `.bun-tag` is the tag of a tarball or git package that the loaded bun.lock has for that name. It checks this in a run with a package.json change and in `bun update`, `bun dedupe` and `bun audit fix`. `bun prune` and the passes that remove nested copies after an install keep the copy below a folder whose `.bun-tag` is not the registry package's own, and print a warning and a note. A tag is the package's own when it is the integrity of the package, or when the package's cache entry has the same file: the package ships it, or a patch adds it. This change does not cover a `file:` folder, a `link:` dependency (#28964), or a tarball folder that Bun 1.4.3 or older installed, which has no `.bun-tag`. When bun.lock changed outside the install, `bun install` still keeps the old folder, and `bun prune` then keeps the copy and warns.
A tarball package gets two entries in `other_tags`: its integrity and the empty tag that `--no-verify` leaves. A tarball package with no integrity in bun.lock got the empty tag twice. The list is now free of duplicates.
The installer listed the `.bun-tag` bytes of the tarball and git packages of the loaded bun.lock and compared the folder's tag with them. `bun update` can give a loaded tarball row a new integrity before the install runs, so the list missed the folder's tag and the old folder stayed. The installer now asks the lockfile from before `clean` if the name has a tarball or a git package. If it has one, a folder with a `.bun-tag` is installed again, unless the cache entry of the registry package has the same `.bun-tag`. A tarball with the bytes of the registry's tarball is replaced too. `bun prune` and the removal of nested copies take every `.bun-tag` in the folder of a registry package for another source. They no longer read the cache. For a tarball package they now compare the folder's `.bun-tag` with the integrity in bun.lock, as the installer does. Before, `bun prune` deleted a tarball copy below a registry folder of the same name.
eb4ad72 to
fea0ca1
Compare
| let tag = sys::File::read_from(package.fd(), BUN_TAG); | ||
| if !tag.is_err_and(|err| err.get_errno() == E::ENOENT) { | ||
| return Installed::OtherSource; | ||
| } |
There was a problem hiding this comment.
🔴 Users of a registry package whose published files include a .bun-tag now get a permanent warning from bun prune and from installs that remove collapsed copies, and the stale nested copy is never removed. prune.rs:1150-1153 answers Installed::OtherSource for any readable or unreadable .bun-tag in a registry folder, while the install side (has_other_bun_tag, PackageInstall.rs:1282-1294) treats a tag equal to the cache entry's own .bun-tag as the package's own and keeps the folder, so the printed note cannot clear it. Fix: make prune and the removal passes use the same own-tag rule as the install (a tag equal to the cache entry's .bun-tag, or the patched entry's, is not OtherSource), with a missing cache entry falling back to Matches.
Why this was flagged
A registry package tagged@ 1.0.0 whose tarball contains a .bun-tag is hoisted to node_modules/tagged, and a dependent has a collapsed copy at node_modules/dependent/node_modules/tagged. The user runs bun prune, or any bun add/bun remove/bun update that reaches plan_and_remove_collapsed_copies. removable (prune.rs:1024) calls installed; prune.rs:1150-1153 reads .bun-tag, it is not ENOENT, and returns Installed::OtherSource without consulting the cache entry. removable at prune.rs:1033-1050 keeps the copy and warns has a .bun-tag but bun.lock expects a registry package; keeping ...; the note at prune.rs:1381 or prune.rs:1744 says to remove the folder and run 'bun install'. Doing so reinstalls the same package from the cache, which has_other_bun_tag at PackageInstall.rs:1291 accepts because the cache entry has the identical .bun-tag, so the next prune warns again: the warning is permanent and the copy is never removed. On the base branch the Npm arm compared only name and version, so the copy was removed with no output.
Verification: In src/install/prune.rs the Npm arm of installed at lines 1150-1153 reads BUN_TAG and returns Installed::OtherSource unless the read fails with ENOENT. On the base branch the Npm arm compared name and version only, so the copy was removed with no output. src/install/PackageInstall.rs:1282-1294 has_other_bun_tag treats a tag equal to the cache entry's .bun-tag as the package's own.
| Ok(tag) => { | ||
| let own = path::resolve_path::join_z::<path::platform::Posix>(&[ | ||
| self.cache_dir_subpath.as_bytes(), | ||
| BUN_TAG, | ||
| ]); | ||
| !sys::File::read_from(self.cache_dir, own).is_ok_and(|own| own == tag.bytes) | ||
| } |
There was a problem hiding this comment.
🟡 nit (optional): a .bun-tag that opens but cannot be read is not treated as another package's tag, so the directory case is decided by the cache entry instead of the read error. src/install/PackageInstall.rs:1286 takes the Ok(tag) arm with empty tag.bytes and ignores tag.err (read_small_file at src/install/PackageInstaller.rs:323-329 wraps a read failure such as EISDIR in Ok), so the folder is kept whenever the cache entry's .bun-tag is empty. Fix: treat tag.err.is_some() like the Err(_) arm (return true), so every unreadable tag counts as another package, matching prune.rs:1151 which uses read_from and already does.
Why this was flagged
On Linux and macOS open(O_RDONLY) on a directory succeeds and the read fails with EISDIR. read_small_file (src/install/PackageInstaller.rs:320-329) returns Ok(ReadToEndResult { bytes: Vec::new(), err: Some(EISDIR) }) for a node_modules/<name>/.bun-tag that is a directory. has_other_bun_tag (src/install/PackageInstall.rs:1283-1293) only maps open errors to true; the Ok(tag) arm at :1286 never looks at tag.err and compares the empty bytes with the cache entry's .bun-tag at :1291. For a registry package whose cache entry has an empty .bun-tag the empty bytes compare equal and needs_install at src/install/PackageInstaller.rs:2038-2040 stays false, so the foreign folder is kept. The prune side at src/install/prune.rs:1150-1153 uses sys::File::read_from, which returns Err on a read failure, so the two checks disagree on the same folder. The base branch had no such check; the inconsistency is new.
Verification: nit. node_modules/<name>/.bun-tag opens but cannot be read while the cache entry ships an empty .bun-tag. read_small_file (src/install/PackageInstaller.rs:320-329) wraps a read failure as Ok; the Ok(tag) arm at src/install/PackageInstall.rs:1286 never inspects tag.err, so the folder is kept, unlike src/install/prune.rs:1150-1153 which returns OtherSource.
|
Closing as stale: this PR conflicts with main, and its last commit is from 2026-10-22. The branch stays, so the work is not lost. To pick it up again, rebase the branch on main and reopen this PR. |
Related to #28964
Problem
bun install --frozen-lockfileexits 0:PackageInstall::verifycompares only name and version.bun updatesince install: pnpm parity — dedupe, prune, pm licenses, audit fix, add --filter/--catalog, nested overrides, transitive update, and workspace fixes #38333, the rest since unreleased install: consolidate the open package manager pull requests #44968).Fix
.bun-tag, when the lockfile had a tarball or git package of that name.bun pruneand the removal of nested copies keep the copy below such a folder and warn. They also check a tarball folder's.bun-tag.test/cli/install/bun-prune.test.ts(26 new cases, 22 fail on main) and 27 more install suites (Notes).Background
.bun-tag(integrity or commit) into tarball and git folders, not into registry packages.file:folders and links, and needs a maintainer's decision.Downsides
file:folders, links, tarball folders from Bun 1.4.3 (copy deleted, 1.4.2 keeps it), and bun.lock changes outside the install.bun remove --offlineexits 1 if the registry package is not cached.openat: 65 to 71 whenbun prunechecks 6 folders, 43 to 44 inbun addbeside a same-name tarball..text: +1,280 bytes.Notes
Origin
A fuzz run of install flows found this. No user reported it. #28964 is a user's report of the same class for a
link:dependency.Example
The root depends on a tarball
dep-c@1.0.1.pdepends on the registrydep-c@^1.0.0, sophas its own copy atnode_modules/p/node_modules/dep-c. Runbun remove dep-c. bun.lock then has onedep-c, the registry package, at the root.node_modules/dep-cis still the tarball, the copy belowpis deleted, andploads the tarball.bun install --frozen-lockfileprintsChecked 2 installs across 3 packages (no changes)and exits 0.node_modules/dep-cis the registry package and the copy belowpis removed, as after a first install.Where the change is (source: +115, -45 in 6 files)
needs_installtest insrc/install/PackageInstaller.rs.had_other_sourceasks the lockfile from beforecleanfor a tarball or git package of the name (Lockfile::packages_named).PackageInstall::has_other_bun_tagreads the folder's.bun-tagand, if there is one, the.bun-tagof the cache entry that the install copies.read_bun_tagis the one reader of the folder's.bun-tag.verify_bun_taguses it too.HoistedTree::installedinsrc/install/prune.rsanswers the newInstalled::OtherSourcefor a folder with a.bun-tag. The tarball arm compares the folder's.bun-tagwith the integrity in bun.lock throughtarball_tag_matches, which the installer and the isolated linker already use.other_source_aboveasks the registry question for the pass that removes what Bun 1.4.3 laid (remove_from_older_places).verified_installedis oneVec<Cell<Option<Installed>>>, because a fourth answer does not fit the three bit sets.install_with_manager.rsgives the removal's existing condition a name (changes_places) and passes the lockfile from beforecleanto the hoisted install under it.Rules
bun update,bun dedupeorbun audit fix. The lockfile from beforecleanhas a tarball or git package of that name. The folder has a.bun-tag, or one that cannot be opened. The cache entry of the registry package does not have the same.bun-tag.--no-verify), a tarball with the registry's own bytes, and a tag that gains a second line all count..bun-tagis another source.bun prunedoes not read the cache..bun-tagthat equals the integrity in bun.lock. An empty tag counts, becausebun install --no-verifyleaves one. Before, the name alone was enough, andbun prunedeleted a workspace's tarball copy below a registry folder of that name.file:folder is still accepted on its name, as on main.New output, for sign-off
The note above is from
bun prune. After an install the note isremove the folder with the .bun-tag and run 'bun install', then 'bun prune'. For a tarball package,bun pruneprints the existingis not the version bun.lock expectswarning and the existingrun 'bun install' firstnote.The commands that replace the folder now print what Bun prints for every run that installs a package:
bun remove dep-c- dep-c,done1 package installedbun add dep-c@1.0.1installed dep-c@1.0.1,doneinstalled dep-c@1.0.1,1 package installedbun installChecked 2 installs across 3 packages (no changes)+ dep-c@1.0.1,1 package installedprint_removedhas neededinstall_summary.success == 0since #38333, so abun removethat installs a package prints no- namerow.Source by flow
Hoisted linker. The root has another source of
dep-c@1.0.1, andpneeds the registrydep-c@1.0.1. Flows:bun remove,bun add, edit andbun install, edit andbun update.bun update..bun-tag)file:folder,link:So main and this PR delete the dependent's copy in the last two rows, where 1.4.2 keeps it. Two
test.todocases record them.Measurements
Release builds of main (9634520) and of this PR. Syscalls counted with gdb catchpoints, all threads.
openatmainopenatPRbun install, no changesbun prune, clean treebun add other, no tarball or git packagebun add other, a tarballdep-c, no registrydep-cbun add other, a tarballdep-cand a registrydep-cbelowpbun prune, 5 stray copies of registry packages (6 folders checked)bun prune, 10 stray copies (11 folders checked)bun prune, a stray copy of a tarball package (2 folders checked)read,newfstatat,unlinkatandgetdents64are equal in every row.closeis equal except in the last row (31 to 32: the.bun-tagof the tarball folder opens).writediffers by one between two runs of one binary..bun-tagof a cache entry only for a folder that has a.bun-tag.bun installwith no changes 364 to 364,bun pruneon a clean tree 312 to 310,bun add other587 to 583.size -A):.text81,932,069 to 81,933,349 (+1,280)..rodata,.dataand.bssare equal.strace,perf,valgrindandbloatyare not installed on this machine.Tests
test/cli/install/bun-prune.test.ts: 26 new cases and 2test.todocases. On a debug build of main (9634520), 22 of the 26 fail. The other 4 are guards: a registry package with and without a shipped.bun-tagbeside a tarball of its name, a registry package whose patch adds the file, andbun pruneover a patched tarball folder..bun-tagthat is a directory (tarball and git), a.bun-tagthat is a link to itself (not on Windows: a file link needs a privilege there), an empty.bun-tag,bun updatethrough a transitive dependency,bun updateafter the tarball got other bytes,bun dedupe. Each row of the table also asserts stdout.has_other_bun_tag, the empty tag in the tarball arm, and theDedupeandUpdatearms ofchanges_places. Six fail their own case. TheUpdatearm fails none: in thebun updatecases here, main's condition is true without it.bun-prune(202 pass, 1 skip, 2 todo),hoist,bun-remove,bun-add,bun-update,bun-update-transitive,bun-install-tarball-integrity,bun-install-patch,bun-patch,bun-dedupe,bun-audit,bun-link,bun-install-staging,bun-workspaces,bun-workspaces-self-contained,overrides,nested-overrides,frozen-lockfile-pruned,frozen-lockfile-outdated,bun-install-offline,lockfile-only,bun-install-backends,bun-lock,catalogs,isolated-install,bun-install,bun-install-registry. Inbun-install-git-depsone case hit its own 30 s limit in the full run ("a long chain of links costs no more than as many links to one file"). Alone it passes in 13 s here and in 16 s on main.cargo checkfor Linux,x86_64-pc-windows-msvcandaarch64-apple-darwin, andcargo clippyonbun_install: no warnings. I could not run Windows or macOS.Limits, in detail
file:folder andlink:: their folders have no.bun-tag. The old folder stays and the dependent's copy is deleted, without a message. install(hoisted): do not accept a link in the place of a registry package #45749 is open for links..bun-tag. If the switch is the first install with the new Bun, the result is main's. One plainbun installbefore the switch lays the tag, if the tarball is still there. A git checkout has a tag in every version.git pull,bun install --lockfile-only,bun install --filterthat leaves the folder out,bun add <tarball> --no-saveand thenbun install):bun installreports no changes and keeps the old folder.bun pruneand an install that leaves out dependencies keep the dependent's copy and warn..bun-tag, or whose patch adds one:bun pruneand the removal after an install keep a nested copy of it and warn, where main removes the copy. The note's remedy does not clear this. I know no published package with a.bun-tagat its root. The install does not replace such a folder.bun prunedoes not clean inside the folder of another source, and prints no message for it. Main has this rule for a folder of another version (owners_installed).bun prunekeeps a copy below a tarball folder without the right.bun-tag, and asks forbun install. A tarball folder that Bun 1.4.3 laid is such a folder until onebun install.bun remove --offlineandbun install --offlineexit 1 (error: --offline: "dep-c" is not in the cache) when the registry package is not in the cache. Main keeps the old folder, deletes the dependent's copy and exits 0.bun pruneaccepts any folder with the right name for afile:folder package, as on main.bun patch <path>andbun patch --commit <path>still take a tarball or git folder for the registry package of its name and version..bun-tag. install: clear a package folder that cannot be renamed aside instead of linking on top of it #45572 is open for that.Other designs
node_modules. It coversfile:folders, links, a pulled bun.lock and the registry-to-registry case. It costs one file read and one write per hoisted install, and it is a new file innode_modules. install: consolidate the open package manager pull requests #44968 names a state file per project as the complete fix for another gap (a killed install) and adds none.file:folders and links in the run that changes them. A branch with that design exists (robobun/a6d3d1e4/relink-changed-source). A review of it found layouts where it installs a folder again that did not change: two trees that share one folder, a saved tree that differs from the linked one, and--no-save._bunPatchHash)..bun-tagfor registry packages, written at extraction. Trees and caches from older versions have none..bun-tagin every registry folder in every install: one moreopenatper registry package in the run with no changes.This PR uses the tag that is already on disk, only in the run that changes the source, because that adds no format and no work to an install without a tarball or git package.
Review
The review before this push raised 28 concerns. 15 changed the code or the tests: the name condition in place of a list of tag bytes, the compare with the cache entry, no rule for a tarball with the registry's bytes, no cache lookup in
bun prune, the tarball arm, and the new test rows. 10 changed this text. Not taken:file:folder switch keeps the dependent's copy as 1.4.2 does. It is a second mechanism and it leaves the old folder in place. It is handed off as its own change.bun pruneshare for every source kind. The tarball arm now uses the sharedtarball_tag_matches. The rest is a larger change.Related open PRs
HoistedTreeinsrc/install/prune.rs: 9 conflict hunks with this PR, and it adds twoInstalledanswers of its own. It merges cleanly into main. Suggested order: this PR first, because it is smaller and its installer half conflicts with nothing there. install: prune the hoisted trees of one node_modules directory as one folder #46002 then takesOtherSourceas one more arm..bun-tagfor a patched tarball or git copy: 1 conflict hunk, atverify_bun_tag. After it, the tarball arm ofHoistedTree::installedmust compare line 1. The case "bun prune removes a copy below the folder of a patched tarball" is the guard for that.needs_installtest and at the.bun-tagreader.needs_installtest..bun-tagof GitHub copies, which this PR does not parse.no test proof · iteration 0 · platform-specific test(s) that do not run on this machine, deferring to CI, which covers all platforms: test/cli/install/bun-prune.test.ts