Repository navigation
Conversation
|
Status: the fix is in the branch and ran on both darwin lanes (build 129314). Reproduction: cache and project on one HFS+ or exFAT disk image (
With the cache on APFS and the project on the image, both linkers exit 0 on main: Without a special volume: Errno of each call on the three volumes (0 is success):
PR: #46022 |
…il with ENOTSUP Darwin has two errnos for "not supported": ENOTSUP (45) and EOPNOTSUPP (102). The kernel reports 45 when a file system does not have an operation, and libcopyfile sets 45 for a source it refuses. Zig's Darwin errno table named 45 OPNOTSUPP, so the gates matched it. The Rust table has EOPNOTSUPP = 102 and the gates were carried over by name, so they compared 45 with 102 and never fired. bun_errno::is_not_supported(E) is true for both values. Five gates call it: install_with_clonefile, install_with_clonefile_each_dir, the clonefile arm of link_cached_package, the .bin symlink gate and the fcopyfile gate of copy_file_with_state. What each gate does when it fires is unchanged.
refusedSyscalls in the harness inserts a dyld interposer (test/fixtures/refuse-syscalls-shim.c) that refuses clonefileat, linkat, symlink in a .bin directory or fcopyfile with a chosen errno, after the path lookup the kernel does first. The hoisted linker copies a package that cannot be cloned. The isolated linker hard-links it and copies it when the hard link is refused too. The copyfile backend reads and writes a file that fcopyfile refuses.
Darwin's EOPNOTSUPP (102) variant is marked deprecated with a note that names bun_errno::is_not_supported. The workspace denies warnings, so a gate that names it alone no longer builds for macOS. On Linux, FreeBSD and Windows the name is an alias and nothing changes. Five places keep the name with an allow: the errno table, the predicate, the two socket gates that want 102 (open(2) and listen(2)), and the rename gate, which is inert on macOS. The manifest save and rename_no_replace already compared both values and now call the predicate.
|
No actionable comments were generated in the recent review. 🎉 ℹ️ Recent review info
WalkthroughThe change adds a shared errno helper for unsupported filesystem operations. Install and file-operation fallbacks use the helper, and macOS tests exercise fallback behavior when selected syscalls are refused. ChangesUnsupported filesystem errors
Suggested reviewers: Priority: ➖ Normal Merge Risk: ⚪ Minimal · up to
|
There was a problem hiding this comment.
Nothing blocking. The comments below are optional suggestions. There is no need to push a fix for them before merging.
Beyond the inline findings, I also checked whether the #[deprecated] on Darwin's EOPNOTSUPP leaves any bare use that would break the macOS build: every remaining E::EOPNOTSUPP / SystemErrno::EOPNOTSUPP in src/ (node_fs.rs, posix_event_loop.rs, sys/copy_file.rs:206, sys/lib.rs:2311) sits inside a linux/android/freebsd cfg block, and the three macOS sites that want 102 carry an allow. is_not_supported also compiles on Linux, FreeBSD and Windows because ENOTSUP/EOPNOTSUPP alias there, so the equality form is not a problem.
Extended reasoning...
The change replaces five install/sys fallback checks that compared against Darwin's EOPNOTSUPP (102) with a shared bun_errno::is_not_supported predicate that also accepts ENOTSUP (45), deprecates the Darwin variant, and adds a dyld-interposer test fixture plus six macOS-only tests. It touches no security-sensitive surface. Inline findings cover the test skip gate, the leaked shim temp dir, and the still-failing exFAT isolated path; the note above records what else was examined and ruled out.
| */ | ||
| export const refusedSyscalls = { | ||
| /** macOS with a C compiler; `skipIf(!refusedSyscalls.available)`. */ | ||
| get available(): boolean { |
There was a problem hiding this comment.
🟡 (optional) On a macOS runner or developer machine without Command Line Tools, the six new tests fail instead of skipping, because /usr/bin/cc exists as Apple's xcode-select stub. available at test/harness.ts:2585 only checks which("cc") || which("clang"), so the describe block runs and compileRefusedSyscallsShim at test/harness.ts:2558 throws in beforeAll, failing every test in the block. Fix: make available prove the compiler works (e.g. probe xcode-select -p or compile a one-line file once and cache the result) so a missing toolchain yields a skip with a reason rather than six failures.
Why this was flagged
Every macOS has /usr/bin/cc and /usr/bin/clang even when Command Line Tools are not installed; they are shims that print xcode-select: note: No developer tools were found and exit non-zero. refusedSyscalls.available at test/harness.ts:2583-2586 returns true from which alone, so describe.skipIf(!refusedSyscalls.available) at test/cli/install/bun-install-backends.test.ts:701 does not skip. beforeAll at test/cli/install/bun-install-backends.test.ts:702-704 awaits compile(), which rejects with refusedSyscalls: compiling the shim failed from test/harness.ts:2569, and all six tests in the block report failures on a machine that merely lacks a toolchain. The sibling Linux helper at test/harness.ts:2538 has the same shape but on Linux which("cc") is false when no compiler is installed, so the stub problem is macOS-specific and new here.
Verification: nit — triggers only on a macOS machine without Command Line Tools, where /usr/bin/cc and /usr/bin/clang are xcode-select stubs that exit non-zero. refusedSyscalls.available at test/harness.ts:2585-2586 is a pure PATH lookup, so the stub satisfies it; beforeAll at :701-703 awaits refusedSyscalls.compile(), failing every test in the block instead of skipping.
| errno if bun_errno::is_not_supported(errno) => { | ||
| InstallMethod::Hardlink | ||
| } |
There was a problem hiding this comment.
🟣 pre-existing, not blocking: pre-existing: macOS users whose cache and project share an exFAT volume still get exit 1 from an isolated install, now at (link) instead of (clonefile). The new arm at src/install/isolated_install/Installer.rs:1796 routes ENOTSUP from clonefileat to InstallMethod::Hardlink, but needs_copy at Installer.rs:3022 only accepts EXDEV | EACCES | EPERM, so linkat's ENOTSUP (45) on exFAT is fatal. Fix: treat bun_errno::is_not_supported(errno) like EXDEV in needs_copy (and its hoisted twin map_linkat_err at src/install/PackageInstall.rs:1905) so the hard-link backend falls through to Copyfile the way the clone backend now does. The PR text defers this to #45761; the base branch fails the same install one step earlier. [also at: src/install/isolated_install/Installer.rs:1800 - pre-existing: macOS users whose cache sits on exFAT still get an isolated bun install that exits 1, now at (link) instead of (clonefile).]
A small fix can ride a push you are already making; otherwise a short reply is enough.
Why this was flagged
Trigger: bun install --linker=isolated on macOS with BUN_INSTALL_CACHE_DIR and the project on one exFAT volume. Path: Installer.rs:1791 cloner.clone() fails with ENOTSUP, Installer.rs:1796 now selects InstallMethod::Hardlink, Hardlinker.rs:236 sys::linkat fails with ENOTSUP, Installer.rs:1832 needs_copy returns false because Installer.rs:3023-3026 only matches EXDEV | EACCES | EPERM, Installer.rs:1836 prints the link error and Installer.rs:1842 breaks with Err. The user gets ENOTSUP: Operation not supported: failed to link package: <pkg> (link) and exit 1. The base branch fails the same install at the clonefile arm with (clonefile), so the outcome is unchanged (pre-existing). The hoisted twin map_linkat_err at PackageInstall.rs:1905-1912 has the same gap for --backend=hardlink. The PR description lists this as a known downside deferred to #45761.
Verification: Trigger: bun install --linker=isolated on macOS with cache and project on one exFAT volume. src/install/isolated_install/Installer.rs:1794-1800 now routes errno if bun_errno::is_not_supported(errno) => InstallMethod::Hardlink; needs_copy(&err) at lines 3022-3027 excludes ENOTSUP, so the code falls to break 'backend Err(err) — install exits 1. On the base commit the same scenario already exited 1.
…46480) ### Problem - Since #44968 (not released), an isolated install that cannot create one directory of one package copies every later package instead of hard-linking it. The failing package prints a wrong error: `ENOENT: No such file or directory: failed to link package: axios@1.7.2 (open)` for a `mkdir` that failed with `EACCES`. - Cause: `needs_copy` (`src/install/isolated_install/Installer.rs:3020`) looks only at the errno. #44968 made `Hardlinker::link` return the error of a `mkdir`, and `EACCES` from it reads as a file system that refuses hard links. ### Fix - `needs_copy` also requires `syscall == Tag::link`. `sys::linkat`, `sys::link_w` and the error that `Hardlinker::link` builds on Windows carry that tag. - Correct because a copy needs the same directory, so the fallback cannot help. - Verified: `test/cli/install/bun-install-hardlink-fallback.test.ts` (one new test: it fails on main and passes on a build from before #44968). Also seven more install suites, listed in Notes. ### Background - The isolated linker links each package with one backend: clonefile, hardlink or copyfile. `supported_backend` holds that choice for the whole install. - Some file systems refuse hard links with `EXDEV`, or with `EACCES` or `EPERM` on FUSE (#36852). The linker then stores copyfile, and every later package is copied. - Considered an error type that marks the link errors of `Hardlinker::link`. It touches every error path of the backends. The tag check fixes the regression alone. ### Downsides - An unreadable directory in a cached package no longer switches the backend either. 1.4.3 and main both switch there. - A `file:` folder whose copy cannot get a new empty directory now fails with the `mkdir` error. 1.4.3 and main exit 0 and leave the directory out. - None found on the success path. `needs_copy` runs only after a link fails. <details><summary>Notes</summary> **Origin.** There is no tracker issue. The case came up during work on how `bun install` reports a directory that it cannot create. **The test.** `axios@1.7.2` of the test registry is installed, then its `package.json` and `dist/node` are removed from the project's store. `<entry>/node_modules` and `axios/dist` lose write permission. The old copy cannot be moved away, so the package is linked into it. `dist` holds only directories, so the `mkdir` of `dist/node` is the first call that fails in it, whatever the order of the directory listing is. The other two packages are not in the cache, so they are linked after a download, when axios has failed. | Build | Error of axios | Exit | The other two packages | |---|---|---|---| | 1.4.3-canary.1 (367d939, before #44968) | `ENOENT: No such file or directory: failed to link package: axios@1.7.2 (link)` | 1 | hard links | | main | `ENOENT: No such file or directory: failed to link package: axios@1.7.2 (open)` | 1 | copies | | this PR | `EACCES: Permission denied: failed to link package: axios@1.7.2 (mkdir)` | 1 | hard links | - Release builds: main fails the test 10 of 10 times, and the build from before #44968 passes it 10 of 10 times. - The test asserts the exit code and the link counts, not the wording of the error. **What the switch costs.** With the global store on a cache that the user cannot write, one store entry that cannot be created plus 120 project-local packages: main copied 30, 2, 2, 36 and 1 of the 120 in five runs. The count depends on whether the walk of the failing package meets a directory before a file. The copies stay: a later install that succeeds leaves them (29, 19 and 1 of 120 in three runs), and `--force` links them again. **The two states under Downsides** (release builds of 1.4.3-canary.1 and main, debug build of this PR): - A cached package with a directory that cannot be read: all three print `EACCES: Permission denied: failed to link package: basic-1@1.0.0 (open)` and exit 1. On 1.4.3 and main the packages linked after it are copies. With this PR they are hard links. - A `file:` folder with a new empty directory, and a copy whose parent directory cannot be written: 1.4.3 and main print nothing and exit 0, and the copy has no such directory. This PR prints the `mkdir` error and exits 1. #44968 made the directory arm return that error on purpose. `needs_copy` then turned it into a copy, and a copy leaves empty directories out. **Other open PRs.** #45761 replaces the call in `link_cached_package` with a new `link_refusal`, which also reads only the errno. It needs the same condition, and the new test fails without it. #45761 and #46022 edit lines next to this change, so the second one to land needs a rebase. **Suites.** Debug build, with `--timeout 60000`: `bun-install-hardlink-fallback` (10 pass), `isolated-install` (283), `bun-install-backends` (28), `bun-install-staging` (51), `isolated-relink` (6), `bun-install-link-path` (49), `bun-patch` (232), `bun-link` (57). **Not run.** Windows and macOS: `cargo check -p bun_install` passes for `x86_64-pc-windows-msvc` and `aarch64-apple-darwin`. On Windows a failed `mkdirat` has the tag `open` (`open_dir_at_windows_nt_path`), so it does not pass the new condition either. The new test is skipped on Windows because it takes a permission away with `chmod`. </details> <!-- robobun:evidence:begin --> --- **no test proof** · iteration 0 · platform-specific test(s) that do not run on this machine, deferring to CI, which covers all platforms: test/cli/install/bun-install-hardlink-fallback.test.ts <!-- robobun:evidence:end -->
The harness helper no longer keeps a directory of its own. The test makes a tempDir, gives it to refusedSyscalls.compile() and removes it in afterAll.
There was a problem hiding this comment.
I re-reviewed the latest push and found no bugs; the shim now builds into a tempDir disposed in afterAll, which settles the cleanup nit. A maintainer should still weigh in, since the PR leaves the hard-link-vs-copy choice for the isolated linker open and the deprecation of Darwin's EOPNOTSUPP is a cross-cutting convention that pins the landing order against three other PRs.
What was reviewed:
- Every remaining bare
E::EOPNOTSUPPin src/ (node_fs.rs, posix_event_loop.rs, copy_file.rs:206, sys/lib.rs:2311) sits under a freebsd/linux/android cfg, so the deprecation cannot break the macOS build; the three#[allow(deprecated)]sites are on a match arm, a let statement and a block, all stable positions. is_not_supportedcompiles on every target:E::ENOTSUP/E::EOPNOTSUPPresolve on Linux, FreeBSD and Windows through the existing associated-const aliases, andbun_sys::E/sys::Errnoare the same type asbun_errno::E.- The bin.rs restructuring is a straight De Morgan of the old
matches!, and the npm.rs/cache_pack.rs rewrites keep the same accepted errno set. - The six tests assert the refusal log, so a successful clonefile on APFS cannot satisfy them without the fallback running.
Extended reasoning...
The diff adds a bun_errno::is_not_supported predicate, marks Darwin's EOPNOTSUPP = 102 variant deprecated, and routes six install-time fallback gates (clonefile, clonefile_each_dir, isolated clonefile, .bin symlink, rename_no_replace, manifest exchange, fcopyfile) through the predicate, with three sites that want the socket value keeping a bare use under an allow. It touches no auth, crypto or input-parsing surface; the behavior change is that macOS installs on HFS+/exFAT now fall back instead of exiting 1. Tests add a dyld interposer fixture compiled with the system C compiler and six concurrent subprocess tests that assert on the refusal log. The source change is small and mechanical, but the author explicitly leaves a design question open (hard link or copy in the isolated linker, with docs depending on the answer) and ties the deprecation commit to the landing order of three sibling PRs, which is a maintainer call rather than a correctness one; no CODEOWNERS entry covers the changed files and no third-party objection is outstanding.
Still open from earlier reviews (2):
- Unresolved: 2 minor or pre-existing.
Land before #45761, #45572, #45929 (Notes).
Problem
bun installexits 1 when the cache and the project share a volume that cannot clone (HFS+, exFAT):Unexpected: failed copying files from cache to destination for package <name>, orENOTSUP: Operation not supported: failed to link package: <pkg> (clonefile)(isolated). 1.3.14 fell back.OPNOTSUPP. The Rust port kept the name, butEOPNOTSUPPis 102 insrc/errno/darwin_errno.rs, so five checks compare 45 with 102.Fix
bun_errno::is_not_supported(E)is true for both values. Five checks call it:install_with_clonefile,install_with_clonefile_each_dir, the isolated clonefile arm, the.binsymlink check, thefcopyfilecheck. Outcomes are unchanged: hoisted copies, isolated hard-links then copies.#[deprecated]on Darwin'sEOPNOTSUPPmakes a bare use fail the macOS build.test/cli/install/bun-install-backends.test.ts(6 tests, dyld interposer), HFS+ and exFAT disk images.Background
cfg).Downsides
(link), until install: fall back to copyfile when linkat fails with EMLINK, ENOTSUP or ENOSYS #45761.node_modulesreach the cache.Notes
Before and after (darwin arm64 and x64, same results on both)
Before is build 128736: main plus tests only, macOS 26.4 arm64 and macOS 14 x64. After is build 129314: commit 23328ad, macOS 15 arm64 and macOS 14 x64.
Cache and project on one HFS+ or exFAT disk image (
hdiutil), one tarball dependency:Unexpected: failed copying files from cache to destination for package baz, exit 1nlink1)ENOTSUP: Operation not supported: failed to link package: baz@./baz-0.0.3.tgz (clonefile), exit 1nlink2)nlink1)ENOTSUP: Operation not supported: failed to link package: baz@./baz.tgz (link), exit 1With the cache on APFS and the project on the image, both linkers exit 0 on main:
clonefileatreports EXDEV first, and that check works.--backend=copyfileis the workaround for 1.4.0 to 1.4.3.test/cli/install/bun-install-backends.test.tson the darwin lanes: 31 pass and 4 fail on main (the 4 tests of the first push). 37 pass, 0 fail and 4 skip on this branch (the skips are the Windows tests).clonefileatcalls of one install, counted with the interposer on APFS (build 129314):The runs on disk images used a temporary probe test. It is not in the branch any more (removed in 2728dab).
Errno of each call (0 is success):
clonefile,clonefileatlinksymlinkrenameatx_np(RENAME_EXCL), free namerenameatx_np(RENAME_EXCL), taken namerenameatx_np(RENAME_SWAP)renameatx_np(RENAME_SWAP), missing targetfcopyfilefrom a pipeWhen the cache is on the project's volume
BUN_INSTALL_CACHE_DIR,--cache-dir,install.cache.dir, orBUN_INSTALL,XDG_CACHE_HOMEorHOMEon that volume.[install.cache] disable = trueor a cache directory that cannot be opened givesnode_modules/.cache. NoHOME,XDG_CACHE_HOMEorBUN_INSTALLgivesnode_modules/.bun-cache(src/install/PackageManager/PackageManagerDirectories.rs).install.globalStorethe isolated linker clones into<cache>/links, so the volume of the cache alone decides.A project on an external drive with the cache in the home directory never failed.
The five gates
install_with_clonefile(src/install/PackageInstall.rs)install_with_clonefile_each_dir(same file)clonefileaton a file is 45 on HFS+ and exFATsrc/install/isolated_install/Installer.rs)fcopyfileincopy_file_with_state(src/sys/copy_file.rs).binsymlink gate (src/install/bin.rs)Where the regression comes from: before the port
bun.sys.Ewas Zig'sstd.posix.E, whose Darwin enum hasOPNOTSUPP = 45and nothing at 102. The port (#30412) madeEthe same type asSystemErrno, whereEOPNOTSUPP = 102, and carried the arms over by name. It corrected the manifest save insrc/install/npm.rsby hand in the same commit.Left alone on purpose
renameat_concurrently_without_fallback(src/sys/lib.rs) still namesEOPNOTSUPP. It is inert on macOS:RENAME_EXCLanswers EEXIST for a taken name before the file system is asked, so ENOTSUP means the target is missing, andRENAME_SWAPthen answers ENOENT. The delete-then-rename path runs in both cases.Bun.writehas its ownfcopyfilegate that falls back on EBADF only (src/runtime/webcore/blob/copy_file.rs). Bun.write(file, Bun.stdin): splice a pipe source into any destination on Linux #36692 has that one.map_linkat_err,needs_copy): install: fall back to copyfile when linkat fails with EMLINK, ENOTSUP or ENOSYS #45761 has them.Landing order
#45572, #45761 and #45929 each rewrite a line that this PR changes and type
EOPNOTSUPPagain there. Their hunks apply on main and not on this branch. With the last commit of this PR, such a line no longer builds for macOS. The resolved lines:errno if bun_errno::is_not_supported(errno) => self.set_backend(InstallMethod::Hardlink),.bingate with install: fall back to copyfile when linkat fails with EMLINK, ENOTSUP or ENOSYS #45761: its hunk adds| sys::Errno::ENOSYSinside thematches!.&& !bun_errno::is_not_supported(errno)stays.link_refusalin install: fall back to copyfile when linkat fails with EMLINK, ENOTSUP or ENOSYS #45761:_ if bun_errno::is_not_supported(errno) => ...fcopyfilegate with sys: four Linux syscall thunks and three path lookups call again after EINTR #45929:Err(err) if bun_errno::is_not_supported(err.get_errno()) => {}Open question: hard link or copy in the isolated linker
1.2.19 to 1.3.14 went from a refused clone to the hardlink backend in the isolated linker (#21122) and to a copy in the hoisted linker. This PR restores both. If a copy is wanted for both linkers, the isolated arm becomes
InstallMethod::Copyfile, the isolated test expectsnlink1, and exFAT then works without #45761. The docs name only the copy fallback. I left them as they are until this is answered. If the hard link stays,docs/pm/isolated-installs.mdx,docs/pm/cli/install.mdxanddocs/pm/global-cache.mdxeach need one sentence.The last commit
#[deprecated]on Darwin'sEOPNOTSUPP = 102makesrustcreject a bare use in code compiled for macOS (the workspace denies warnings).bun run rust:check-allshows it from Linux. Five places keep the name with anallow: the errno table, the predicate, the two socket gates that want 102 (open(2)insrc/dotenv/env_loader.rs,listen(2)insrc/runtime/socket/Listener.rs) and the rename gate. The manifest save andrename_no_replacealready compared both values and now call the predicate, because the guard requires it. Checked by puttingE::EOPNOTSUPPback at thefcopyfilegate: the darwin check fails with the note, the Linux check passes.Measurements
CI artifacts are on a host that the machine that made this PR cannot reach, and it has no macOS SDK. So these are release objects of single crates (
cargo rustc --release --emit=objwith the release flags, no LTO), main against this branch, compared withllvm-sizeand a per-functionllvm-objdumpdiff.bun_install__TEXT,__text: 2,327,924 to 2,328,052 bytes (+128).Task::run(isolated) +22 instructions of 3,331,install_into+5,install_with_clonefile_each_dir+2,link_bin_or_create_shim+1,rename_no_replace-3.bun_sys__TEXT,__text: 71,076 bytes on both.copy_file_with_state+5 instructions.bun_installtext 3,029,269 to 3,029,255 bytes (-14, all inrename_no_replace).bun_sysis identical.is_not_supportedleaves no symbol: it is inlined.clonefileatcalls to its success branch the instructions are the same. So isfcopyfile..bingate's errno check takes 9 instructions for EEXIST on darwin arm64, 7 on main. Linux is identical.Checks
cargo check --workspacefor aarch64-apple-darwin, x86_64-apple-darwin, x86_64-unknown-linux-gnu, x86_64-pc-windows-msvc and x86_64-unknown-freebsd, after the merge of main at 9634520.cargo clippyfor the touched crates on Linux and darwin.bun-install-backends,bun-install-hardlink-fallback,dt-unknown-readdir,bun-link,bun-install-staging,isolated-relink,bun-pm-cache-pack,test/cli/run/env.test.ts.test/fixtures/refuse-syscalls-shim.c) looks the names up before it refuses, as the kernel does. Its logic ran on Linux in a unit harness.Prior work
The predicate and the guard arms come from commit 579f46c on the branch
robobun/0d97fe3b/install-link-or-copy-per-file, which was never opened as a PR.no test proof · iteration 1 · platform-specific test(s) that do not run on this machine, deferring to CI, which covers all platforms: test/cli/install/bun-install-backends.test.ts