Skip to content

glob: return absolute literal patterns from scan()/scanSync() - #34989

Closed
robobun wants to merge 1 commit into
mainfrom
claude/farm/ad110a26/glob-abs-literal-scan
Closed

robobun wants to merge 1 commit into
mainfrom
claude/farm/ad110a26/glob-abs-literal-scan

Conversation

@robobun

@robobun robobun commented Jul 21, 2026 •

Copy link
Copy Markdown
Collaborator

Fixes #16709

Problem

  • new Bun.Glob("/abs/file.js").scanSync() and .scan() return [] for an existing path (absolute pattern, no glob syntax). A regression from Glob support for workspace names in bun install #10462 (1.1.7).
  • Iterator::init (src/glob/GlobWalker.rs:425) reads the errno of a directory open and skips prepare_matched_path. So matched_paths (the scan() result) stays empty.
  • bun run --filter fails (error: Unexpected) for an absolute workspaces entry.

Fix

  • Only Iterator::match_literal sets IterState::Matched: one Accessor::statat (empty handle for the absolute branch), then prepare_matched_path. No accessor changes.
  • Absolute rule: ENOENT and ENOTDIR are no match, onlyFiles applies, a trailing separator needs a directory.
  • The literal tail calls it with its old rule. walk() asserts each path is recorded.
  • Verified: test/js/bun/glob/scan.test.ts (7 tests fail on main). Self-reviewed: 16 concerns raised, 16 addressed.

Background

  • The walker is generic over an Accessor: filesystem, resolver cache (bun run --filter), compiled-executable files.
  • The literal tail: a pattern ending in a plain name (packages/*/package.json) takes one stat per directory, no directory read.
  • Considered a required Accessor::stat (the first version): it broke the build on main's third accessor.

Downsides

  • Two spellings differ, as on main: a.js/ matches a file (glob: keep the trailing separator out of the last pattern component #34200), a literal link through a file throws ENOTDIR. The absolute forms return [].
  • Absolute literal errors report syscall fstatat (was open). An unreadable directory now matches (was EACCES).
  • Cost: +15 instructions per scan (pattern parse: 1,061, was 1,046), +4 per directory opened (293, was 289). Release .text: -5,376 bytes. Measured before the rebase (7c96a12).
Notes

Reproduction

import fs from "node:fs";
const D = fs.mkdtempSync("/tmp/glob-abs-literal-");
fs.writeFileSync(D + "/a.js", "x");
fs.mkdirSync(D + "/dir");
[...new Bun.Glob(D + "/a.js").scanSync()];                    // [] on 1.4.3, [D + "/a.js"] here
[...new Bun.Glob(D + "/dir").scanSync({ onlyFiles: false })]; // [] on 1.4.3, [D + "/dir"] here
[...new Bun.Glob(D + "/*.js").scanSync()];                    // [D + "/a.js"] on both

The regression

#10462 (303f86a, in 1.1.7 and not in 1.1.6) made matched_paths the result of scan() and moved the insert into prepareMatchedPath*. walk() had appended every path the iterator gave it. The two sites that set .matched for an absolute literal pattern never called prepareMatchedPath, so their match was dropped from then on. This is from the history. No old release was run.

What changes

match_literal(Literal::Pattern, ..), for an absolute pattern with no glob syntax:

  • ENOENT or ENOTDIR from the stat is no match. Any other errno is thrown with the path.
  • A trailing separator needs a directory (and onlyFiles: false), the rule match_pattern_file applies to wildcards. The match is returned without the separator. / and C:\ keep theirs. The strip uses the parser's separator test, so a \ in a name on POSIX stays in the name.
  • Without a trailing separator a regular file matches. Anything else that exists matches only with onlyFiles: false.

match_literal(Literal::LastComponent, ..), for the literal tail, keeps what main does: ISREG || !onlyFiles, no trailing separator test, only ENOENT is silent, and a thrown error carries the path from the cwd (packages/a/package.json, from #44968).

A name with a NUL byte is never classified as a literal (SYNTAX_TOKENS), so it is only compared with directory entries and never becomes a C string. a.js\0junk returned itself after a stat of a.js, and <dir>\0junk/*.txt returned a path that was never probed. Both return [] now. A debug build of main stops on the relative form with ZStr::as_cstr: interior NUL would truncate the C view (a debug assertion). No test pins this: #33181 rejects a NUL in the pattern and in cwd with ERR_INVALID_ARG_VALUE, and that PR owns the answer. A NUL in cwd is unchanged here. On Windows the NT name filter now rejects a NUL as well, so such a name does not reach NtQueryDirectoryFile.

Rows (75 rows on the unfixed 1.4.3-canary and on this branch before the rebase, scanSync and scan, Linux)

  • Absolute literal rows: 19 of 41 go from [] to the match. None goes from [] to a throw. Error rows report syscall fstatat where they reported open (stat on Windows).
  • Relative rows: 32 of 34 are unchanged. The other two are the NUL name, in both onlyFiles states.
  • A directory with no read permission (mode 311) named by an absolute literal pattern with onlyFiles: false threw EACCES. It now matches, as the relative form does (run as uid nobody).
  • /dev/null named by an absolute literal pattern matches only with onlyFiles: false.

Measurements (Linux x64, release builds from one build directory that differ only in GlobWalker.rs, on main 7c96a12. Not measured again after the rebase onto e4c638f: the rebase did not change this diff on the measured paths.)

  • Syscalls for an absolute literal scan: file 1, directory 1, missing 1 (was 1 / 2 / 1, by gdb catch syscall). Each is one newfstatat(AT_FDCWD, <pattern>).
  • Other paths: relative literal 3, <D>/*.js 6, <D>/./a.js 5, pkgs/*/package.json 3 per package directory (was 3 / 6 / 5 / 3). The syscall logs of the two builds differ in 0 lines.
  • Instructions for one transition_to_dir_iter_state call, counted with gdb stepi. perf, valgrind, strace and bloaty are not installed in the build container, and it has no network to install them. A call that sets up a directory read and reaches no literal: 289 on main, 293 here (+4). A literal-tail hit for one pkgs/* directory: 1,475 on main, 1,416 here (-59). A relative literal match (Iterator::init): 1,405 on main, 1,337 here.
  • Pattern parse (GlobWalker::init_with_cwd, once per scan): pkgs/*/package.json 1,046 instructions on main, 1,061 here. /tmp/measure/D/a.js: 1,181 on main, 1,194 here.
  • Heap allocations (gdb breakpoints on the mimalloc entry points): literal tail per directory 4 (was 5). Relative literal match 7 (was 8), miss 3 (was 4). Absolute literal match 5, miss 2 (was 3 and 3, and the match was dropped).
  • Release binary: .text -5,376 bytes (size -A), walker symbols -2,310 bytes (nm -S).
  • Structure: sites that put a path into IterState::Matched 1 (was 3). matched_paths.insert sites 3 (was 3). Accessor functions 5 (was 5). Accessor implementations touched: 0.

Not in this PR

Suites run (debug build, Linux x64)

  • test/js/bun/glob/scan.test.ts: 211 pass, 3 skip, 6 fail. The six are the glob.match recursive searches, which time out at 30 s in the build container without this change as well.
  • test/bundler/compile-asset-bunfs.test.ts 9 pass. test/cli/run/filter-workspace.test.ts 126 pass. test/js/bun/shell/bunshell.test.ts 451 pass. test/js/bun/shell/brace.test.ts 47 pass. test/js/node/fs/glob.test.ts 34 pass. test/cli/run/workspaces.test.ts 4 pass. test/cli/install/bun-workspaces.test.ts 288 pass. test/cli/install/bun-workspaces-self-contained.test.ts 25 pass. test/js/bun/glob/match.test.ts: 569 pass, 5 fail in a whole-file run (the dot segments tests start bun lint and time out at 5 s under load). All 8 pass when run alone.
  • Fail on main e4c638f (a build with main's GlobWalker.rs), pass here: seven scan.test.ts tests (the absolute rows, the name table, the bracket comparison, the root, the device node, the absolute symlink rows, the error rows), the absoluteLiteralFile and absoluteLiteralDir rows of compile-asset-bunfs.test.ts, and absolute workspace path without glob syntax in filter-workspace.test.ts (from bun run --filter: accept an absolute workspaces entry without glob syntax #43362, error: Unexpected on main).
  • Pass before and after, to show that the literal tail did not move: literal last component (two tests), the relative symlink rows, the tail's error path (after a wildcard, names the path from the cwd), and A literal name after a wildcard in bunshell.test.ts, which covers the shell's copy of the tail. Main's a failed stat reports the path from cwd skips as root. A probe with its two assertions passes here as uid nobody. No test asserts a.js/ on a regular file for the tail: glob: keep the trailing separator out of the last pattern component #34200 decides that row.
  • cargo check -p bun_glob passes for x86_64-pc-windows-msvc and aarch64-apple-darwin after the rebase (bun run rust:check-all passed for both Windows targets and aarch64-apple-darwin before it).
  • Windows and macOS: the tests of this PR passed on the Windows x64, Windows arm64 and macOS lanes at 5b0e97a and at 171082f, before the rebase. The Windows-only rows are the drive root in both spellings and the rooted pattern with no drive letter.

no test proof · iteration 2 · platform-specific test(s) that do not run on this machine, deferring to CI, which covers all platforms: test/js/bun/shell/bunshell.test.ts, test/js/bun/glob/scan.test.ts, test/cli/run/filter-workspace.test.ts, test/bundler/compile-asset-bunfs.test.ts

@coderabbitai

coderabbitai Bot commented Jul 21, 2026 •

Copy link
Copy Markdown
Contributor

Review in Change Stack →

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info
⚙️ Run configuration
  • Configuration used: Repository: oven-sh/bun/.coderabbit.yaml
  • Review profile: ASSERTIVE
  • Plan: Advanced
  • Run ID: 6382c810-8240-4f40-98e4-b99c20306793
📥 Commits

Reviewing files that changed from the base of the PR and between 171082f and 8f6c026.

📒 Files selected for processing: 4
  • src/glob/GlobWalker.rs
  • test/cli/run/filter-workspace.test.ts
  • test/js/bun/glob/scan.test.ts
  • test/js/bun/shell/bunshell.test.ts

Included review availability: This review used your included allowance. Your plan provides up to 10 included reviews per hour; 5 remain after this review.

Walkthrough

The glob walker now matches absolute literal patterns and final literal components through stat operations. Tests cover match results, path errors, and behavior in bundler, workspace, and shell flows.

Changes

Absolute literal glob matching

Layer / File(s) Summary
Literal matching implementation
src/glob/GlobWalker.rs
Absolute patterns and final literal components use match_literal for stat-based matching. The walker applies directory and files-only checks, handles missing paths and stat errors, and records successful matches.
Scan behavior tests
test/js/bun/glob/scan.test.ts
Tests compare synchronous and asynchronous scan outcomes across absolute literals, trailing separators, symlinks, literal tails, and stat errors.
CLI caller tests
test/bundler/compile-asset-bunfs.test.ts, test/cli/run/filter-workspace.test.ts, test/js/bun/shell/bunshell.test.ts
Tests cover absolute literal matching for embedded assets and workspace paths, plus wildcard expansion in the shell.

Suggested reviewers: dylan-conway, jarred-sumner

Priority: ➖ Normal

Severity of issue fixed: Medium

Merge Risk: ⚪ Minimal · up to 8f6c0

The absolute-literal glob change is mergeable after normal checks; no outstanding issue was established.

Pre-merge checks: Passed 4
Check name Status Explanation
Linked Issues check Passed Issue #16709 requires absolute literal paths to return existing files from Bun.Glob.scan(). Iterator::match_literal now stats absolute paths, applies onlyFiles and trailing-separator rules, and …
Out of Scope Changes check Passed The source changes support the absolute-literal matching fix and preserve the literal-tail behavior. The added tests cover the changed glob rules, platform paths, errors, embedded assets, workspace fi…
Description check Passed The description clearly explains the problem, fix, scope, limitations, and verification results. It does not use the exact template headings, but it provides equivalent information and is substantiall…
Title check Passed The title clearly and concisely describes the main change: returning matches for absolute literal patterns from glob scans.
Full details: Linked Issues check

Explanation

Issue #16709 requires absolute literal paths to return existing files from Bun.Glob.scan(). Iterator::match_literal now stats absolute paths, applies onlyFiles and trailing-separator rules, and records successful matches through prepare_matched_path. The added scan tests cover synchronous and asynchronous scans. The workspace test covers the related absolute-path integration case.

Full details: Out of Scope Changes check

Explanation

The source changes support the absolute-literal matching fix and preserve the literal-tail behavior. The added tests cover the changed glob rules, platform paths, errors, embedded assets, workspace filtering, and related shell expansion. The available change summary shows no unrelated product behavior.

Full details: Description check

Explanation

The description clearly explains the problem, fix, scope, limitations, and verification results. It does not use the exact template headings, but it provides equivalent information and is substantially complete.

  • Autofix · Keep fixing CodeRabbit findings and required CI, and resolving merge conflicts

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

@robobun

robobun commented Jul 21, 2026 •

Copy link
Copy Markdown
Collaborator Author
Updated 7:05 AM PT - Oct 16th, 2026

❌ @robobun, your commit 8f6c026 has 1 failures in Build #126569 (All Failures):


🧪   To try this PR locally:

bunx bun-pr 34989

That installs a local version of the PR into your bun-34989 executable, so you can run:

bun-34989 --bun

@github-actions

Copy link
Copy Markdown
Contributor

Found 1 issue this PR may fix:

  1. Bun Glob does not work with absolute paths #16709 - Reports that new Bun.Glob("/absolute/path").scan() returns empty results for literal absolute paths without wildcards, which is exactly the bug this PR fixes in the GlobWalker fast path

If this is helpful, copy the block below into the PR description to auto-close this issue on merge.

Fixes #16709

🤖 Generated with Claude Code

Comment thread src/glob/GlobWalker.rs Resolved
Comment thread src/glob/GlobWalker.rs Resolved

@claude claude Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The stat() rewrite resolves both earlier 🔴 findings: /etc/passwd/foo now hits stat → ENOTDIR → [], and Windows no longer depends on the libuv-dropped O_DIRECTORY flag. Also checked the trailing-separator-on-a-file case — on POSIX stat("…/file/") returns ENOTDIR so it correctly yields []; the inline nit suggests pinning that with a test.

Extended reasoning...

The follow-up commit c8a61ab replaces the open(O_DIRECTORY) probe with A::stat() and branches on S::ISREG(mode) || !only_files, which addresses both prior blockers (non-existent-prefix ENOTDIR now goes to GetNext; Windows classification no longer relies on a flag libuv discards). The only remaining finding is a test-coverage nit on the ENOTDIR arm — the two negative cases my earlier review named (path.join(file, "nope") and file + path.sep) still aren't asserted, so that clause is unpinned. Not approving on that basis alone, but the substantive correctness concerns from the previous round are resolved.

Comment thread test/js/bun/glob/scan.test.ts Resolved
@robobun

robobun commented Oct 12, 2026 •

Copy link
Copy Markdown
Collaborator Author

This PR is rebased on main (e4c638f), after #44968 and #45043 merged.

  • Iterator::match_literal is the one step that stats, classifies and records a literally named final path. The absolute branch and the literal tail both call it. No accessor changes.
  • Relative patterns keep their results. Only absolute patterns with no glob syntax change.
  • It also fixes bun run --filter for an absolute workspaces entry with no glob syntax (error: Unexpected on main), and it updates the absoluteLiteralFile expectation in test/bundler/compile-asset-bunfs.test.ts.

Reproduction: new Bun.Glob("/abs/path/a.js").scanSync() returns [] on main for a file that exists. Seven new tests in test/js/bun/glob/scan.test.ts fail on a build with main's GlobWalker.rs and pass on this branch.

The tests of this PR passed on the Windows and macOS lanes before the rebase (171082f).

Comment thread src/glob/GlobWalker.rs Resolved
Comment thread src/glob/GlobWalker.rs Resolved

@claude claude Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nothing blocking. The comments below are optional suggestions. There is no need to push a fix for them before merging.

Comment thread test/js/bun/glob/scan.test.ts Resolved

@claude claude Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I re-reviewed after 171082f and found no bugs; the two test rows from my earlier note are gone. Because this rewrites the only path that produces a literal match in the shared glob walker (Bun.Glob, the shell, bun run --filter, compiled executables) with Windows branches I cannot execute here, a human look is still worthwhile.

What was reviewed:

  • The new "statat may receive EMPTY" contract against all three Accessor impls: POSIX fstatat maps an invalid fd to AT_FDCWD, statat_windows gets the early Syscall::stat, and StandaloneAccessor/DirEntryAccessor already pass an absolute path through untouched.
  • fd release in the literal-tail path: close_disallowing_cwd now runs after match_literal on both outcomes; the ? sites inside it (join, prepare_matched_path) cannot return Err in practice, so no new leak path.
  • prepare_matched_path(name, b"") for the absolute branch: join_sep_vec skips empty parts, so no doubled separator; the absolute: true option now routes the literal through the normalizing join like every other match.
  • Trailing-separator strip stops at the root on both platforms (windows_filesystem_root(..).len().max(1) / 1), and the "filesystem root" test pins / and C:\ returning themselves.
Extended reasoning...

The diff rewrites the absolute-literal and literal-tail paths in src/glob/GlobWalker.rs into a single match_literal method (stat-based, ENOENT/ENOTDIR mapping differs per Literal arm), adds NUL to the glob-syntax byte sets, deletes dupe_matched, and adds tests across scan.test.ts, filter-workspace, compile-asset-bunfs and bunshell. It touches no auth, crypto or injection surface; the sensitive part is filesystem probing semantics shared by several Bun subsystems. The bug hunt ran dry with no findings, no third-party objections are outstanding, and the latest commit removed the rows my prior nit flagged. I deferred rather than approved because the change alters user-visible glob semantics in a core walker with cfg-gated Windows code that cannot be exercised in this environment.

`Iterator::init` answered an absolute pattern with no glob syntax from the
errno of a directory open, and set `IterState::Matched` without
`prepare_matched_path`. `Glob.scan()` and `scanSync()` read their result from
`matched_paths`, so the match was lost. The same branch ignored `onlyFiles`,
matched `<file>/child`, and on Windows could not tell a file from a directory.

`Iterator::match_literal` is now the only place that sets
`IterState::Matched`. It stats the name, classifies it, and records it through
`prepare_matched_path`. The absolute branch calls it with
`AccessorHandle::EMPTY` and `Literal::Pattern`: ENOENT and ENOTDIR are no
match, `onlyFiles` applies, and a trailing separator needs a directory.
`statat_windows` stats the path as given for an invalid handle.

The literal tail of `transition_to_dir_iter_state` calls it with
`Literal::LastComponent` and keeps its rule and its error path.

A component with a NUL byte is no longer classified as `Literal`, so it never
becomes a C string that names another path. `walk()` asserts that every path
it is given is in `matched_paths`. `dupe_matched` has no caller left and is
removed.
@robobun
robobun force-pushed the claude/farm/ad110a26/glob-abs-literal-scan branch from 171082f to 8f6c026 Compare October 16, 2026 09:56

@claude claude Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I reviewed this push and didn't find any bugs; it addresses what I raised on the earlier versions (classification now comes from a stat instead of the errno of an O_DIRECTORY open, the ENOTDIR rows are covered by underFile/fileTrailingSepAll/missingParent, and the literal-tail test no longer pins a.js/ matching a file). Because it reworks the one path that all three Accessor implementations share and includes Windows-only code that cannot be built here, a human look would still be worthwhile.

What was reviewed:

  • match_literal(Literal::LastComponent) against the removed literal-tail block: same ISREG || !onlyFiles rule, same ENOENT-only silence, same joined error path; close_disallowing_cwd(fd) now also runs on the join/prepare alloc-error exits, which the old block skipped.
  • A::Handle::EMPTY into statat for each accessor: SyscallAccessor maps an invalid fd to AT_FDCWD (and to Syscall::stat on Windows), DirEntryAccessor and StandaloneAccessor pass an absolute path through untouched.
  • Trailing-separator strip and root guard for /, C:\, a POSIX-style rooted path on Windows, and dir//; join skips the empty dir_path, so the result is the pattern as written minus the separator.
  • NUL in SYNTAX_TOKENS: such a component is never Literal, so dupe_z never builds a C string from it and the debug_assert in match_literal holds.
Extended reasoning...

The change rewrites the two literal fast paths in src/glob/GlobWalker.rs (absolute pattern with no glob syntax, and a literal last component) into one match_literal that stats the path, classifies by S_ISREG/S_ISDIR with onlyFiles and a trailing-separator rule, and records the hit through prepare_matched_path, which is what makes scan()/scanSync() return absolute literal matches; it also adds NUL to the syntax tokens and the Windows NT name filter. It touches no security-sensitive surface beyond path handling, and the NUL change tightens C-string construction. All of my earlier inline findings are addressed in this commit and the added tests pin the absolute rules, the error paths, and the standalone/--filter consumers. Deferring rather than approving because the walker is shared by three accessors across platforms and the Windows-gated branch cannot be type-checked in this environment.

@robobun

robobun commented Nov 17, 2026

Copy link
Copy Markdown
Collaborator Author

Closing as stale: this PR conflicts with main, and its last commit is from 2026-10-16.

The branch stays, so the work is not lost. To pick it up again, rebase the branch on main and reopen this PR.

@robobun robobun closed this Nov 17, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Bun Glob does not work with absolute paths

2 participants