Skip to content

bun_core: seed each thread's fast_random() from the process seed plus a thread index - #46905

Open
robobun wants to merge 1 commit into
mainfrom
robobun/37d2ad6b/fast-random-distinct-per-thread
Open

robobun wants to merge 1 commit into
mainfrom
robobun/37d2ad6b/fast-random-distinct-per-thread

Conversation

@robobun

@robobun robobun commented Oct 25, 2026 •

Copy link
Copy Markdown
Collaborator

Not the requested mechanism: this adds a per-thread index to the process seed, not os_entropy per thread. The swap is one line. Please choose (Notes).

Problem

  • bun_core::fast_random() returns the same sequence on every thread. No user-visible symptom is known.
  • Cause: src/bun_core/util.rs:3521 gives every thread's generator the one process seed. The Rust port made Zig's single generator thread-local.
  • Four workarounds exist. On main: LinkJob.random and the lint COUNT suffix. In open PRs: os_entropy at the call site (Native OpenTelemetry tracing (Bun.otel) #39965, Remove libuv on Windows #42819).

Fix

  • Each thread's generator starts from the process seed plus a thread index: init(random_seed().wrapping_add(THREADS.fetch_add(1, SeqCst))).
  • The two workarounds on main are deleted.
  • Verified: test/cli/format/temporary-name.test.ts fails on main and passes here (debug and release). The existing test "two threads link at the same time" fails without the index.
  • Self-reviewed: 5 concerns raised, 4 addressed. Rejected: moving the test into format.test.ts (Notes).

Background

  • fast_random() is a non-cryptographic generator for temporary names, retry jitter and bundler keys.
  • BUN_DEBUG_HASH_RANDOM_SEED pins the process seed in builds with assertions.
  • Both mechanisms give distinct sequences in release builds. The index adds no getrandom call. os_entropy per thread matches Native OpenTelemetry tracing (Bun.otel) #39965 and Remove libuv on Windows #42819, but repeats under the pinned seed.

Downsides

  • bun fmt temporary names and installer .old- names now rely on distinct sequences. A counter and a main-thread draw guaranteed them before.
  • Release linux-x64: .text +512 bytes, stripped binary +0 bytes. One lock xadd runs per drawing thread.
  • The new test runs on Linux only (inotify).
Notes

The choice of mechanism

The request (July) was: seed each thread from os_entropy, and keep the BUN_DEBUG_HASH_RANDOM_SEED override. This PR keeps the one process seed and adds an index that is never reused. In release builds the result is the same: each thread has its own sequence.

index (this PR) os_entropy per thread
os_entropy reads once per process, as on main once per drawing thread
with the seed pinned (builds with assertions) each thread has its own sequence every thread repeats one sequence
the SEED memo in random_seed() stays can go
the request, #39965, #42819, and the os_entropy comment on main ("once per thread") differs matches
  • To switch: make random_seed() read os_entropy on every call and remove THREADS. That is the util.rs hunk of bun_core: seed fast_random() per thread, not per process #35894.
  • After a switch, the second test case goes, because it pins the seed. The two deletions stay correct in release builds. In a build with assertions and a pinned seed, the threads of one LinkBatch would draw the same .old- name again.
  • The request also said that Zig seeded each thread from OS entropy. It did not. fastRandom() before the port had one seed and one generator for the process.

Earlier attempts

The workarounds

Conflicts with open PRs (one hunk each)

PR Where Resolution
#37260 the same initialiser _ => rand::DefaultPrng::init(random_seed().wrapping_add(THREADS.fetch_add(1, O::SeqCst))),
#42819 the comment above os_entropy keep its text and change "once per thread" to "once per process"
#45572 LinkBatch::link passes job.random to move_aside pass bun_core::fast_random()

The test

  • fs.watch (inotify) records the temporary names of one bun fmt run of 64 files. A name is .<16 hex digits of fast_random()>.tmp. The test expects 64 distinct numbers.
  • The first file has 20,000 lines. bun fmt takes the largest file first, so one thread is still formatting it while the other threads write the rest. With 64 one-line files, one thread can write every file on a release build. Then the unfixed build passes: 76 of 200 cases at 1 CPU.
  • The second case pins the seed. It runs only on builds that read the variable (isDebug || isASAN).
  • Linux only, because only inotify is known to report every short-lived file, in order. The install test below covers the index on every platform.

Runs of the file. Every failure is the count assertion.

build CPUs main 179664e this PR
debug 12 40 of 40 cases fail (11 to 32 of 64 distinct) 40 of 40 pass
debug 2 40 of 40 fail (63 of 64) 40 of 40 pass
debug 1 20 of 20 fail (63 of 64) 20 of 20 pass
release 12 100 of 100 fail (16 to 63 of 64) 100 of 100 pass
release 4 100 of 100 fail 100 of 100 pass
release 2 150 of 150 fail 150 of 150 pass
release 1 150 of 150 fail 150 of 150 pass
  • In parallel on this PR: 320 runs with 8 at a time on release and 48 runs with 6 at a time on debug. No failure.
  • Why a new file: bun bd test test/cli/format/format.test.ts does not pass at the default timeout on main. One run had 29 timeouts, and "isn't corrupted when the run reaches it through two hard links" timed out when it ran alone. So a debug build cannot show that file fail on main and pass here. The test fits in describe("a written file") if a maintainer prefers it there.

The installer

  • bun-install-staging.test.ts, "two threads link at the same time" (5 rows, added with LinkJob.random in install: consolidate the open package manager pull requests #44968). Release: 50 of 50 rows pass on main, and 150 of 150 here at 12 CPUs and at 2 CPUs. On a debug build of this PR with util.rs as on main, the rows --force and node_modules has other versions fail in 3 of 3 runs. The packages keep a file of the old copy.
  • A reinstall of 40 packages over a hoisted node_modules, with the seed pinned, on debug builds: main draws all 40 .old- names on one thread. This PR draws them on 7 to 9 threads. All 40 names differ, and no file of the old version stays.

Measurements (linux-x64, 12 CPUs, main 179664e against this PR)

  • fast_random, release (nm, objdump): the draw path has 49 instructions on both builds, instruction for instruction the same, none lock-prefixed. The whole symbol goes from 217 to 220 instructions and from 960 to 977 bytes. It has one lock-prefixed instruction, the fetch_add, which runs once per drawing thread.
  • Size, release (size -A, nm -S, stat): .text +512 bytes. The sizes of .bss and .tbss do not change (fast_random::THREADS +8, temporary_name::COUNT -4). The stripped bun has 107,730,656 bytes on both builds.
  • size_of::<LinkJob>(): 80 to 72 bytes (gdb, debug).
  • os_entropy calls per process, counted as 8-byte getrandom calls by an LD_PRELOAD interposer, 20 runs, debug, main / this PR. bun -e 1: 0 / 0. bun fmt --check: 0 / 0. bun fmt on 240 files, min-median-max: 1-1-4 / 1-1-5. A count above 1 is the first-draw race in random_seed(), which this PR does not change.
  • bun fmt, distinct random parts among N temporary names, 5 runs each. Main, debug: 14 to 27 of 64 and 39 to 48 of 240. Main, debug, seed pinned: 16 to 25 and 40 to 49. Main, release: 20 to 43 and 44 to 72. This PR: N of N in 45 of 45 runs. Each run with the seed pinned to 42 has the first number of that seed (d0764d4f4476689f) exactly once.
  • A BigInt model of DefaultPrng: seeds 42, 43 and 44 start with d0764d4f4476689f, 2b05935e6f17747d and 004ffeb0127bf876, as the binary does. 1,024 thread indexes with 512 draws each give 524,288 distinct numbers. Two neighbouring indexes share no state word. A golden-ratio step would share 3 of 4, because init steps the seed by that constant.

Suites (the src/ diff did not change after these runs)

suite build timeout result
test/cli/format/format.test.ts release 120 s 1664 pass, 0 fail on main and on this PR
test/cli/lint/lint.test.ts release 120 s 714 pass, 0 fail on main and on this PR
test/cli/install/hoist.test.ts debug, this PR 120 s 175 pass, 0 fail
test/cli/install/bun-prune.test.ts debug, this PR 120 s 176 pass, 0 fail
test/cli/install/bun-install-staging.test.ts debug, this PR default 51 pass, 0 fail
test/cli/run/markdown-entrypoint.test.ts debug, this PR default 30 pass, 0 fail
test/internal/source-lints release, this PR 300 s 183 pass, 0 fail
  • At the default timeout, format.test.ts on release had 14 timeouts on main and 19 on this PR, on a loaded 12-CPU machine. No assertion failed.
  • cargo clippy on bun_core, bun_install and bun_lint_driver is clean.

Not changed

  • The first-draw race in random_seed(): several threads can each read OS entropy and keep different seeds. It does no harm here, because the indexes differ. Startup snapshots (2/4): take/restore runtime and Bun.startupSnapshot #37260 rewrites those lines.
  • The canary note in random_seed(), FileSystem::tmpname (its timestamp XOR and its counter come from Zig), and the random parameter of move_aside.

no test proof · iteration 3 · platform-specific test(s) that do not run on this machine, deferring to CI, which covers all platforms: test/cli/format/temporary-name.test.ts

… a thread index

fast_random() keeps one xoshiro256++ generator per thread. It seeded each
generator with the same process seed, so every thread returned the same
sequence. Each thread's generator now starts from the process seed plus a
per-thread index that an atomic counter supplies.

Two call sites worked around the repeated sequence. `bun lint` and `bun fmt`
appended a counter to the names of their temporary files. The installer drew
the `.old-` suffix of a link job on the main thread. Both workarounds are
removed: the temporary name is the random number alone, and LinkBatch::link
draws on the thread that links.

The test watches the temporary names of one `bun fmt` run of 64 files and
expects 64 distinct random parts. A second case pins the seed with
BUN_DEBUG_HASH_RANDOM_SEED in builds that read it.
@coderabbitai

coderabbitai Bot commented Oct 25, 2026

Copy link
Copy Markdown
Contributor

Warning

Review limit reached

You've used all free OSS reviews for now. Wait for the free limit to reset to keep reviewing this public repository.

Next included review available in 1 minute.

Check out review usage here.

View limit details

Limit details: You’ve used all 10 included reviews currently available.

Learn how review limits work.

Review configuration:

Run configuration
Commits

Reviewing files that changed from the base of the PR and between 179664e and f573743.

Files selected for processing: 5
  • src/bun_core/util.rs
  • src/install/PackageInstall.rs
  • src/install/PackageInstaller.rs
  • src/lint/driver/fs.rs
  • test/cli/format/temporary-name.test.ts
  • Autofix · Keep fixing CodeRabbit findings and required CI, and resolving merge conflicts

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

@robobun

robobun commented Oct 25, 2026 •

Copy link
Copy Markdown
Collaborator Author
Updated 12:20 PM PT - Oct 25th, 2026

✅ @robobun, your commit f573743c01dacfbf1844aeb87dab02a9882370ff passed in Build #132695! 🎉


🧪   To try this PR locally:

bunx bun-pr 46905

That installs a local version of the PR into your bun-46905 executable, so you can run:

bun-46905 --bun

@robobun

robobun commented Oct 25, 2026

Copy link
Copy Markdown
Collaborator Author

Status: reproduced on main (179664e), on debug and release builds.

  • bun fmt on 64 files writes 64 temporary names. On main the names have 11 to 63 distinct random parts. With this PR every run has 64.
  • test/cli/format/temporary-name.test.ts is that check.

@claude claude Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code review completed

Nothing new to post: everything this review found is already covered by existing comments on this pull request or didn't merit a separate one.

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants