Repository navigation
Conversation
…-36v6-832x and lifecycle script tests Adds githubTarball() to test/harness.ts. It builds the tarball layout GitHub returns for /repos/<owner>/<repo>/tarball/<ref>. The GHSA-pfwx-36v6-832x and lifecycle script tests serve that tarball from a local server through GITHUB_API_URL. isolated-install.test.ts drops its inline tar writer for the helper.
The dummy registry server now answers /repos/<owner>/<repo>/tarball/<ref> with tarballs registered through setGithubRepository(). The seven tests that add an owner/repo dependency run with GITHUB_API_URL pointed at it.
…un-install.test.ts The owner/repo tests run with GITHUB_API_URL pointed at the dummy registry, which serves stand-ins for mishoo/UglifyJS, dylan-conway/install-test2 and kangax/html-minifier. The tarball URL tests reach a local server through http_proxy, so the specifiers keep the hosts bun classifies them by.
The two github: cases install a stand-in for piuccio/cowsay from a local server through GITHUB_API_URL. They also assert the request paths, so the cached run fails if bunx installs again.
|
Status: ready for review. Supersedes #39115, #35149 and #37919. Reproduced with outbound network removed ( |
|
No actionable comments were generated in the recent review. 🎉 ℹ️ Recent review info⚙️ Run configurationConfiguration used: Path: .coderabbit.yaml Review profile: ASSERTIVE Plan: Essentials Run ID: 📒 Files selected for processing (1)
Included review availability: Your plan provides up to 10 included reviews per hour; 0 remain after this review. WalkthroughThe tests replace external GitHub downloads with local repository archives, redirected API endpoints, and request proxies. They add coverage for integrity handling, cache reuse, tarball redirects, lifecycle scripts, GitHub references, and ChangesGitHub installation tests
Possibly related PRs
Suggested reviewers: Priority: ⬇️ Low Merge Risk: ⚪ Minimal · up to The test changes replace external GitHub fixture downloads with local routing; no concrete merge-blocking risk remains. 🚥 Pre-merge checks | ✅ 4✅ Passed checks (4 passed)
Comment |
There was a problem hiding this comment.
I reviewed this PR and didn't find any bugs. Because it rewrites ~28 install tests and reshapes what several of them assert (https→http specifiers for the tarball-URL cases, html-minifier's 12 transitive npm packages replaced by a single dummy baz, the bunfig cache=false/saveTextLockfile=false write dropped from that test), a human look at whether the coverage each test originally protected is preserved would still be worthwhile.
What was reviewed:
githubTarball()in harness and the/repos/...route in dummy.registry.ts — root-dir entry ordering, path matching, and cleanup indummyAfterAll.- GHSA-pfwx-36v6-832x rewrite — buffered/streaming variant matrix,
test.concurrentisolation (per-testBun.serve({port:0})+ per-test cache dir), request-count and lockfile-entry assertions. urlTarballProxyin bun-install.test.ts —no_proxyset to the registry host so registry traffic bypasses the proxy; response cloning for reusedResponseobjects.- bunx
localGithubserver disposal (server.stop(true)) and cache-hit request-count assertions.
Extended reasoning...
Overview
This is a test-only change across 8 files (~630 added / ~523 removed) that removes the install/bunx suite's dependence on api.github.com and remote tarball hosts. A new githubTarball(rootDir, files) helper in test/harness.ts builds GitHub-shaped gzipped tarballs via Bun.Archive; dummy.registry.ts gains setGithubRepository/setUglifyJsRepository and a /repos/<owner>/<repo>/tarball/<ref> route so tests can point GITHUB_API_URL at the existing dummy server. The GHSA-pfwx-36v6-832x file is fully rewritten (5→10 tests, buffered + streaming extractor variants, exact lockfile-entry and request-path assertions). bun-install.test.ts folds two near-duplicate tarball-URL tests into a for loop and adds a urlTarballProxy that serves URL dependencies through http_proxy while excluding the registry via no_proxy. bunx.test.ts gets a localGithub() per-test server that records requests. isolated-install.test.ts drops its hand-rolled tar writer for the shared helper. bun-add.test.ts and bun-install-lifecycle-scripts.test.ts register fixture repos in beforeAll and swap env for one carrying GITHUB_API_URL.
Security risks
None. No production code is touched; only test fixtures and harness helpers. The GHSA test still exercises the same integrity-check code path (and now covers both extractors), and the tarballs are built and served in-process from local Bun.serve({ port: 0 }) instances. No credentials, no new external endpoints.
Level of scrutiny
Moderate. The mechanical parts (swapping env for githubEnv(), replacing an inline tar writer with the shared helper) are low-risk. The parts that deserve human attention are the places where what a test asserts changed shape: (1) the two "GitHub tarball URL" tests now install http://github.com/... instead of https://github.com/... (necessitated by the proxy approach — CONNECT-tunneled https can't be intercepted), so a reviewer should confirm bun's URL-classification path treats both schemes equivalently for this dependency kind; (2) "should handle GitHub URL with existing lockfile" previously installed html-minifier plus 11 real npm transitive deps and now installs a fixture with one dummy-registry transitive (baz), and its explicit bunfig.toml write (cache=false, saveTextLockfile=false) was removed — worth confirming the context setup already provides equivalent config given the test still asserts on bun.lockb; (3) the GHSA rewrite strengthens assertions substantially but is a full rewrite of a security-adjacent regression test.
Other factors
The bug hunt ran to dry_streak with no findings and no candidates ruled out. No CODEOWNERS entry covers these paths. There are no prior reviews or unresolved threads on the timeline. The PR description is thorough and documents per-file pass counts with and without network, but the size of the assertion reshaping across five heavily-used install test files is enough that I'd rather a human sign off on the coverage trade-offs than auto-approve.
|
@robobun update PR body with timing diff for these test files before/after comparing a recent build from anotehr Pr or main vs this |
|
@Jarred-Sumner Done. The body has a Timing section now, and the per-lane table is in the Notes. Short version:
A side observation from the same logs: whichever test file starts verdaccio first in a shard pays about 10 to 15 s on alpine for that first start, on main too. |
|
are we calling .bytes() repeatedly for the same archive? |
|
@robobun are we calling .bytes repeatedly for the same archive?> |
urlTarballProxy() built its archive inside the fetch handler, once per request, and the two GitHub tarball URL variants and the two bunx github: cases each built the same archive. The proxy now takes bytes, and each of those fixtures is built once and shared.
|
@Jarred-Sumner Yes, in two places. Fixed in 687599a.
The proxy now takes bytes, and both fixtures are built once and shared by their two tests. Every other file already built its tarball once (module scope or It does not move the timing numbers. One build of those small archives takes 0.12 to 0.15 ms on a release build and 1.3 to 1.9 ms on debug + ASAN, and two builds were redundant. The one large archive (600 KiB, GHSA) takes 11 ms on release and 66 ms on ASAN, and was already built once. So this is not the cause of the |
The three URL tarball tests go back to the https:// dependencies and titles they have on main. The install reaches them through a CONNECT proxy that opens every tunnel to one local TLS server, and the tests assert the tunnel targets.
|
This is Claude (an AI assistant) writing on Jarred's behalf. Included in #44968. |
Problem
test/cli/install/bun-install-lifecycle-scripts.test.tsfailed on every retry in build 115920:error: GET https://api.github.com/repos/dylan-conway/lifecycle-install-test/tarball/3ba6af5b64f2d27456e08df21d750072dffd3eee - 504. The same build failedbun-add.test.tsandbun-install.test.tswith the same 504 for other repositories.trustedDependenciesand run lifecycle scripts during installation #7132.Fix
test/harness.tsgetsgithubTarball(rootDir, files). It builds the tarball layout GitHub returns for/repos/<owner>/<repo>/tarball/<ref>.Bun.serveand passes that server asGITHUB_API_URL. The fixtures copy the file lists, package names, bins and short commits of the real repositories, so the assertions stay as they were.https://dependencies and titles. The install reaches them through a CONNECT proxy (https_proxy) that opens every tunnel to one local TLS server.https://(35ab073), every fixture tarball is built once (687599a), and this description is corrected in four places. Not done here and listed in the Notes: thebun createandbun-install-registrycases and the 21git clonecases. Not taken: a split into several PRs, because the three PRs this one supersedes are closed.DNSResolveFailed) and pass on this branch. Debug and release builds on Linux, release build on Windows x64.Timing
bun-add5.8 s to 3.9 s,bun-install8.4 s to 7.6 s, GHSA 0.7 s to 0.2 s with twice the tests. The other three files do not change.isolated-installis 3 to 14 s slower on five lanes. On the two alpine lanes that is the runner's order for modified files, shown in the Notes. On the other three it is not explained. An A/B of the file on Linux and on Windows shows no difference.Background
owner/repo#refdependency. It downloads a tarball from${GITHUB_API_URL}/repos/<owner>/<repo>/tarball/<ref>. The default ishttps://api.github.com(alloc_github_url,src/install/PackageManager/runTasks.rs:1857).<owner>-<repo>-<short sha>. bun reads that name from the first archive entry and records it as the resolved commit.owner/repocases ofbun-install.test.ts, which none of them covered. Draft install: fold the open bun install / pm / pack+publish PRs into one branch (171 PRs) #39403 folds test(install): serve GitHub tarball fixtures locally in the GHSA-pfwx-36v6-832x and lifecycle script tests #39115 and test(install): serve URL tarball dependencies locally in bun-install.test.ts #37919 (and test(bunx): install fixture packages from a local registry instead of real ones #38467, test(install): speed up bun-install.test.ts and tighten its assertions #38464) into one branch, so it overlaps with this PR: the one that merges second drops the shared hunks.Notes
Timing
Method. Each of these files runs alone in its own
bun testprocess in the serial phase of one CI shard. The wall time below is the span of the file's log group in the Buildkite job log: process start, the tests, and the runner's cleanup of the file's TMPDIR. No file was retried in any of these builds. "Before" is three main builds: 115824, 115833 (the commit this branch is based on,7e56b40) and 115994. "After" is this PR's build 115949 (bold). Main builds have no ASAN lane and use other darwin lane names, so those three rows use build 115954 (#42802, same base commit, does not touch these files) as the one baseline.Wall seconds per file, before (three builds) → after:
bun-install-lifecycle-scriptsGHSA-pfwx-36v6-832xbun-addbun-installbunxisolated-installHow to read it:
bun-install-lifecycle-scripts,GHSA-pfwx-36v6-832xandbunxare inside or within 2 s of the spread of the three main builds on every lane. GHSA now runs 10 tests where it ran 5.bun-installis 1.5 to 2.9 s faster on five of the eight lanes, equal on alpine, and 1.9 s slower on windows 2019 x64. 12 of its tests no longer wait for GitHub.bun-addis 2 to 3 s slower on the two alpine lanes and equal or about 1 s faster on the others.isolated-installis outside the spread on five lanes: alpine aarch64 +11.2 s, alpine x64 +13.3 s, windows 2019 x64 +14.3 s, ubuntu x64 +3.8 s, debian x64 +2.7 s. It is inside the spread on debian aarch64, ubuntu aarch64 and windows 11 aarch64. This PR changes one test in that file: it builds the same tarball with the shared helper.The runner's order explains the two alpine lanes.
scripts/runner.node.mjsruns the test files that a PR modifies before the rest of the shard. On alpine x64isolated-install.test.tsis test 12 of its shard on main and test 3 in this build, which makes it the first file in the shard that starts verdaccio. On main that file isbun-add-catalog.test.ts:bun-add-catalog.test.tsisolated-install.test.tsbun-add-catalog)So the first verdaccio start of a shard costs 10 to 15 s there, it lands on whichever file comes first, and the shard total does not change. Alpine aarch64 shows the same (rest of the shard: 124.1, 118.7, 120.9 s on main, 109.1 s here). After the merge the file goes back to position 12.
What this does not explain: on debian x64 and ubuntu x64
isolated-installis already the first verdaccio test of its shard on main (position 6, position 3 here), and the rest of the shard did not get faster. On windows 2019 x64 the rest of the shard varies by 30 s between main builds, so a 14 s shift cannot be seen. I have one build for "after". I could not reproduce a difference outside CI: an A/B of the file on one Linux x64 machine (3.6 s and 3.5 s, median of 3) and on one Windows x64 machine (5.1 s and 5.0 s) shows none.A/B on one machine. Linux x64, release build
1.4.3-canary.1+09bb54630, network available, base7e56b40and this branch in turn, three rounds, whole file, wall seconds (min / median):bun-install-lifecycle-scriptsGHSA-pfwx-36v6-832xbun-addbun-installbunxisolated-installOn the base side of that A/B
should treat non-GitHub http(s) URLs as tarballsfailed in all three rounds: the machine's egress proxy answers 403 for gitpkg-fork.vercel.sh. It passes on this branch.Outage day. Build 115920 (another PR, 2026-09-15 11:16Z, api.github.com answered 504), debian 13 x64: the lifecycle file ran four attempts of 18.0, 3.8, 3.9 and 3.7 s with the runner's waits between them, 59.5 s from the first start to the last failure, and the job went red. The same file takes 11.8 to 14.6 s there in the four builds above.
Tests converted
bun-install-lifecycle-scripts.test.tsgit dependencies also run preprepare, prepare, and postprepare scripts(2, one per spawn mode)dylan-conway/lifecycle-install-test: six scripts that runbun <script>.js, each writes<script>.txt, same as the real repositoryGHSA-pfwx-36v6-832x.test.tsbun-add.test.tsshould add dependency (GitHub),should add aliased dependency (GitHub), the threegit dep ...name cases,should not save git urls twice,should add dependency without duplication (GitHub)mishoo/UglifyJS@e219a9a,dylan-conway/install-test-3v1.0.0 to v1.0.2,liz3/empty-bun-repobun-install.test.tsshould edit package json correctly with git dependencies, the sevenshould handle GitHub URL in dependencies (...)forms,should handle GitHub URL with existing lockfile, the twoGitHub tarball URLcases,should treat non-GitHub http(s) URLs as tarballsmishoo/UglifyJS@e219a9a,dylan-conway/install-test2@5ed489c,kangax/html-minifier@4beb325,cujojs/when,gitpkg-fork.vercel.shbunx.test.tsshould work for github repository,... with committishpiuccio/cowsayisolated-install.test.tsalready served a local tarball. It drops its inline tar writer for the shared helper.Changes to what the tests assert
[pkg, {}, resolved, sha512 of the served bytes], a rejected tarball is not innode_modules, and the request count proves that the cache-hit case downloads nothing.--verboseoutput proves which extractor ran.should handle GitHub URL with existing lockfileinstalled the real html-minifier and its 11 transitive packages from registry.npmjs.org. The stand-in depends onbaz@^0.0.3from the dummy registry, which has a bin likeheanduglify-jsdo. The test now asserts the registry URLs of both installs (baz+ tarball, then the tarball only).[tarball_url, codeload_url]for GitHub (github.com answers with a 302 to codeload.github.com) and[tarball_url]for the gitpkg URL, and the tunnels the proxy opened (github.com:443,codeload.github.com:443,gitpkg-fork.vercel.sh:443). The local certificate is not for those hosts, so these three installs run withNODE_TLS_REJECT_UNAUTHORIZED=0, asbun-create.test.tsandbun-upgrade.test.tsalready do. test(install): serve URL tarball dependencies locally in bun-install.test.ts #37919 had changed the dependencies tohttp://on the grounds thathttps://could not be served locally. That was wrong, and this PR does not carry it over./repos/<owner>/<repo>/tarball/and/tarball/HEAD) and that the cached run makes no request and prints nothing on stderr.bun-addandbun-installcases keep the real repository names (mishoo/UglifyJSand others) so that their output assertions do not change. The price: a case that loses itsGITHUB_API_URLgoes back to api.github.com and still passes. The lifecycle, GHSA and bunx cases use names that do not exist on GitHub, and the GHSA, bunx and URL tarball cases assert the requests they received.gh api repos/<owner>/<repo>/tarball/<ref>on 2026-09-15), so lines like+ uglify@github:mishoo/UglifyJS#e219a9aand the@GH@mishoo-UglifyJS-e219a9a@@@1cache folder are unchanged.Still on the network after this PR
bun-create.test.ts:should not mention cd prompt when created in current directoryand the twoshould create and install github template from ...cases downloaddylan-conway/create-testfrom api.github.com. The two install cases also getjqueryfrom registry.npmjs.org. All three return early when GitHub answers 403, 429 or 5xx, so an outage does not turn them red.bun-install-registry.test.ts:bundledDependencies > git dependenciesruns once per lockfile format. It installsdylan-conway/bundled-install-testfrom api.github.com and throughgit+ssh.git clone(21 tests): 6Git URLtests,should handle modified git resolutions in bun.lock, 8 bitbucket and 4 gitlab cases inbun-install.test.ts, andgit dep without package.json and with default branchandshould handle Git URL in dependencies (SCP-style)inbun-add.test.ts. These need local repositories, not tarballs.should perform bin-linking across multiple dependenciesandshould handle @scoped name that contains tildeinbun-install.test.ts, and most ofbunx.test.ts. test(bunx): install fixture packages from a local registry instead of real ones #38467 converts all ofbunx.test.ts. It contains the twogithub:cases that this PR converts (this PR takes them from it), so the one that merges second drops them.Verification
Outbound network removed means
HTTP_PROXY/HTTPS_PROXYunset in a container whose only other route is loopback.-t "git dependencies also run"git clonecases)git cloneand registry.npmjs.org cases)-t "github repository"The three URL tarball cases (CONNECT proxy to a local TLS server) pass with the network removed on the release and debug builds on Linux, five debug runs in a row, and three runs in a row on Windows x64.
With the network available,
bun bd teston the full files: GHSA 10/10, bun-add 71/71, isolated-install 85/85, bun-install 227 pass and 12 fail, bunx 33 pass and 1 fail, lifecycle 119 pass and 3 fail. The same tests fail on main in this container: the bitbucket and gitlab clones are blocked by its egress,should set "npm_config_user_agent" to bun,ensureTempNodeGypScript worksandnode -p should work in postinstall scriptsneedbun/nodeonPATH, whichbun bd testdoes not provide there.no test proof · iteration 0 · platform-specific test(s) that do not run on this machine, deferring to CI, which covers all platforms: test/cli/install/isolated-install.test.ts, test/cli/install/bunx.test.ts, test/cli/install/bun-install.test.ts, test/cli/install/bun-install-lifecycle-scripts.test.ts, test/cli/install/bun-add.test.ts