Skip to content

test(install): serve GitHub tarballs from a local server in the install tests - #42800

Closed
robobun wants to merge 6 commits into
mainfrom
robobun/b845a64b/hermetic-github-install-tests
Closed

robobun wants to merge 6 commits into
mainfrom
robobun/b845a64b/hermetic-github-install-tests

Conversation

@robobun

@robobun robobun commented Sep 15, 2026 •

Copy link
Copy Markdown
Collaborator

Problem

  • test/cli/install/bun-install-lifecycle-scripts.test.ts failed on every retry in build 115920: error: GET https://api.github.com/repos/dylan-conway/lifecycle-install-test/tarball/3ba6af5b64f2d27456e08df21d750072dffd3eee - 504. The same build failed bun-add.test.ts and bun-install.test.ts with the same 504 for other repositories.
  • 28 tests in five install test files download a real repository from api.github.com or github.com. A GitHub 5xx window fails them on every open PR. There is no culprit commit: the lifecycle cases have done this since add default trustedDependencies and run lifecycle scripts during installation #7132.

Fix

  • test/harness.ts gets githubTarball(rootDir, files). It builds the tarball layout GitHub returns for /repos/<owner>/<repo>/tarball/<ref>.
  • Each test serves such a tarball from a local Bun.serve and passes that server as GITHUB_API_URL. The fixtures copy the file lists, package names, bins and short commits of the real repositories, so the assertions stay as they were.
  • The three URL tarball cases keep their https:// dependencies and titles. The install reaches them through a CONNECT proxy (https_proxy) that opens every tunnel to one local TLS server.
  • Self-reviewed: nine concerns, six addressed. The URL tarball cases keep https:// (35ab073), every fixture tarball is built once (687599a), and this description is corrected in four places. Not done here and listed in the Notes: the bun create and bun-install-registry cases and the 21 git clone cases. Not taken: a split into several PRs, because the three PRs this one supersedes are closed.
  • Verified with outbound network removed: the 28 tests fail on main (DNSResolveFailed) and pass on this branch. Debug and release builds on Linux, release build on Windows x64.

Timing

  • Close to neutral on a day when GitHub is healthy. A/B on one machine (same build, base and branch in turn, median of 3): bun-add 5.8 s to 3.9 s, bun-install 8.4 s to 7.6 s, GHSA 0.7 s to 0.2 s with twice the tests. The other three files do not change.
  • CI (build 115949 against main builds 115824, 115833 and 115994) agrees, except that isolated-install is 3 to 14 s slower on five lanes. On the two alpine lanes that is the runner's order for modified files, shown in the Notes. On the other three it is not explained. An A/B of the file on Linux and on Windows shows no difference.
  • On an outage day: build 115920 spent 59.5 s on four failed attempts of the lifecycle file on debian 13 x64. Its normal time there is 12 to 15 s.

Background

Notes

Timing

Method. Each of these files runs alone in its own bun test process in the serial phase of one CI shard. The wall time below is the span of the file's log group in the Buildkite job log: process start, the tests, and the runner's cleanup of the file's TMPDIR. No file was retried in any of these builds. "Before" is three main builds: 115824, 115833 (the commit this branch is based on, 7e56b40) and 115994. "After" is this PR's build 115949 (bold). Main builds have no ASAN lane and use other darwin lane names, so those three rows use build 115954 (#42802, same base commit, does not touch these files) as the one baseline.

Wall seconds per file, before (three builds) → after:

Lane bun-install-lifecycle-scripts GHSA-pfwx-36v6-832x bun-add bun-install bunx isolated-install
alpine 3.23 aarch64 8.3 / 8.4 / 9.9 → 8.4 0.5 / 0.2 / 0.4 → 0.3 5.2 / 5.1 / 4.2 → 7.3 20.2 / 21.7 / 24.3 → 19.3 26.3 / 26.0 / 23.7 → 28.0 5.8 / 2.2 / 1.8 → 17.0
alpine 3.23 x64 9.8 / 6.9 / 8.7 → 10.2 0.5 / 0.2 / 0.3 → 0.2 4.2 / 4.3 / 5.0 → 8.0 25.9 / 24.5 / 25.2 → 25.7 23.9 / 26.7 / 22.6 → 28.0 5.6 / 4.3 / 2.8 → 18.9
debian 13 aarch64 13.9 / 12.0 / 13.2 → 13.5 0.2 / 0.2 / 0.2 → 0.1 3.7 / 3.8 / 4.2 → 2.5 52.6 / 51.6 / 52.5 → 49.9 2.9 / 2.9 / 3.2 → 3.2 5.5 / 5.6 / 6.5 → 6.2
debian 13 x64 14.2 / 12.1 / 11.8 → 14.6 0.2 / 0.2 / 0.2 → 0.1 3.9 / 3.9 / 4.0 → 4.2 51.6 / 52.0 / 52.5 → 49.8 3.6 / 3.2 / 3.9 → 3.4 7.5 / 10.0 / 7.3 → 12.7
ubuntu 25.04 aarch64 11.7 / 7.3 / 11.5 → 11.5 0.2 / 0.2 / 0.1 → 0.1 3.4 / 3.0 / 3.4 → 2.6 9.3 / 12.0 / 9.8 → 7.8 3.0 / 7.9 / 3.2 → 3.2 8.2 / 7.2 / 6.8 → 9.1
ubuntu 25.04 x64 14.4 / 12.0 / 12.7 → 14.6 0.2 / 0.2 / 0.2 → 0.1 4.1 / 4.2 / 4.3 → 3.2 13.2 / 15.1 / 12.7 → 10.9 3.4 / 3.8 / 4.1 → 3.3 7.2 / 7.2 / 7.2 → 11.1
windows 11 aarch64 17.8 / 21.0 / 19.3 → 19.2 0.3 / 0.8 / 0.7 → 0.8 10.6 / 12.4 / 24.2 → 10.8 37.8 / 38.0 / 41.2 → 34.9 30.9 / 30.1 / 33.9 → 31.9 15.3 / 13.0 / 14.8 → 15.0
windows 2019 x64 14.2 / 13.2 / 12.2 → 13.5 0.3 / 0.2 / 0.3 → 0.3 6.9 / 6.7 / 10.5 → 8.5 32.7 / 31.8 / 31.6 → 34.6 11.0 / 11.0 / 13.5 → 12.8 6.3 / 5.2 / 5.2 → 20.7
debian 13 x64-asan (one baseline) 36.7 → 47.3 0.7 → 5.3 13.0 → 14.2 69.1 → 70.2 10.9 → 24.2 21.5 → 36.4
darwin any aarch64 (one baseline) 2.5 → 3.0 0.9 → 0.1 6.2 → 14.5 8.1 → 6.1 7.1 → 8.1 2.7 → 3.0
darwin any x64 (one baseline) 4.2 → 10.1 3.5 → 0.3 15.0 → 9.5 19.0 → 17.7 17.8 → 17.1 3.9 → 3.7

How to read it:

  • bun-install-lifecycle-scripts, GHSA-pfwx-36v6-832x and bunx are inside or within 2 s of the spread of the three main builds on every lane. GHSA now runs 10 tests where it ran 5.
  • bun-install is 1.5 to 2.9 s faster on five of the eight lanes, equal on alpine, and 1.9 s slower on windows 2019 x64. 12 of its tests no longer wait for GitHub.
  • bun-add is 2 to 3 s slower on the two alpine lanes and equal or about 1 s faster on the others.
  • The ASAN and darwin rows have one baseline build each, so their spread is not known. Every one of the six files also moves to the front of its shard there, as described below.
  • isolated-install is outside the spread on five lanes: alpine aarch64 +11.2 s, alpine x64 +13.3 s, windows 2019 x64 +14.3 s, ubuntu x64 +3.8 s, debian x64 +2.7 s. It is inside the spread on debian aarch64, ubuntu aarch64 and windows 11 aarch64. This PR changes one test in that file: it builds the same tarball with the shared helper.

The runner's order explains the two alpine lanes. scripts/runner.node.mjs runs the test files that a PR modifies before the rest of the shard. On alpine x64 isolated-install.test.ts is test 12 of its shard on main and test 3 in this build, which makes it the first file in the shard that starts verdaccio. On main that file is bun-add-catalog.test.ts:

alpine 3.23 x64, same shard 115824 115833 115994 115949 (this PR)
bun-add-catalog.test.ts 14.1 s 14.2 s 20.2 s 4.1 s
isolated-install.test.ts 5.6 s 4.3 s 2.8 s 18.9 s
all serial tests of the shard except the six files of this PR (includes bun-add-catalog) 126.5 s 124.5 s 134.3 s 116.9 s

So the first verdaccio start of a shard costs 10 to 15 s there, it lands on whichever file comes first, and the shard total does not change. Alpine aarch64 shows the same (rest of the shard: 124.1, 118.7, 120.9 s on main, 109.1 s here). After the merge the file goes back to position 12.

What this does not explain: on debian x64 and ubuntu x64 isolated-install is already the first verdaccio test of its shard on main (position 6, position 3 here), and the rest of the shard did not get faster. On windows 2019 x64 the rest of the shard varies by 30 s between main builds, so a 14 s shift cannot be seen. I have one build for "after". I could not reproduce a difference outside CI: an A/B of the file on one Linux x64 machine (3.6 s and 3.5 s, median of 3) and on one Windows x64 machine (5.1 s and 5.0 s) shows none.

A/B on one machine. Linux x64, release build 1.4.3-canary.1+09bb54630, network available, base 7e56b40 and this branch in turn, three rounds, whole file, wall seconds (min / median):

File Base This branch
bun-install-lifecycle-scripts 5.4 / 5.8 5.6 / 5.7
GHSA-pfwx-36v6-832x 0.6 / 0.7 (5 tests) 0.1 / 0.2 (10 tests)
bun-add 5.7 / 5.8 3.6 / 3.9
bun-install 7.6 / 8.4 7.3 / 7.6
bunx 6.0 / 6.1 5.9 / 6.0
isolated-install 3.3 / 3.6 3.3 / 3.5

On the base side of that A/B should treat non-GitHub http(s) URLs as tarballs failed in all three rounds: the machine's egress proxy answers 403 for gitpkg-fork.vercel.sh. It passes on this branch.

Outage day. Build 115920 (another PR, 2026-09-15 11:16Z, api.github.com answered 504), debian 13 x64: the lifecycle file ran four attempts of 18.0, 3.8, 3.9 and 3.7 s with the runner's waits between them, 59.5 s from the first start to the last failure, and the job went red. The same file takes 11.8 to 14.6 s there in the four builds above.

Tests converted

File Tests Repository stand-in
bun-install-lifecycle-scripts.test.ts git dependencies also run preprepare, prepare, and postprepare scripts (2, one per spawn mode) dylan-conway/lifecycle-install-test: six scripts that run bun <script>.js, each writes <script>.txt, same as the real repository
GHSA-pfwx-36v6-832x.test.ts all 5, now run against the buffered and the streaming extractor (10) 600 KiB of incompressible filler so the body arrives in more than one chunk
bun-add.test.ts 7: should add dependency (GitHub), should add aliased dependency (GitHub), the three git dep ... name cases, should not save git urls twice, should add dependency without duplication (GitHub) mishoo/UglifyJS@e219a9a, dylan-conway/install-test-3 v1.0.0 to v1.0.2, liz3/empty-bun-repo
bun-install.test.ts 12: should edit package json correctly with git dependencies, the seven should handle GitHub URL in dependencies (...) forms, should handle GitHub URL with existing lockfile, the two GitHub tarball URL cases, should treat non-GitHub http(s) URLs as tarballs mishoo/UglifyJS@e219a9a, dylan-conway/install-test2@5ed489c, kangax/html-minifier@4beb325, cujojs/when, gitpkg-fork.vercel.sh
bunx.test.ts 2: should work for github repository, ... with committish a repository named after its bin, like piuccio/cowsay

isolated-install.test.ts already served a local tarball. It drops its inline tar writer for the shared helper.

Changes to what the tests assert

  • The GHSA cases keep every assertion and add: the lockfile entry equals [pkg, {}, resolved, sha512 of the served bytes], a rejected tarball is not in node_modules, and the request count proves that the cache-hit case downloads nothing. --verbose output proves which extractor ran.
  • should handle GitHub URL with existing lockfile installed the real html-minifier and its 11 transitive packages from registry.npmjs.org. The stand-in depends on baz@^0.0.3 from the dummy registry, which has a bin like he and uglify-js do. The test now asserts the registry URLs of both installs (baz + tarball, then the tarball only).
  • The tarball URL cases assert the exact URLs the local TLS server received, [tarball_url, codeload_url] for GitHub (github.com answers with a 302 to codeload.github.com) and [tarball_url] for the gitpkg URL, and the tunnels the proxy opened (github.com:443, codeload.github.com:443, gitpkg-fork.vercel.sh:443). The local certificate is not for those hosts, so these three installs run with NODE_TLS_REJECT_UNAUTHORIZED=0, as bun-create.test.ts and bun-upgrade.test.ts already do. test(install): serve URL tarball dependencies locally in bun-install.test.ts #37919 had changed the dependencies to http:// on the grounds that https:// could not be served locally. That was wrong, and this PR does not carry it over.
  • The bunx cases assert the request paths (/repos/<owner>/<repo>/tarball/ and /tarball/HEAD) and that the cached run makes no request and prints nothing on stderr.
  • The bun-add and bun-install cases keep the real repository names (mishoo/UglifyJS and others) so that their output assertions do not change. The price: a case that loses its GITHUB_API_URL goes back to api.github.com and still passes. The lifecycle, GHSA and bunx cases use names that do not exist on GitHub, and the GHSA, bunx and URL tarball cases assert the requests they received.
  • The fixture short commits are the real ones (gh api repos/<owner>/<repo>/tarball/<ref> on 2026-09-15), so lines like + uglify@github:mishoo/UglifyJS#e219a9a and the @GH@mishoo-UglifyJS-e219a9a@@@1 cache folder are unchanged.

Still on the network after this PR

  • GitHub tarballs that this PR does not convert:
    • bun-create.test.ts: should not mention cd prompt when created in current directory and the two should create and install github template from ... cases download dylan-conway/create-test from api.github.com. The two install cases also get jquery from registry.npmjs.org. All three return early when GitHub answers 403, 429 or 5xx, so an outage does not turn them red.
    • bun-install-registry.test.ts: bundledDependencies > git dependencies runs once per lockfile format. It installs dylan-conway/bundled-install-test from api.github.com and through git+ssh.
  • git clone (21 tests): 6 Git URL tests, should handle modified git resolutions in bun.lock, 8 bitbucket and 4 gitlab cases in bun-install.test.ts, and git dep without package.json and with default branch and should handle Git URL in dependencies (SCP-style) in bun-add.test.ts. These need local repositories, not tarballs.
  • registry.npmjs.org: should perform bin-linking across multiple dependencies and should handle @scoped name that contains tilde in bun-install.test.ts, and most of bunx.test.ts. test(bunx): install fixture packages from a local registry instead of real ones #38467 converts all of bunx.test.ts. It contains the two github: cases that this PR converts (this PR takes them from it), so the one that merges second drops them.

Verification

Outbound network removed means HTTP_PROXY/HTTPS_PROXY unset in a container whose only other route is loopback.

File main, no network this branch, no network
lifecycle, -t "git dependencies also run" 0/2 2/2
GHSA 0/5 10/10
bun-add 62 pass, 9 fail 69 pass, 2 fail (the two git clone cases)
bun-install 206 pass, 33 fail 218 pass, 21 fail (git clone and registry.npmjs.org cases)
bunx, -t "github repository" 0/2 2/2
isolated-install 85/85 85/85

The three URL tarball cases (CONNECT proxy to a local TLS server) pass with the network removed on the release and debug builds on Linux, five debug runs in a row, and three runs in a row on Windows x64.

With the network available, bun bd test on the full files: GHSA 10/10, bun-add 71/71, isolated-install 85/85, bun-install 227 pass and 12 fail, bunx 33 pass and 1 fail, lifecycle 119 pass and 3 fail. The same tests fail on main in this container: the bitbucket and gitlab clones are blocked by its egress, should set "npm_config_user_agent" to bun, ensureTempNodeGypScript works and node -p should work in postinstall scripts need bun/node on PATH, which bun bd test does not provide there.


no test proof · iteration 0 · platform-specific test(s) that do not run on this machine, deferring to CI, which covers all platforms: test/cli/install/isolated-install.test.ts, test/cli/install/bunx.test.ts, test/cli/install/bun-install.test.ts, test/cli/install/bun-install-lifecycle-scripts.test.ts, test/cli/install/bun-add.test.ts

…-36v6-832x and lifecycle script tests

Adds githubTarball() to test/harness.ts. It builds the tarball layout GitHub
returns for /repos/<owner>/<repo>/tarball/<ref>. The GHSA-pfwx-36v6-832x and
lifecycle script tests serve that tarball from a local server through
GITHUB_API_URL. isolated-install.test.ts drops its inline tar writer for the
helper.
The dummy registry server now answers /repos/<owner>/<repo>/tarball/<ref> with
tarballs registered through setGithubRepository(). The seven tests that add an
owner/repo dependency run with GITHUB_API_URL pointed at it.
…un-install.test.ts

The owner/repo tests run with GITHUB_API_URL pointed at the dummy registry,
which serves stand-ins for mishoo/UglifyJS, dylan-conway/install-test2 and
kangax/html-minifier. The tarball URL tests reach a local server through
http_proxy, so the specifiers keep the hosts bun classifies them by.
The two github: cases install a stand-in for piuccio/cowsay from a local
server through GITHUB_API_URL. They also assert the request paths, so the
cached run fails if bunx installs again.
@robobun

robobun commented Sep 15, 2026

Copy link
Copy Markdown
Collaborator Author

Status: ready for review. Supersedes #39115, #35149 and #37919.

Reproduced with outbound network removed (HTTP_PROXY/HTTPS_PROXY unset, loopback only). On main the two git dependencies also run preprepare, prepare, and postprepare scripts cases fail the assertion from build 115920 (bun-install-lifecycle-scripts.test.ts:1982), with DNSResolveFailed in place of the 504. The other 26 tests that this PR converts fail the same way. On this branch all of them pass under the same conditions (bun bd test and the release build on Linux, the release build on Windows x64).

@coderabbitai

coderabbitai Bot commented Sep 15, 2026 •

Copy link
Copy Markdown
Contributor

Review Change StackReview Change Stack

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info
⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: ASSERTIVE

Plan: Essentials

Run ID: c95bee1f-9bae-44c9-9988-1a34b97f1577

📥 Commits

Reviewing files that changed from the base of the PR and between 687599a and 35ab073.

📒 Files selected for processing (1)
  • test/cli/install/bun-install.test.ts

Included review availability: Your plan provides up to 10 included reviews per hour; 0 remain after this review.


Walkthrough

The tests replace external GitHub downloads with local repository archives, redirected API endpoints, and request proxies. They add coverage for integrity handling, cache reuse, tarball redirects, lifecycle scripts, GitHub references, and bunx execution.

Changes

GitHub installation tests

Layer / File(s) Summary
Shared GitHub fixture infrastructure
test/harness.ts, test/cli/install/dummy.registry.ts
Adds shared GitHub archive generation, repository registration, request serving, and cleanup.
Integrity and cache validation
test/cli/install/GHSA-pfwx-36v6-832x.test.ts
Uses isolated local servers to test integrity persistence, matching and mismatched archives, lockfile migration, extraction, and cache reuse.
GitHub API and tarball routing
test/cli/install/bun-add.test.ts, test/cli/install/bun-install.test.ts
Routes GitHub lookups through local fixtures and proxies tarball requests, redirects, URL forms, and lockfile dependencies.
Execution and lifecycle fixtures
test/cli/install/bun-install-lifecycle-scripts.test.ts, test/cli/install/bunx.test.ts, test/cli/install/isolated-install.test.ts
Uses local GitHub archives for lifecycle-script execution, bunx reference and cache tests, and isolated installation coverage.

Possibly related PRs

  • oven-sh/bun#39115: Replaces live GitHub tarball downloads with local fixtures in overlapping installation tests and adds shared githubTarball support.

Suggested reviewers: jarred-sumner

Priority: ⬇️ Low

Merge Risk: ⚪ Minimal · up to 35ab0

The test changes replace external GitHub fixture downloads with local routing; no concrete merge-blocking risk remains.

🚥 Pre-merge checks | ✅ 4
✅ Passed checks (4 passed)
Check name Status Explanation
Linked Issues check ✅ Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check ✅ Passed Check skipped because no linked issues were found for this pull request.
Title check ✅ Passed The title clearly and concisely describes the main change: replacing external GitHub tarball access with local-server fixtures in install tests.
Description check ✅ Passed The description is detailed and covers the problem, fix, scope, verification, performance impact, and remaining network-dependent tests. It does not use the template headings exactly, but it provides …

Comment @coderabbitai help to get the list of available commands.

@claude claude Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I reviewed this PR and didn't find any bugs. Because it rewrites ~28 install tests and reshapes what several of them assert (https→http specifiers for the tarball-URL cases, html-minifier's 12 transitive npm packages replaced by a single dummy baz, the bunfig cache=false/saveTextLockfile=false write dropped from that test), a human look at whether the coverage each test originally protected is preserved would still be worthwhile.

What was reviewed:

  • githubTarball() in harness and the /repos/... route in dummy.registry.ts — root-dir entry ordering, path matching, and cleanup in dummyAfterAll.
  • GHSA-pfwx-36v6-832x rewrite — buffered/streaming variant matrix, test.concurrent isolation (per-test Bun.serve({port:0}) + per-test cache dir), request-count and lockfile-entry assertions.
  • urlTarballProxy in bun-install.test.ts — no_proxy set to the registry host so registry traffic bypasses the proxy; response cloning for reused Response objects.
  • bunx localGithub server disposal (server.stop(true)) and cache-hit request-count assertions.
Extended reasoning...

Overview

This is a test-only change across 8 files (~630 added / ~523 removed) that removes the install/bunx suite's dependence on api.github.com and remote tarball hosts. A new githubTarball(rootDir, files) helper in test/harness.ts builds GitHub-shaped gzipped tarballs via Bun.Archive; dummy.registry.ts gains setGithubRepository/setUglifyJsRepository and a /repos/<owner>/<repo>/tarball/<ref> route so tests can point GITHUB_API_URL at the existing dummy server. The GHSA-pfwx-36v6-832x file is fully rewritten (5→10 tests, buffered + streaming extractor variants, exact lockfile-entry and request-path assertions). bun-install.test.ts folds two near-duplicate tarball-URL tests into a for loop and adds a urlTarballProxy that serves URL dependencies through http_proxy while excluding the registry via no_proxy. bunx.test.ts gets a localGithub() per-test server that records requests. isolated-install.test.ts drops its hand-rolled tar writer for the shared helper. bun-add.test.ts and bun-install-lifecycle-scripts.test.ts register fixture repos in beforeAll and swap env for one carrying GITHUB_API_URL.

Security risks

None. No production code is touched; only test fixtures and harness helpers. The GHSA test still exercises the same integrity-check code path (and now covers both extractors), and the tarballs are built and served in-process from local Bun.serve({ port: 0 }) instances. No credentials, no new external endpoints.

Level of scrutiny

Moderate. The mechanical parts (swapping env for githubEnv(), replacing an inline tar writer with the shared helper) are low-risk. The parts that deserve human attention are the places where what a test asserts changed shape: (1) the two "GitHub tarball URL" tests now install http://github.com/... instead of https://github.com/... (necessitated by the proxy approach — CONNECT-tunneled https can't be intercepted), so a reviewer should confirm bun's URL-classification path treats both schemes equivalently for this dependency kind; (2) "should handle GitHub URL with existing lockfile" previously installed html-minifier plus 11 real npm transitive deps and now installs a fixture with one dummy-registry transitive (baz), and its explicit bunfig.toml write (cache=false, saveTextLockfile=false) was removed — worth confirming the context setup already provides equivalent config given the test still asserts on bun.lockb; (3) the GHSA rewrite strengthens assertions substantially but is a full rewrite of a security-adjacent regression test.

Other factors

The bug hunt ran to dry_streak with no findings and no candidates ruled out. No CODEOWNERS entry covers these paths. There are no prior reviews or unresolved threads on the timeline. The PR description is thorough and documents per-file pass counts with and without network, but the size of the assertion reshaping across five heavily-used install test files is enough that I'd rather a human sign off on the coverage trade-offs than auto-approve.

@Jarred-Sumner

Copy link
Copy Markdown
Collaborator

@robobun update PR body with timing diff for these test files before/after comparing a recent build from anotehr Pr or main vs this

@robobun

robobun commented Sep 15, 2026

Copy link
Copy Markdown
Collaborator Author

@Jarred-Sumner Done. The body has a Timing section now, and the per-lane table is in the Notes. Short version:

  • Before: main builds 115824, 115833 (the commit this branch is based on) and 115994. After: this PR's build 115949. No file was retried in any of them.
  • Speed is close to neutral when GitHub is healthy. bun-install is 1.5 to 2.9 s faster on five of eight lanes. The lifecycle, GHSA and bunx files are inside the spread of the three main builds. A/B on one machine (same build, base and branch in turn): bun-add 5.8 s to 3.9 s, GHSA 0.7 s to 0.2 s with twice the tests.
  • One file looks worse in CI and I cannot fully explain it. isolated-install is 3 to 14 s slower on five lanes in this build. On the two alpine lanes the cause is the runner: it starts the files a PR modifies first, and the first verdaccio start of a shard costs 10 to 15 s. That cost moved from bun-add-catalog.test.ts (14.2 s to 4.1 s) to this file, and the shard total did not change. On debian x64, ubuntu x64 and windows 2019 x64 that does not account for the rise. An A/B of the file on a Linux machine and on a Windows machine shows no difference (3.6 s and 3.5 s, 5.1 s and 5.0 s), and the PR changes one fixture in that file. I have one "after" build, so I would not read more into it than that.
  • Where the time goes on a bad day: in build 115920 the lifecycle file took 59.5 s over four failed attempts on debian 13 x64. Its normal time there is 12 to 15 s.

A side observation from the same logs: whichever test file starts verdaccio first in a shard pays about 10 to 15 s on alpine for that first start, on main too.

@Jarred-Sumner

Copy link
Copy Markdown
Collaborator

are we calling .bytes() repeatedly for the same archive?

@Jarred-Sumner

Copy link
Copy Markdown
Collaborator

@robobun are we calling .bytes repeatedly for the same archive?>

urlTarballProxy() built its archive inside the fetch handler, once per request,
and the two GitHub tarball URL variants and the two bunx github: cases each
built the same archive. The proxy now takes bytes, and each of those fixtures
is built once and shared.
@robobun

robobun commented Sep 15, 2026

Copy link
Copy Markdown
Collaborator Author

@Jarred-Sumner Yes, in two places. Fixed in 687599a.

  • urlTarballProxy() in bun-install.test.ts built its archive inside the fetch handler, so once per request. Each test makes one request for it today, but the two GitHub tarball URL variants also each built the same cujojs/when archive.
  • The two github: cases in bunx.test.ts each built the same cowsay archive.

The proxy now takes bytes, and both fixtures are built once and shared by their two tests. Every other file already built its tarball once (module scope or beforeAll) and served the bytes from memory: GHSA, lifecycle, dummy.registry.ts for bun-add and bun-install, and isolated-install.

It does not move the timing numbers. One build of those small archives takes 0.12 to 0.15 ms on a release build and 1.3 to 1.9 ms on debug + ASAN, and two builds were redundant. The one large archive (600 KiB, GHSA) takes 11 ms on release and 66 ms on ASAN, and was already built once. So this is not the cause of the isolated-install numbers above.

@claude claude Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code review found no issues

No high-confidence issues detected in this change.

The three URL tarball tests go back to the https:// dependencies and titles they
have on main. The install reaches them through a CONNECT proxy that opens every
tunnel to one local TLS server, and the tests assert the tunnel targets.

@claude claude Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

No new blocking issues. 1 optional suggestion (a nit or a note on pre-existing code) was found and not posted. Nothing in this review needs a push before merging.

@Jarred-Sumner

Copy link
Copy Markdown
Collaborator

This is Claude (an AI assistant) writing on Jarred's behalf.

Included in #44968.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants