Skip to content

test: speed up and strengthen the cc() C_INCLUDE_PATH regression test - #41105

Closed
robobun wants to merge 2 commits into
mainfrom
robobun/cc2e694d/faster-ffi-env-path-test
Closed

robobun wants to merge 2 commits into
mainfrom
robobun/cc2e694d/faster-ffi-env-path-test

Conversation

@robobun

@robobun robobun commented Sep 1, 2026 •

Copy link
Copy Markdown
Collaborator

Problem

  • test/regression/issue/26249.test.ts took 10 s on the debian 13 x64-asan lane (build #108977). Its two tests ran in sequence, each with a child bun start-up and a TinyCC compile under ASAN.
  • The assertions were loose: stderr only had to not contain the header name. Nothing covered LIBRARY_PATH.

Fix

  • The two C_INCLUDE_PATH cases are one test.concurrent.each table with a shared fixture, so the children overlap.
  • The child spawn stays. cc() reads both variables through the bun_core::env_var cache (src/runtime/ffi/ffi_body.rs:752). It fills on the first cc() call and lives for the process, out of reach of process.env in the test runner.
  • Each test compares the whole spawn result with toEqual: exact stdout, stderr: "", exitCode: 0. A new case links a shared library that TinyCC finds only through LIBRARY_PATH. The system C compiler builds it, and the case skips when none is installed.
  • Verified: bun bd test, 3 runs, local debug+ASAN: 2.80 to 2.99 s before (2 tests, 490 + 370 ms in sequence), 2.40 to 2.47 s after (3 tests, slowest 480 ms). A no-op file that imports harness takes 1.90 s. All three tests fail with the variables unset (Notes).

Background

  • cc() in bun:ffi compiles a C file in memory with TinyCC. library: ["name"] is -lname: TinyCC searches its library paths for libname.so (.dylib on macOS) and dlopens it.
  • C_INCLUDE_PATH and LIBRARY_PATH are the standard C compiler search path variables. fix(ffi): respect C_INCLUDE_PATH and LIBRARY_PATH env vars #26250 made cc() honor them (NixOS has no FHS paths).
Notes

Negative probe: a copy of the file with the variables renamed (C_INCLUDE_PATH_DISABLED, LIBRARY_PATH_DISABLED) fails all three tests under bun bd test:

/tmp/ffi-include-path_IwaWkj/test.c:2: error: include file 'myheader.h' not found
(fail) cc() finds headers through C_INCLUDE_PATH (one directory)
/tmp/ffi-include-path_4Rw6V6/test.c:2: error: include file 'header1.h' not found
(fail) cc() finds headers through C_INCLUDE_PATH (two directories)
tcc: error: library 'bunffitest' not found
(fail) cc() finds libraries through LIBRARY_PATH
 0 pass
 3 fail

Wall time of bun bd test including the build check: 3.58 s, 3.59 s, 3.82 s before. 3.14 s, 3.16 s, 3.21 s after.

The file also passes with the release binary (USE_SYSTEM_BUN=1 bun test, 100 ms).

The compiler gate follows test/regression/issue/29585.test.ts: Bun.which("cc") || Bun.which("clang") || Bun.which("gcc"), and skipIf when none is found.

The fixture closes the library with lib.close(). This frees the TinyCC state and the dlopen handle before exit, which keeps the leak check quiet without suppressions (see #38385).

The library is named libbunffitest on purpose. A real library name such as libmagic exists in the system library directory, which TinyCC searches before LIBRARY_PATH.

process.env assignments in bun do not call setenv, and the env_var cache would hold the first value anyway. So an in-process test cannot exercise these variables.


no test proof · iteration 1 · platform-specific test(s) that do not run on this machine, deferring to CI, which covers all platforms: test/regression/issue/26249.test.ts

The two C_INCLUDE_PATH cases ran one after the other. Each paid a
child bun start-up and a TinyCC compile under ASAN. They now run as
one concurrent test.each table with a shared fixture.

The child spawn stays. cc() reads C_INCLUDE_PATH and LIBRARY_PATH
through the bun_core::env_var cache, which fills on the first cc()
call for the life of the process.

Assertions compare the whole spawn result: exact stdout, empty
stderr, exit code 0. The old test only checked that stderr did not
name the header.

A new case links a shared library that TinyCC can only find through
LIBRARY_PATH. It builds the library with the system C compiler and
skips when none is installed.
@github-actions github-actions Bot added the claude label Sep 1, 2026
@robobun

robobun commented Sep 1, 2026

Copy link
Copy Markdown
Collaborator Author

Status: ready for review.

  • Origin: daily slow-test sweep, CI build #108977. The file took 10 s on the debian 13 x64-asan lane.
  • Verified locally with bun bd test test/regression/issue/26249.test.ts (debug+ASAN), 3 runs before and after. The numbers are in the PR body.
  • With C_INCLUDE_PATH and LIBRARY_PATH unset, all three tests fail with the TinyCC "not found" errors shown in the PR body.

@coderabbitai

coderabbitai Bot commented Sep 1, 2026 •

Copy link
Copy Markdown
Contributor

Review Change Stack

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info
⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: ASSERTIVE

Plan: Essentials

Run ID: 71457630-fb42-449a-a4c4-8e5909d253f7

📥 Commits

Reviewing files that changed from the base of the PR and between 561641b and 7c9c417.

📒 Files selected for processing (1)
  • test/regression/issue/26249.test.ts

Included review availability: Your plan provides up to 10 included reviews per hour; 5 remain after this review.


Walkthrough

Changes

FFI environment path regression tests

Layer / File(s) Summary
Fixture helpers and include-path coverage
test/regression/issue/26249.test.ts
Reusable helpers compile and execute C fixtures. Table-driven tests cover one and two C_INCLUDE_PATH directories and assert complete process results.
Library-path loading coverage
test/regression/issue/26249.test.ts
A compiler-dependent test builds a shared library and verifies loading through LIBRARY_PATH. Windows and unavailable-compiler cases are skipped.

Merge Risk: ⚪ Minimal · up to 7c9c4

This localized test-only change speeds the regression test and adds coverage for header and library search paths with exact result assertions; no actionable merge-blocking risk remains after normal checks and review.

🚥 Pre-merge checks | ✅ 4
✅ Passed checks (4 passed)
Check name Status Explanation
Linked Issues check ✅ Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check ✅ Passed Check skipped because no linked issues were found for this pull request.
Title check ✅ Passed The title clearly describes the main change: improving the speed and coverage of the C_INCLUDE_PATH regression test. It is concise and related to the changeset, although it does not mention the added …
Description check ✅ Passed The description explains the problem, implementation, verification results, background, and test limitations. It does not use the exact template headings, but it provides the required information and …
Full details: Title check

Explanation

The title clearly describes the main change: improving the speed and coverage of the C_INCLUDE_PATH regression test. It is concise and related to the changeset, although it does not mention the added LIBRARY_PATH coverage.

Full details: Description check

Explanation

The description explains the problem, implementation, verification results, background, and test limitations. It does not use the exact template headings, but it provides the required information and is sufficiently complete.


Comment @coderabbitai help to get the list of available commands.

@claude claude Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM — test-only refactor that tightens assertions and adds LIBRARY_PATH coverage.

What was reviewed:

  • Harness conventions: tempDir + using, bunExe/bunEnv spread, await using on spawns, concurrent pipe drain, test.concurrent.skipIf().each() (matches existing usage in sql-connection-socket-uaf.test.ts).
  • Assertion shape: combined toEqual({stdout, stderr, exitCode}) per REVIEW.md; exact stderr "" replaces the loose .not.toContain checks.
  • New LIBRARY_PATH case: skipIf(isWindows || !systemCC) gate, .dylib vs .so branch, -shared -fPIC build with stderr surfaced on failure; systemCC! is safe under the skip guard.
Extended reasoning...

Overview

This PR touches a single file, test/regression/issue/26249.test.ts, which covers bun:ffi's cc() honoring C_INCLUDE_PATH and LIBRARY_PATH. The two near-duplicate C_INCLUDE_PATH tests are collapsed into a table-driven test.concurrent.skipIf(isWindows).each(includeCases) with shared fixtureJs() and runFixture() helpers, and a third test is added that builds a tiny shared library with the system C compiler and verifies TinyCC finds it via LIBRARY_PATH. No runtime source is touched.

Security risks

None. The change is confined to the test suite; it spawns local subprocesses in a tempDir, contacts no network, and reads no untrusted input. The system-compiler invocation is a fixed argv on a file the test itself wrote.

Level of scrutiny

Low-to-moderate. It is a mechanical test refactor plus one new coverage case, following documented harness conventions exactly (tempDir with using, bunExe/bunEnv, await using proc, Promise.all pipe drain, combined toEqual result assertion, test.concurrent for independent subprocess tests). The test.concurrent.skipIf(...).each(...) chaining already appears elsewhere in the suite, and DirectoryTree is an exported harness type. No CODEOWNERS entry covers this path.

Other factors

The header comment justifying the child-process spawn (process-wide bun_core::env_var cache) is exactly the kind of non-obvious rationale CLAUDE.md asks to keep. The new LIBRARY_PATH test is properly gated (skipIf(isWindows || !systemCC)), picks .dylib on macOS and .so otherwise, surfaces compiler stderr on build failure, and uses a unique library name to avoid colliding with system libs. The PR description includes a negative probe (renamed env vars → all three tests fail) and before/after timings. Bug hunt exited on dry_streak with no findings and no outstanding reviewer objections.

@robobun

robobun commented Nov 17, 2026

Copy link
Copy Markdown
Collaborator Author

Folded into #49196, which combines the open test-only PRs into one. The commit f638f8327 in that PR is this change, unchanged.

Closing this one in favor of #49196.

@robobun robobun closed this Nov 17, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant