Skip to content

Enable BOLT for Linux x86-64 Ruff releases - #27588

Closed
charliermarsh wants to merge 3 commits into
mainfrom
charlie/ruff-bolt-prototype
Closed

charliermarsh wants to merge 3 commits into
mainfrom
charlie/ruff-bolt-prototype

Conversation

@charliermarsh

@charliermarsh charliermarsh commented Aug 7, 2026 •

Copy link
Copy Markdown
Member

Summary

Add post-link LLVM BOLT optimization to the existing Linux x86-64 Ruff release build after compiler PGO and fat LTO. This is stacked on #27574; macOS, Windows, Linux ARM64, and all other targets remain compiler-PGO-only or retain their existing release behavior.

The manylinux build preserves symbols and linker relocations, then instruments the linked executable and trains ruff check and ruff format on the same eight immutable ecosystem projects already used by Ruff and ty PGO. BOLT reorders hot functions and basic blocks using the collected binary profile.

The optimized executable uses --use-old-text --no-huge-pages to reuse the original code section instead of publishing BOLT's much larger default duplicate-text layout. The helper requires in-place rewriting to succeed, preserves GNU stack and RELRO hardening, smoke-tests the result, strips with the Rust toolchain's llvm-strip, and atomically rebuilds wheel RECORD entries. The wheel and standalone archive contain the same optimized binary.

LLVM BOLT 22 and its instrumentation runtime are fetched from the official LLVM repository with pinned SHA-256 digests. The dependency-free helper follows the same PEP 723 script conventions as the compiler-PGO helper.

Linux x86-64 production-artifact measurements

Compared the actual standalone release binaries uploaded by this PR and its immediate PGO-only parent. Both contain identical Ruff 0.16.2 sources, the same Rust toolchain and manylinux build configuration, and the current shared eight-project PGO corpus. Benchmarks covered 5,396 held-out Python and stub files totaling 90.57 MB and used eight physical AMD EPYC cores, three warmup pairs, and 20 alternating baseline/candidate pairs per workload.

Held-out project ruff check ruff format
Django 1.62% faster 0.35% faster
pandas 0.70% faster 1.43% faster
scikit-learn 0.86% faster 1.00% faster
SciPy 1.06% faster 1.77% faster
SymPy 2.13% faster 1.46% faster
Geometric mean 1.28% faster 1.20% faster

Across all ten workloads, BOLT improves wall time by 1.24% and CPU time by 1.91% beyond compiler PGO. Paired bootstrap confidence intervals are 0.36–1.75% for wall time and 1.59–2.03% for CPU time. The BOLT binary won 133 of 200 wall-time comparisons and 184 of 200 CPU-time comparisons.

The effect is smaller than compiler PGO because PGO and fat LTO already optimize the same workload: BOLT's production dynamic statistics show only 0.7% fewer executed instructions, while loads, stores, and function-call counts remain unchanged. BOLT reduces taken conditional branches by 22.5%, but that mostly improves layout rather than eliminating work.

Additional no-build BOLT strategy comparisons did not uncover a better compact layout: hfsort+ produced a byte-for-byte identical executable, profile2 increased compressed size by another 4.10%, disabling function splitting increased executable size by 1.89%, and --lite could not satisfy the required in-place rewrite.

Artifact PGO PGO + BOLT Increase
Stripped executable 26,111,360 bytes 27,397,944 bytes 4.93%
Standalone release .tar.gz 10,573,500 bytes 11,417,642 bytes 7.98%
manylinux wheel 10,981,512 bytes 11,468,195 bytes 4.43%

BOLT successfully rewrites executable code in-place but still appends regenerated exception/unwind metadata and jump-table data: the new .gcc_except_table, .eh_frame_hdr, and relocated .rodata account for nearly all executable growth. The production rewrite has 73 KB of remaining text-section capacity and adds 45 seconds to the release job. It fails if a future binary cannot fit rather than silently using a much larger fallback layout.

ARM64 BOLT remains intentionally disabled: the previous ARM experiment inserted 24,430 branch-range stubs, increased the executable by 11.2%, and improved wall time by only 0.08%.

@astral-sh-bot

astral-sh-bot Bot commented Aug 7, 2026 •

Copy link
Copy Markdown

ruff-ecosystem results

Linter (stable)

✅ ecosystem check detected no linter changes.

Linter (preview)

✅ ecosystem check detected no linter changes.

Formatter (stable)

✅ ecosystem check detected no format changes.

Formatter (preview)

✅ ecosystem check detected no format changes.

@charliermarsh
charliermarsh force-pushed the charlie/ruff-bolt-prototype branch from 453e492 to f83832c Compare August 10, 2026 14:11
@charliermarsh
charliermarsh force-pushed the charlie/ruff-bolt-prototype branch from f83832c to ade5540 Compare August 10, 2026 14:52
@charliermarsh
charliermarsh force-pushed the charlie/ruff-bolt-prototype branch from ade5540 to 3deb129 Compare August 10, 2026 14:55
Base automatically changed from charlie/ruff-pgo-linux-arm64 to main August 10, 2026 15:56
charliermarsh added a commit that referenced this pull request Aug 10, 2026
## Summary

This PR enables PGO for Ruff releases, starting with Linux x86-64.

### Design

The release pipeline is modified as follows:

- We build an instrumented, stripped release binary.
- We run `check` and `format` on a corpus of projects from our ecosystem
reports -- specifically, eight of the pinned ecosystem projects that we
use in the ty CI. (In total, it's 502 Python and stub files.)
- We merge the profiles via `llvm-profdata`.
- We feed the result back into the existing `maturin` build.

### Results

We evaluate performance on a held-out corpus: Prefect, Django, Pandas,
scikit-learn, SciPy, and SymPy.

Results are as follows:

| Held-out project | `ruff check` | `ruff format` |
| --- | ---: | ---: |
| Django | 10.9% faster | 6.6% faster |
| pandas | 16.3% faster | 9.7% faster |
| scikit-learn | 10.8% faster | 7.9% faster |
| SciPy | 14.7% faster | 5.6% faster |
| SymPy | 17.1% faster | 11.4% faster |
| Geometric mean | **14.0% faster** | **8.3% faster** |

Beyond runtime:

- **Binary size decreased by 6.2%** (27.96 MB to 26.23 MB).
- **Release pipeline gets about 2x longer** (non-PGO release build took
7m22s; PGO pipeline took 15m14s (8m35s instrumented training plus 6m39s
optimized wheel).

### Stack

Additional platforms are covered in subsequent PRs in the stacked;
platforms that are lower-priority at at-all difficult to run on natively
are omitted. In the end, I'm targeting Linux x86-64, Linux ARM, macOS
ARM, and Windows x86-64.

I also attempted BOLT in #27588,
but I've decided against pursuing that for now; see the results in that
PR which speak for themselves.

ty and uv will follow the same approach; see the stacks here:

- astral-sh/ty#4213
- astral-sh/uv#21001

See: #7055.
@charliermarsh
charliermarsh force-pushed the charlie/ruff-bolt-prototype branch from 3deb129 to 01e18ea Compare August 10, 2026 15:56
George-Ogden pushed a commit to George-Ogden/ruff that referenced this pull request Aug 16, 2026
## Summary

This PR enables PGO for Ruff releases, starting with Linux x86-64.

### Design

The release pipeline is modified as follows:

- We build an instrumented, stripped release binary.
- We run `check` and `format` on a corpus of projects from our ecosystem
reports -- specifically, eight of the pinned ecosystem projects that we
use in the ty CI. (In total, it's 502 Python and stub files.)
- We merge the profiles via `llvm-profdata`.
- We feed the result back into the existing `maturin` build.

### Results

We evaluate performance on a held-out corpus: Prefect, Django, Pandas,
scikit-learn, SciPy, and SymPy.

Results are as follows:

| Held-out project | `ruff check` | `ruff format` |
| --- | ---: | ---: |
| Django | 10.9% faster | 6.6% faster |
| pandas | 16.3% faster | 9.7% faster |
| scikit-learn | 10.8% faster | 7.9% faster |
| SciPy | 14.7% faster | 5.6% faster |
| SymPy | 17.1% faster | 11.4% faster |
| Geometric mean | **14.0% faster** | **8.3% faster** |

Beyond runtime:

- **Binary size decreased by 6.2%** (27.96 MB to 26.23 MB).
- **Release pipeline gets about 2x longer** (non-PGO release build took
7m22s; PGO pipeline took 15m14s (8m35s instrumented training plus 6m39s
optimized wheel).

### Stack

Additional platforms are covered in subsequent PRs in the stacked;
platforms that are lower-priority at at-all difficult to run on natively
are omitted. In the end, I'm targeting Linux x86-64, Linux ARM, macOS
ARM, and Windows x86-64.

I also attempted BOLT in astral-sh#27588,
but I've decided against pursuing that for now; see the results in that
PR which speak for themselves.

ty and uv will follow the same approach; see the stacks here:

- astral-sh/ty#4213
- astral-sh/uv#21001

See: astral-sh#7055.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant