Skip to content

Checkpoint incompatibility storage across solver forks - #91

Merged
charliermarsh merged 4 commits into
mainfrom
charlie/pubgrub-arena-rewind
Oct 21, 2026
Merged

charliermarsh merged 4 commits into
mainfrom
charlie/pubgrub-arena-rewind

Conversation

@charliermarsh

@charliermarsh charliermarsh commented Oct 20, 2026 •

Copy link
Copy Markdown
Member

We reuse one incompatibility arena across depth-first solver forks. We save a checkpoint before branching, then discard each completed branch's additions before starting its sibling. Pending solver states can be cloned without copying the arena's records.

The contradiction cache moves into each partial solution so the incompatibility records remain immutable.

With uv #22784, a large project's 7,777-package resolves improve as follows against uv main (384d963, which includes deferred forks):

Resolve Main peak RSS This PR peak RSS RSS reduction Runtime
Fresh 858.8 MiB 809.1 MiB 5.9% 2.6% faster
Upgrade 1,109.8 MiB 1,058.8 MiB 4.6% 2.3% faster

Six matched runs per variant and resolve across two Linux hosts, using warm offline caches. RSS values are medians; percentage changes use the matched runs.

@codspeed

codspeed Bot commented Oct 20, 2026 •

Copy link
Copy Markdown

Merging this PR will improve performance by 12.55%

1 warning

⚠️ Base and head ran on different runner types

Some benchmarks with significant performance changes were compared across different runner types, so the changes might not come from the code. Run base and head on the same runner type, or use CodSpeed Macro Runners for more stable results.

Learn more about Macro Runners

Open the report in CodSpeed to investigate

⚡ 1 improved benchmark
✅ 6 untouched benchmarks

Performance Changes

Benchmark BASE HEAD Efficiency
⚡ backtracking_disjoint_versions 1.2 s 1.1 s +12.55%

Tip

Curious why performance improved? Comment @codspeedbot explain why performance improved on this PR, or directly use the CodSpeed MCP with your agent.


Comparing charlie/pubgrub-arena-rewind (fd701d9) with main (c930bd7)

Open in CodSpeed

@charliermarsh
charliermarsh force-pushed the charlie/pubgrub-arena-rewind branch from b988f52 to 35b786c Compare October 21, 2026 02:04
@charliermarsh
charliermarsh changed the base branch from charlie/shared-incompatibility-arena-pr to main October 21, 2026 02:05
@charliermarsh
charliermarsh force-pushed the charlie/pubgrub-arena-rewind branch from 35b786c to f0721f6 Compare October 21, 2026 02:08
Keep immutable incompatibility records in one contiguous arena and move the contradiction cache into the partial solution. Expose checked checkpoint transfers for depth-first forks so suspended solver states can be cloned without copying arena records.

Preserve the original version ranges when reporting self-dependencies.
@charliermarsh
charliermarsh force-pushed the charlie/pubgrub-arena-rewind branch from 1ddd00d to fd701d9 Compare October 21, 2026 14:15
@charliermarsh
charliermarsh marked this pull request as ready for review October 21, 2026 16:08
@charliermarsh
charliermarsh merged commit a98a2c1 into main Oct 21, 2026
11 checks passed
charliermarsh added a commit that referenced this pull request Oct 21, 2026
Prepare astral-pubgrub 0.7.1 with checkpoint support from #91.
konsti-openai pushed a commit that referenced this pull request Oct 22, 2026
Sharing arena segments lets uv clone the solver at both fork sites.
The uv changes are confined to the two Git dependency pins and their
lockfile entries.

Implementation diff    Merged checkpoints    Shared segments    Fewer lines*
PubGrub (#91)          +388/-144             +96/-92            65%
uv (#22784)             +43/-15               +4/-6             83%

* Additions plus deletions, excluding the benchmark report and fixtures.

On a frozen 8,670-package workspace with 18 resolution environments,
shared segments saved 182–184 MiB of peak RSS over pre-PR uv. Peak RSS
was within 3 MiB of merged uv. All runs produced byte-identical locks.
The original PR measured a different, 7,777-package snapshot.

Shared segments vs.    Solve      Wall time    Peak RSS     Paired runs
Pre-PR uv              Fresh     -3.4% ± 1.6   -182.1 MiB       3
                       Upgrade   -2.0% ± 0.2   -184.4 MiB       2
Merged checkpoints     Fresh     +1.2% ± 1.7     -2.5 MiB       5
                       Upgrade   +2.9% ± 1.0     +2.7 MiB       4

Full offline resolver benchmarks on two Linux devboxes per case, with
12 paired blocks each. Times are medians. Changes are paired wall-time
estimates, with one standard error in percentage points in both tables.

Workload             Merged checkpoints    Shared segments    Time change
Jupyter                     9.06 ms              8.93 ms      -1.5% ± 1.4
Jupyter universal          49.4 ms              49.9 ms      +0.8% ± 1.1
Airflow                   132.7 ms             132.8 ms      +0.7% ± 1.2
Airflow universal         487.5 ms             495.8 ms      +0.9% ± 0.4

All 17 public uv workloads produced matching outputs. The five uv
backtracking fixtures were within 0.8% of merged uv. In standalone
PubGrub, disjoint backtracking took 2.1% longer and singletons took
6.3% longer. Raw samples and reproduction commands are under
benchmarks/fork-sharing/.
konsti-openai added a commit that referenced this pull request Oct 22, 2026
Cloning the arena shares the incompatibilities recorded so far. When
either fork appends to a shared tail, it starts a segment of its own.
Each partial solution keeps its own contradiction cache because
backtracking can invalidate an entry in one fork while its sibling
still needs it. Both uv fork sites can use an ordinary solver clone.

Implementation diff        Merged checkpoints    Shared segments
PubGrub (PR #91)                  +388/-144              +96/-92
uv (PR #22784)                     +43/-15                +4/-6

Counting additions and deletions, this changes 65% fewer lines in
PubGrub and 83% fewer in uv. The uv diff only updates dependency pins.

On a frozen 8,670-package workspace with 18 resolution environments,
shared segments saved 182–184 MiB of peak RSS over pre-PR uv. All
fresh locks and upgrades produced byte-identical lockfiles. The
original PR measured a different snapshot with 7,777 packages.

Shared segments vs.    Solve      Wall time    Peak RSS     Paired runs
Pre-PR uv              Fresh     -3.4% ± 1.6   -182.1 MiB       3
                       Upgrade   -2.0% ± 0.2   -184.4 MiB       2
Merged checkpoints     Fresh     +1.2% ± 1.7     -2.5 MiB       5
                       Upgrade   +2.9% ± 1.0     +2.7 MiB       4

Full offline uv resolver benchmarks used two Linux devboxes and 12
paired blocks per workload. Times below are medians. Percentage changes
in both tables are paired estimates ± one standard error.

Workload             Merged checkpoints    Shared segments    Time change
Jupyter                     9.06 ms              8.93 ms      -1.5% ± 1.4
Jupyter universal          49.4 ms              49.9 ms      +0.8% ± 1.1
Airflow                   132.7 ms             132.8 ms      +0.7% ± 1.2
Airflow universal         487.5 ms             495.8 ms      +0.9% ± 0.4

All 17 public uv workloads produced matching outputs. The five uv
backtracking fixtures were within 0.8% of merged uv. In standalone
PubGrub, disjoint backtracking took 2.1% longer and singletons took
6.3% longer than with merged checkpoints.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant