Repository navigation
Checkpoint incompatibility storage across solver forks - #91
Merged
Merged
Conversation
Merging this PR will improve performance by 12.55%1 warning
Performance Changes
Tip Curious why performance improved? Comment Comparing |
charliermarsh
force-pushed
the
charlie/pubgrub-arena-rewind
branch
from
October 21, 2026 02:04
b988f52 to
35b786c
Compare
charliermarsh
changed the base branch from
charlie/shared-incompatibility-arena-pr
to
main
October 21, 2026 02:05
charliermarsh
force-pushed
the
charlie/pubgrub-arena-rewind
branch
from
October 21, 2026 02:08
35b786c to
f0721f6
Compare
Keep immutable incompatibility records in one contiguous arena and move the contradiction cache into the partial solution. Expose checked checkpoint transfers for depth-first forks so suspended solver states can be cloned without copying arena records. Preserve the original version ranges when reporting self-dependencies.
charliermarsh
force-pushed
the
charlie/pubgrub-arena-rewind
branch
from
October 21, 2026 14:15
1ddd00d to
fd701d9
Compare
This was referenced Oct 21, 2026
charliermarsh
marked this pull request as ready for review
October 21, 2026 16:08
charliermarsh
added a commit
that referenced
this pull request
Oct 21, 2026
Prepare astral-pubgrub 0.7.1 with checkpoint support from #91.
konsti-openai
pushed a commit
that referenced
this pull request
Oct 22, 2026
Sharing arena segments lets uv clone the solver at both fork sites. The uv changes are confined to the two Git dependency pins and their lockfile entries. Implementation diff Merged checkpoints Shared segments Fewer lines* PubGrub (#91) +388/-144 +96/-92 65% uv (#22784) +43/-15 +4/-6 83% * Additions plus deletions, excluding the benchmark report and fixtures. On a frozen 8,670-package workspace with 18 resolution environments, shared segments saved 182–184 MiB of peak RSS over pre-PR uv. Peak RSS was within 3 MiB of merged uv. All runs produced byte-identical locks. The original PR measured a different, 7,777-package snapshot. Shared segments vs. Solve Wall time Peak RSS Paired runs Pre-PR uv Fresh -3.4% ± 1.6 -182.1 MiB 3 Upgrade -2.0% ± 0.2 -184.4 MiB 2 Merged checkpoints Fresh +1.2% ± 1.7 -2.5 MiB 5 Upgrade +2.9% ± 1.0 +2.7 MiB 4 Full offline resolver benchmarks on two Linux devboxes per case, with 12 paired blocks each. Times are medians. Changes are paired wall-time estimates, with one standard error in percentage points in both tables. Workload Merged checkpoints Shared segments Time change Jupyter 9.06 ms 8.93 ms -1.5% ± 1.4 Jupyter universal 49.4 ms 49.9 ms +0.8% ± 1.1 Airflow 132.7 ms 132.8 ms +0.7% ± 1.2 Airflow universal 487.5 ms 495.8 ms +0.9% ± 0.4 All 17 public uv workloads produced matching outputs. The five uv backtracking fixtures were within 0.8% of merged uv. In standalone PubGrub, disjoint backtracking took 2.1% longer and singletons took 6.3% longer. Raw samples and reproduction commands are under benchmarks/fork-sharing/.
konsti-openai
added a commit
that referenced
this pull request
Oct 22, 2026
Cloning the arena shares the incompatibilities recorded so far. When either fork appends to a shared tail, it starts a segment of its own. Each partial solution keeps its own contradiction cache because backtracking can invalidate an entry in one fork while its sibling still needs it. Both uv fork sites can use an ordinary solver clone. Implementation diff Merged checkpoints Shared segments PubGrub (PR #91) +388/-144 +96/-92 uv (PR #22784) +43/-15 +4/-6 Counting additions and deletions, this changes 65% fewer lines in PubGrub and 83% fewer in uv. The uv diff only updates dependency pins. On a frozen 8,670-package workspace with 18 resolution environments, shared segments saved 182–184 MiB of peak RSS over pre-PR uv. All fresh locks and upgrades produced byte-identical lockfiles. The original PR measured a different snapshot with 7,777 packages. Shared segments vs. Solve Wall time Peak RSS Paired runs Pre-PR uv Fresh -3.4% ± 1.6 -182.1 MiB 3 Upgrade -2.0% ± 0.2 -184.4 MiB 2 Merged checkpoints Fresh +1.2% ± 1.7 -2.5 MiB 5 Upgrade +2.9% ± 1.0 +2.7 MiB 4 Full offline uv resolver benchmarks used two Linux devboxes and 12 paired blocks per workload. Times below are medians. Percentage changes in both tables are paired estimates ± one standard error. Workload Merged checkpoints Shared segments Time change Jupyter 9.06 ms 8.93 ms -1.5% ± 1.4 Jupyter universal 49.4 ms 49.9 ms +0.8% ± 1.1 Airflow 132.7 ms 132.8 ms +0.7% ± 1.2 Airflow universal 487.5 ms 495.8 ms +0.9% ± 0.4 All 17 public uv workloads produced matching outputs. The five uv backtracking fixtures were within 0.8% of merged uv. In standalone PubGrub, disjoint backtracking took 2.1% longer and singletons took 6.3% longer than with merged checkpoints.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
We reuse one incompatibility arena across depth-first solver forks. We save a checkpoint before branching, then discard each completed branch's additions before starting its sibling. Pending solver states can be cloned without copying the arena's records.
The contradiction cache moves into each partial solution so the incompatibility records remain immutable.
With uv #22784, a large project's 7,777-package resolves improve as follows against uv main (
384d963, which includes deferred forks):Six matched runs per variant and resolve across two Linux hosts, using warm offline caches. RSS values are medians; percentage changes use the matched runs.