Skip to content

Resolve large conflict markers without expanding DNF - #2148

Draft
zaniebot wants to merge 6 commits into
zb/bench-universal-conflict-markersfrom
zb/resolve-activated-extras-in-bdd
Draft

zaniebot wants to merge 6 commits into
zb/bench-universal-conflict-markersfrom
zb/resolve-activated-extras-in-bdd

Conversation

@zaniebot

@zaniebot zaniebot commented Sep 29, 2026 •

Copy link
Copy Markdown
Collaborator

Exporting existing lockfiles with many conflicting extras expands their markers into DNF before substituting activation conditions, making uv export slow even after locking has completed. Substitute those conditions directly in the decision diagram for markers with more than eight extras, and coalesce adjacent ranges so equivalent markers remain canonical during reachability analysis.

Hyperfine 2.0.0 measurements of fa69b46dd → 1f7c651fd used release builds of the complete CLI with rustc 1.99.0 (b940084d7 2026-09-28) on an AMD EPYC-Genoa Linux VM, pinned to four CPUs. Output went to /dev/null. The supplied interpreter was CPython 3.12.13.

Commands include uv export --frozen. Synthetic conflict workspaces use frozen parent-generated locks with 4 to 50 mutually exclusive extras, conditional or constant activation, and wider graphs; a no-conflict case is included. Within the supported Python range >=3.12,<3.14, conditional cases declare extra-000 below Python 3.13 and extra-001 from Python 3.13; constant activation selects extra-000. The eight- and nine-extra conflict cases contain up to eight and nine distinct extras in an input marker, bracketing the targeted representation threshold. Each command starts a fresh process; OS pages, local wheel metadata, and interpreter-query caches are warm. Frozen exports read the same parent-generated locks. No external index, source backend, or package installation is used.

Values are means, parent → head. Changes are (head / parent - 1) with 95% confidence intervals; negative values indicate a reduction.

Workload Wall time Change, 95% CI Peak RSS Change, 95% CI
export / 4 extras / 1 shared name / 1 common dependency / conflicts / conditional activation 16.7 → 16.7 ms -0.29% [-1.35%, +0.74%] 28.13 → 27.95 MiB -0.64% [-1.02%, -0.22%]
export / 8 extras / 1 shared name / 1 common dependency / conflicts / conditional activation 17.5 → 17.4 ms -0.51% [-2.13%, +1.02%] 28.61 → 28.42 MiB -0.67% [-1.13%, -0.18%]
export / 9 extras / 1 shared name / 1 common dependency / conflicts / conditional activation 17.9 → 18.4 ms +2.40% [-0.74%, +4.97%] 28.64 → 29.02 MiB +1.34% [+0.84%, +1.78%]
export / 16 extras / 1 shared name / 1 common dependency / conflicts / conditional activation 22.1 → 22.0 ms -0.48% [-2.58%, +1.88%] 30.01 → 30.70 MiB +2.31% [+2.04%, +2.59%]
export / 30 extras / 1 shared name / 1 common dependency / conflicts / conditional activation 72.1 → 35.6 ms -50.64% [-51.19%, -50.00%] 38.82 → 38.39 MiB -1.10% [-1.45%, -0.73%]
export / 50 extras / 1 shared name / 1 common dependency / conflicts / conditional activation 500.5 → 89.9 ms -82.04% [-82.22%, -81.82%] 74.70 → 74.49 MiB -0.28% [-0.43%, -0.15%]
export / 50 extras / 1 shared name / 1 common dependency / conflicts / constant activation 138.5 → 89.2 ms -35.62% [-36.31%, -34.84%] 74.69 → 74.48 MiB -0.28% [-0.37%, -0.18%]
export / 16 extras / 16 shared names / 32 common dependencies / conflicts / conditional activation 51.1 → 30.3 ms -40.57% [-41.62%, -39.61%] 31.21 → 32.36 MiB +3.68% [+3.29%, +4.02%]
export / 16 extras / 16 shared names / 32 common dependencies / no conflicts / conditional activation 18.3 → 17.9 ms -1.86% [-2.96%, -0.70%] 28.58 → 28.32 MiB -0.87% [-1.28%, -0.50%]

Each workload used ten comparison blocks with alternating revision order, one warmup followed by three measured runs per revision in each block (30 measurements per revision). The pointwise 95% percentile-bootstrap intervals use 20,000 resamples of paired block means and retain all samples. They estimate within-run uncertainty, not variation across machines, independent builds, or separate benchmark campaigns.

The 30- and 50-extra conditional cases ran 50.64% and 82.04% faster; the 50-extra constant case ran 35.62% faster. The wider 16-extra conflict case ran 40.57% faster with 3.68% higher RSS. Runtime intervals for the single-name four-, eight-, nine-, and 16-extra cases include zero; RSS rose in the nine- and 16-extra cases and fell in the four- and eight-extra cases. The no-conflict control ran 1.86% faster and used 0.87% less RSS, although its input markers contained at most one extra. The full source pair includes changes to extra visitation and range canonicalization, so these measurements do not isolate substitution. RSS is Hyperfine's Linux wait4 peak, not an allocation count or summed simultaneous process-tree memory.

@codspeed

codspeed Bot commented Sep 29, 2026 •

Copy link
Copy Markdown

Merging this PR will improve performance by ×6.1

⚠️ Different runtime environments detected

Some benchmarks with significant performance changes were compared across different runtime environments,
which may affect the accuracy of the results.

Open the report in CodSpeed to investigate

⚡ 28 improved benchmarks
✅ 37 untouched benchmarks
🆕 8 new benchmarks
⏩ 47 skipped benchmarks1

Performance Changes

Mode Benchmark BASE HEAD Efficiency
⚡ WallTime resolve_activated_extras[32] 19,679.5 µs 85.5 µs ×230
⚡ Simulation resolve_activated_extras[32] 84.4 ms 1.2 ms ×71
⚡ WallTime resolve_activated_extras[16] 1,077.2 µs 22.7 µs ×47
⚡ WallTime unzip_wheel_many_files 377.3 ms 12.3 ms ×31
⚡ WallTime install_wheel_many_files 384.4 ms 13.9 ms ×28
⚡ WallTime prepare_wheel_many_files 390.3 ms 14.9 ms ×26
⚡ Simulation resolve_activated_extras[16] 4,260.7 µs 347.3 µs ×12
⚡ WallTime unpack_sdist_many_files 316.1 ms 36.6 ms ×8.6
⚡ WallTime resolve_activated_extras[4] 27.9 µs 3.6 µs ×7.7
⚡ WallTime resolve_warm_jupyter_universal 87.6 ms 11.9 ms ×7.4
⚡ WallTime resolve_warm_jupyter 16.7 ms 2.5 ms ×6.7
⚡ WallTime discover_workspace_from_all_members_with_excludes 53.4 ms 8.2 ms ×6.5
⚡ WallTime discover_workspace_from_all_members 50.5 ms 7.9 ms ×6.4
⚡ WallTime run_python_version_synthetic_workspace 269.3 ms 42.8 ms ×6.3
⚡ WallTime imbibe_and_format[4] 19.1 µs 3.3 µs ×5.8
⚡ WallTime resolve_warm_airflow 234.3 ms 42.3 ms ×5.5
⚡ WallTime imbibe_and_format[32] 18.4 ms 3.4 ms ×5.4
⚡ WallTime imbibe_and_format[16] 888.3 µs 163.8 µs ×5.4
⚡ WallTime hash_sha256 1,165.7 µs 406.5 µs ×2.9
⚡ Simulation from_hex/lowercase[64] 438.5 µs 269.6 µs +62.65%
... ... ... ... ... ...

ℹ️ Only the first 20 benchmarks are displayed. Go to the app to view all benchmarks.

Tip

Curious why performance improved? Comment @codspeedbot explain why performance improved on this PR, or directly use the CodSpeed MCP with your agent.


Comparing zb/resolve-activated-extras-in-bdd (3b3a7b5) with zb/bench-universal-conflict-markers (7c5b215)2

Open in CodSpeed

Footnotes

  1. 47 benchmarks were skipped, so the baseline results were used instead. If they were deleted from the codebase, click here and archive them to remove them from the performance reports. ↩

  2. No successful run was found on zb/bench-universal-conflict-markers (fa69b46) during the generation of this report, so 7163639 was used instead as the comparison base. There might be some changes unrelated to this pull request in this report. ↩

@zaniebot
zaniebot added this pull request to stack #2155 September 29, 2026 22:34
@zaniebot zaniebot added risk:3 Credible risk of meaningful incorrect behavior or regression; bounded or recoverable. priority:3 Normal: useful features, documentation, usability, or routine efficiency improvements. size:3 Substantial: interacting behaviors, difficult compatibility questions, or a careful migration. labels Sep 30, 2026
@zaniebot
zaniebot removed this pull request from stack #2155 October 7, 2026 23:37
@zaniebot
zaniebot added this pull request to stack #2337 October 7, 2026 23:38
@zaniebot
zaniebot deployed to automations October 7, 2026 23:38 — with GitHub Actions Active
@zaniebot
zaniebot deployed to automations October 7, 2026 23:42 — with GitHub Actions Active
Comment on lines +869 to +871
marker.visit_extras(|_, extra| {
if !extras.is_empty() {
extras.insert(extra.clone());

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[P2] Traverse shared BDD nodes only once

MarkerTree::visit_extras follows both branches recursively without remembering visited nodes; the set here only deduplicates the resulting names. For a conjunction of n independent two-extra disjunctions, with names ordered pair by pair, it invokes this callback 2^(n+1)-2 times despite only 2n distinct decision nodes. Large compact markers therefore still incur exponential work before reaching the new BDD path. Use node-memoized extra collection, and cache the restrictions used below (restrict_by also revisits shared subgraphs). Add this shared-subgraph shape to the benchmark coverage.

Comment on lines +949 to +951
// Substitute extras in BDD variable order. Restricting a later variable before an earlier one
// can leave adjacent version/string ranges pointing at the same child, which is semantically
// equivalent but not canonical and can prevent downstream reachability from converging.

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[P3] Update the sorting rationale after fixing range coalescing

The new Edges::map_ranges coalesces adjacent ranges after restriction, and simplify_extras_preserves_canonical_ranges explicitly asserts canonical results for both substitution orders. This comment nevertheless describes ordering as necessary for canonicality and reachability convergence, obscuring the invariant now enforced by the algebra layer. Retain sorting if deterministic ordering is useful, but describe that purpose and remove the obsolete correctness claim.

@zaniebot
zaniebot removed this pull request from stack #2337 October 8, 2026 00:02
@zaniebot
zaniebot added this pull request to stack #2339 October 8, 2026 00:02
@zaniebot
zaniebot removed this pull request from stack #2339 October 8, 2026 00:19
@zaniebot
zaniebot added this pull request to stack #2340 October 8, 2026 00:19

This branch was successfully deployed

1 active (outdated) deployment
automations — 3b3a7b5e Deployed Oct 7, 2026 by zaniebot via review-code-quality / report #7887
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

performance priority:3 Normal: useful features, documentation, usability, or routine efficiency improvements. risk:3 Credible risk of meaningful incorrect behavior or regression; bounded or recoverable. size:3 Substantial: interacting behaviors, difficult compatibility questions, or a careful migration.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants