Skip to content

Use restriction to simplify conflict markers - #22637

Draft
charliermarsh wants to merge 10 commits into
charlie/marker-isopfrom
charlie/restrict-conflict-markers
Draft

charliermarsh wants to merge 10 commits into
charlie/marker-isopfrom
charlie/restrict-conflict-markers

Conversation

@charliermarsh

Copy link
Copy Markdown
Member

We can simplify conflict markers under the assumption that the configured conflicts hold by calling MarkerTree::restrict. Use it in UniversalMarker::imbibe instead of intersecting with the conflicts, projecting away extras, and comparing the result against the intersection.

We accept the restricted marker only when it is nonempty and no longer depends on extras. Otherwise, we retain the implication fallback so lockfile readers can discover and report transitive conflicts.

This is a draft follow-up to #22615: the simpler implementation has a significant performance tradeoff. A paired operation harness on an idle Linux devbox produced the following results in two runs, with the same optimized binary, a warm interner, and seven alternating samples per case:

Operation #22615 This change Change
Environment-only, 4 extras 0.334 µs 0.389–0.395 µs 16–18% slower
Environment-only, 32 extras 1.95–1.98 µs 0.397–0.405 µs 4.9× faster
First extra selected, 32 extras 1.76–1.79 µs 0.724–0.737 µs 2.4× faster
Last extra selected, 32 extras 1.78–1.80 µs 7.15–7.30 µs 4.0× slower

Selecting a late extra requires restriction to traverse the preceding conflict decisions, after which we discard the conflict-dependent result and take the fallback. We add a last-extra case to the operation and formatting benchmarks; the existing cases selected only the first extra. These timings isolate imbibe, not end-to-end resolution or formatting.

All 64 focused lockfile tests passed. An exhaustive comparison of all 65,536 Boolean functions over one environment condition and three extras, under four conflict configurations, found no output differences from #22615.

@codspeed

codspeed Bot commented Oct 16, 2026 •

Copy link
Copy Markdown

Merging this PR will regress 5 benchmarks

2 warnings

⚠️ 6 benchmarks are dominated by system calls

System calls cannot be simulated consistently and are left out of the measure. The reported value understates the real cost. Switch to the Walltime instrument to measure them.

Learn more about system calls

See affected benchmarks

⚠️ Base and head ran on different runner types

Some benchmarks with significant performance changes were compared across different runner types, so the changes might not come from the code. Run base and head on the same runner type, or use CodSpeed Macro Runners for more stable results.

Learn more about Macro Runners

Open the report in CodSpeed to investigate

⚡ 13 improved benchmarks
❌ 5 regressed benchmarks
✅ 99 untouched benchmarks
🆕 12 new benchmarks
⏩ 54 skipped benchmarks1

Warning

Please fix the performance issues or acknowledge them on CodSpeed.

Performance Changes

Mode Benchmark BASE HEAD Efficiency
❌ CPU Simulation imbibe_marker_conflict_dependent[4] 20.1 µs 26.2 µs -23.48%
❌ Walltime imbibe_marker_conflict_dependent[4] 309 ns 379 ns -18.47%
❌ Walltime imbibe_marker_environment_only[4] 184 ns 209 ns -11.96%
❌ CPU Simulation imbibe_marker_environment_only[4] 17 µs 18.4 µs -7.34%
❌ CPU Simulation imbibe_and_format_marker_conflict_dependent[4] 108.6 µs 114.5 µs -5.17%
⚡ Walltime imbibe_marker_environment_only[32] 955 ns 209 ns ×4.6
⚡ Walltime imbibe_and_format_marker_environment_only[32] 1,234 ns 469 ns ×2.6
⚡ Walltime imbibe_marker_environment_only[16] 524 ns 209 ns ×2.5
⚡ Walltime imbibe_marker_conflict_dependent[32] 880 ns 378 ns ×2.3
⚡ CPU Simulation imbibe_marker_environment_only[32] 39.8 µs 18.4 µs ×2.2
⚡ Walltime imbibe_and_format_marker_environment_only[16] 812 ns 471 ns +72.4%
⚡ Walltime imbibe_marker_conflict_dependent[16] 589 ns 379 ns +55.41%
⚡ CPU Simulation imbibe_and_format_marker_environment_only[32] 61.4 µs 39.9 µs +53.85%
⚡ CPU Simulation imbibe_marker_environment_only[16] 27.1 µs 18.4 µs +47.35%
⚡ CPU Simulation imbibe_marker_conflict_dependent[32] 33 µs 26.3 µs +25.24%
⚡ CPU Simulation imbibe_and_format_marker_environment_only[16] 48.7 µs 39.9 µs +22.09%
⚡ CPU Simulation imbibe_marker_no_conflicts[0] 398.5 ns 343 ns +16.2%
⚡ Walltime resolve_activated_extras_independent_pairs[2] 332 ns 315 ns +5.4%
🆕 Walltime imbibe_and_format_marker_conflict_dependent_last[16] N/A 360.1 µs N/A
🆕 Walltime imbibe_and_format_marker_conflict_dependent_last[32] N/A 6.5 ms N/A
... ... ... ... ... ...

ℹ️ Only the first 20 benchmarks are displayed. Go to the app to view all benchmarks.

Tip

Investigate this regression by commenting @codspeedbot fix this regression on this PR, or directly use the CodSpeed MCP with your agent.


Comparing charlie/restrict-conflict-markers (95af9b1) with charlie/compact-conflict-markers (1954319)

Open in CodSpeed

Footnotes

  1. 54 benchmarks were skipped, so the baseline results were used instead. If they were deleted from the codebase, click here and archive them to remove them from the performance reports. ↩

Large conflict sets produce marker DNFs with many clauses. Simplification repeatedly compares their terms, including string and version values. Intern those terms and represent clauses as bit sets so we can compare integer IDs and machine words instead.

Keep direct comparisons for small DNFs and use indexed term lookups for sparse clauses. Cap the dense bit-set index at 32 MiB and fall back to the linear algorithm for larger indexes or repeated terms within a clause. Terms and clauses are eliminated in the same order, so earlier removals affect later comparisons and serialized markers remain consistent.

This reduces the cost of each comparison; DNF materialization and the quadratic number of clause comparisons remain. Related to #21954.
@charliermarsh
charliermarsh force-pushed the charlie/compact-conflict-markers branch from d8dd421 to 494c832 Compare October 16, 2026 15:10
charliermarsh and others added 9 commits October 16, 2026 11:22
Large marker DNFs often reuse a small set of terms across many clauses. The indexed simplifier recomputes the difference between two clauses for each term it considers removing. Compute viable differences once per clause and reuse them across those term checks.

Count missing terms first to discard candidates that cannot justify a removal, then update the cached differences after each removal so later checks use the surviving terms. Enable this path only when enough clauses reuse relatively few terms; smaller and sparser DNFs retain direct comparisons. Differential tests check that both paths produce the same ordered output as the linear simplifier.
Keep an environment marker when it describes the dependency under every valid conflict selection. Retain conflict-dependent and impossible edges so lockfile readers can still discover conflicting transitive extras.

Port astral-sh#2149. Related to #21954.
@charliermarsh
charliermarsh force-pushed the charlie/restrict-conflict-markers branch from c019421 to 95af9b1 Compare October 16, 2026 17:14
@charliermarsh
charliermarsh force-pushed the charlie/compact-conflict-markers branch from 494c832 to 1954319 Compare October 16, 2026 17:14
@charliermarsh
charliermarsh force-pushed the charlie/compact-conflict-markers branch 2 times, most recently from 567cd3a to 2b99716 Compare October 18, 2026 02:42
Base automatically changed from charlie/compact-conflict-markers to charlie/marker-isop October 18, 2026 03:24

This branch was previously deployed

1 inactive deployment
automations — 95af9b16 Deployed Oct 16, 2026 by charliermarsh via review / security review #51374
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants