Skip to content

Simplify selection markers under conflict constraints - #22639

Draft
charliermarsh wants to merge 32 commits into
charlie/marker-isopfrom
charlie/validate-marker-selections
Draft

charliermarsh wants to merge 32 commits into
charlie/marker-isopfrom
charlie/validate-marker-selections

Conversation

@charliermarsh

@charliermarsh charliermarsh commented Oct 16, 2026 •

Copy link
Copy Markdown
Member

We add a temporary, constraint-aware view of dependency markers for installation and export. This lets us simplify markers that still depend on extras or groups, while keeping the original markers for lockfiles and conflict discovery.

For example, suppose cpu and gpu are mutually exclusive extras:

[tool.uv]
conflicts = [
    [{ extra = "cpu" }, { extra = "gpu" }],
]

A dependency needed by cpu on Linux can have a marker like this, written here as Boolean shorthand:

(cpu AND gpu) OR (cpu AND linux)

The first term keeps the dependency edge reachable when both conflicting extras are active. That matters because dependencies can activate extras transitively: the existing lockfile reader follows those edges to discover the conflict and report an error. We cannot simply delete that term from the stored marker.

For a valid selection, however, cpu AND gpu is always false. We can evaluate the simpler marker:

cpu AND linux

This still depends on cpu. #22615 handles the narrower case where every valid selection reduces to the same environment-only marker. This PR also allows simplification when the result still depends on an extra or group.

The new ConflictMarkerContext keeps the assumptions together with the simplified view. It uses the existing MarkerTree::restrict operation to simplify the decision diagram, and only uses that view where the assumptions hold. For example, selecting both extras on macOS makes the compact marker false but the original marker true. In that case we retain the original marker, so the existing conflict-discovery and error-reporting behavior keeps its input. A reachable edge does not mean we permit the invalid installation.

The context is integrated in two places:

  • Installation: the first traversal discovers activated extras using the original markers. Once that set is fixed, we check its conflicts and use the compact views for valid selections.
  • Export: an extra can be active only on certain platforms. We substitute those platform conditions into the constraints, use the compact view where the constraints hold, and use the original marker elsewhere.

The context caches compact views and shares constraints between markers that reference the same conflict items. Constraint construction lets each item exclude the union of preceding items, avoiding enumeration of every conflicting pair. Collecting referenced extras visits each shared decision node once.

This generalizes in-memory simplification. It preserves the v1 lockfile representation and the existing activation-discovery algorithm, so it does not shrink lockfiles or remove every marker-scaling limit.

Local measurements compare #22615 at e93cb5696c with this PR at 962d915168, using 31 paired trials on the same Linux devbox:

Operation #22615 This PR
Construct a 32-item constraint, first call 6.34 ms 1.31 ms
Frozen sync, 32 mutually exclusive versions 103.36 ms 104.40 ms
Frozen export, 32 mutually exclusive versions 102.52 ms 102.55 ms

Constraint construction improves for larger sets, but preparing compact views adds work. With a warm BDD interner, preparing a context and evaluating 32 markers takes 309 µs, compared with 14 µs for direct evaluation. The roughly 1% sync increase should be treated as a small regression; these measurements do not establish a general end-to-end speedup.

All 11 lockfiles are byte-for-byte identical, all 88 command-result comparisons match, and all three installed environments match.

@codspeed

codspeed Bot commented Oct 16, 2026 •

Copy link
Copy Markdown

Merging this PR will degrade performance by 4.06%

1 warning

⚠️ 6 benchmarks are dominated by system calls

System calls cannot be simulated consistently and are left out of the measure. The reported value understates the real cost. Switch to the Walltime instrument to measure them.

Learn more about system calls

See affected benchmarks

⚡ 2 improved benchmarks
❌ 1 regressed benchmark
✅ 114 untouched benchmarks
⏩ 54 skipped benchmarks1

Warning

Please fix the performance issues or acknowledge them on CodSpeed.

Performance Changes

Mode Benchmark BASE HEAD Efficiency
❌ CPU Simulation imbibe_and_format_marker_environment_only[16] 44.4 µs 55.9 µs -20.7%
⚡ CPU Simulation imbibe_marker_environment_only[32] 40.6 µs 38.3 µs +5.86%
⚡ Walltime marker_to_dnf_conflicting_extras[4] 1.3 µs 1.2 µs +5.21%

Tip

Investigate this regression by commenting @codspeedbot fix this regression on this PR, or directly use the CodSpeed MCP with your agent.


Comparing charlie/validate-marker-selections (962d915) with charlie/compact-conflict-markers (e93cb56)

Open in CodSpeed

Footnotes

  1. 54 benchmarks were skipped, so the baseline results were used instead. If they were deleted from the codebase, click here and archive them to remove them from the performance reports. ↩

Large conflict sets produce marker DNFs with many clauses. Simplification repeatedly compares their terms, including string and version values. Intern those terms and represent clauses as bit sets so we can compare integer IDs and machine words instead.

Keep direct comparisons for small DNFs and use indexed term lookups for sparse clauses. Cap the dense bit-set index at 32 MiB and fall back to the linear algorithm for larger indexes or repeated terms within a clause. Terms and clauses are eliminated in the same order, so earlier removals affect later comparisons and serialized markers remain consistent.

This reduces the cost of each comparison; DNF materialization and the quadratic number of clause comparisons remain. Related to #21954.
@charliermarsh charliermarsh changed the title Prototype validation of marker selections Prototype separate request validation and installation selection Oct 16, 2026
zanieb and others added 10 commits October 16, 2026 13:06
Large marker DNFs often reuse a small set of terms across many clauses. The indexed simplifier recomputes the difference between two clauses for each term it considers removing. Compute viable differences once per clause and reuse them across those term checks.

Count missing terms first to discard candidates that cannot justify a removal, then update the cached differences after each removal so later checks use the surviving terms. Enable this path only when enough clauses reuse relatively few terms; smaller and sparser DNFs retain direct comparisons. Differential tests check that both paths produce the same ordered output as the linear simplifier.
Keep an environment marker when it describes the dependency under every valid conflict selection. Retain conflict-dependent and impossible edges so lockfile readers can still discover conflicting transitive extras.

Port astral-sh#2149. Related to #21954.
@charliermarsh charliermarsh changed the title Prototype separate request validation and installation selection Prototype request-graph validation for marker selections Oct 16, 2026
@charliermarsh
charliermarsh force-pushed the charlie/validate-marker-selections branch from c15e2df to 989c236 Compare October 16, 2026 17:14
@charliermarsh
charliermarsh force-pushed the charlie/compact-conflict-markers branch from 494c832 to 1954319 Compare October 16, 2026 17:14
@charliermarsh charliermarsh changed the title Prototype request-graph validation for marker selections Validate lockfile selections from retained dependency requests Oct 16, 2026
@charliermarsh charliermarsh changed the title Validate lockfile selections from retained dependency requests Avoid repeated traversal of shared marker nodes Oct 16, 2026
Keep an environment marker when it describes the dependency under every valid conflict selection. Retain conflict-dependent and impossible edges so lockfile readers can still discover conflicting transitive extras.

Port astral-sh#2149. Related to #21954.
@charliermarsh
charliermarsh force-pushed the charlie/compact-conflict-markers branch from 1954319 to e93cb56 Compare October 16, 2026 23:53
@charliermarsh charliermarsh changed the title Avoid repeated traversal of shared marker nodes Simplify selection markers under conflict constraints Oct 17, 2026
@charliermarsh
charliermarsh force-pushed the charlie/compact-conflict-markers branch from 567cd3a to 2b99716 Compare October 18, 2026 02:42
Base automatically changed from charlie/compact-conflict-markers to charlie/marker-isop October 18, 2026 03:24

This branch was previously deployed

1 inactive deployment
automations — 962d9151 Deployed Oct 17, 2026 by charliermarsh via review / security review #51474
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants