Repository navigation
Use restriction to simplify conflict markers - #22637
charliermarsh wants to merge 10 commits into
Conversation
Merging this PR will regress 5 benchmarks2 warnings
Warning Please fix the performance issues or acknowledge them on CodSpeed. Performance Changes
Tip Investigate this regression by commenting Comparing Footnotes
|
Large conflict sets produce marker DNFs with many clauses. Simplification repeatedly compares their terms, including string and version values. Intern those terms and represent clauses as bit sets so we can compare integer IDs and machine words instead. Keep direct comparisons for small DNFs and use indexed term lookups for sparse clauses. Cap the dense bit-set index at 32 MiB and fall back to the linear algorithm for larger indexes or repeated terms within a clause. Terms and clauses are eliminated in the same order, so earlier removals affect later comparisons and serialized markers remain consistent. This reduces the cost of each comparison; DNF materialization and the quadratic number of clause comparisons remain. Related to #21954.
d8dd421 to
494c832
Compare
Large marker DNFs often reuse a small set of terms across many clauses. The indexed simplifier recomputes the difference between two clauses for each term it considers removing. Compute viable differences once per clause and reuse them across those term checks. Count missing terms first to discard candidates that cannot justify a removal, then update the cached differences after each removal so later checks use the surviving terms. Enable this path only when enough clauses reuse relatively few terms; smaller and sparser DNFs retain direct comparisons. Differential tests check that both paths produce the same ordered output as the linear simplifier.
Keep an environment marker when it describes the dependency under every valid conflict selection. Retain conflict-dependent and impossible edges so lockfile readers can still discover conflicting transitive extras. Port astral-sh#2149. Related to #21954.
c019421 to
95af9b1
Compare
494c832 to
1954319
Compare
567cd3a to
2b99716
Compare
We can simplify conflict markers under the assumption that the configured conflicts hold by calling
MarkerTree::restrict. Use it inUniversalMarker::imbibeinstead of intersecting with the conflicts, projecting away extras, and comparing the result against the intersection.We accept the restricted marker only when it is nonempty and no longer depends on extras. Otherwise, we retain the implication fallback so lockfile readers can discover and report transitive conflicts.
This is a draft follow-up to #22615: the simpler implementation has a significant performance tradeoff. A paired operation harness on an idle Linux devbox produced the following results in two runs, with the same optimized binary, a warm interner, and seven alternating samples per case:
Selecting a late extra requires restriction to traverse the preceding conflict decisions, after which we discard the conflict-dependent result and take the fallback. We add a last-extra case to the operation and formatting benchmarks; the existing cases selected only the first extra. These timings isolate
imbibe, not end-to-end resolution or formatting.All 64 focused lockfile tests passed. An exhaustive comparison of all 65,536 Boolean functions over one environment condition and three extras, under four conflict configurations, found no output differences from #22615.