Skip to content

Use bit sets to simplify large marker DNFs - #22604

Merged
charliermarsh merged 0 commit into
charlie/compact-conflict-markersfrom
charlie/marker-dnf-bitsets
Oct 16, 2026
Merged

charliermarsh merged 0 commit into
charlie/compact-conflict-markersfrom
charlie/marker-dnf-bitsets

Conversation

@charliermarsh

@charliermarsh charliermarsh commented Oct 15, 2026 •

Copy link
Copy Markdown
Member

Summary

We convert decision diagrams to disjunctive normal form (DNF) when rendering markers, then simplify the result by comparing clauses and removing redundant terms. When there are many conflicting extras, the DNF can contain a lot of clauses, and those comparisons can be expensive (see #21954).

This PR makes the comparisons cheaper for larger DNFs. We assign each distinct marker term an integer and represent each clause as a bit set, so we can compare sets instead of repeatedly comparing full marker expressions. We keep the same removal order, since removing one term can affect later comparisons.

For small DNFs, clauses with repeated terms, or expressions that would need a bit-set index larger than 32 MiB, we fall back to the existing simplifier. We also use indexed term lookups for sparse clauses.

(We still build the DNF and perform up to a quadratic number of clause comparisons; #22608 avoids DNF for Boolean operations, while #22615 prevents redundant conflict clauses on environment-only markers.)

CodSpeed comparison against #22615 measures the 25 marker cases in both walltime and simulation. CodSpeed reports different simulation runners (AMD and Intel); the wall-time results have no runner mismatch.

@codspeed

codspeed Bot commented Oct 15, 2026 •

Copy link
Copy Markdown

Merging this PR will improve performance by ×6.7

⚠️ 8 benchmarks spent significant time in system calls

System calls cannot be consistently instrumented, so they are not included in the measure, which understates the real cost. Please switch to the Walltime instrument to accurately measure system calls.

Measurement and system calls

⚠️ Different runtime environments detected

Some benchmarks with significant performance changes were compared across different runtime environments,
which may affect the accuracy of the results.

Open the report in CodSpeed to investigate

⚡ 16 improved benchmarks
✅ 103 untouched benchmarks
⏩ 54 skipped benchmarks1

Performance Changes

Mode Benchmark BASE HEAD Efficiency
⚡ WallTime marker_to_dnf_conflicting_extras[32] 230.3 ms 4 ms ×57
⚡ Simulation marker_to_dnf_conflicting_extras[32] 4,084.3 ms 73.6 ms ×56
⚡ WallTime imbibe_and_format_marker_conflict_dependent[32] 627.9 ms 15.8 ms ×40
⚡ Simulation imbibe_and_format_marker_conflict_dependent[32] 11,610 ms 297.2 ms ×39
⚡ Simulation marker_to_dnf_conflicting_extras[16] 71.3 ms 3.2 ms ×23
⚡ WallTime marker_to_dnf_conflicting_extras[16] 3,840.6 µs 177 µs ×22
⚡ Simulation imbibe_and_format_marker_conflict_dependent[16] 227.3 ms 10.9 ms ×21
⚡ WallTime imbibe_and_format_marker_conflict_dependent[16] 11,774.2 µs 594.2 µs ×20
⚡ WallTime imbibe_and_format_marker_conflict_dependent[4] 6.7 µs 3.5 µs +88.74%
⚡ Simulation marker_to_dnf_sparse_clauses[512] 150 ms 89.4 ms +67.72%
⚡ Simulation imbibe_and_format_marker_conflict_dependent[4] 177.4 µs 111.5 µs +59.1%
⚡ Simulation marker_to_dnf_sparse_clauses[128] 9 ms 5.7 ms +57.61%
⚡ WallTime marker_to_dnf_sparse_clauses[512] 9.6 ms 6.8 ms +40.1%
⚡ WallTime marker_to_dnf_sparse_clauses[128] 606.1 µs 472.1 µs +28.38%
⚡ Simulation imbibe_marker_no_conflicts[0] 398.5 ns 343 ns +16.2%
⚡ ⚠️ Simulation discover_workspace_from_all_members 143.3 ms 134.5 ms +6.6%

Tip

Curious why performance improved? Comment @codspeedbot explain why performance improved on this PR, or directly use the CodSpeed MCP with your agent.


Comparing charlie/marker-dnf-bitsets (74ec208) with charlie/compact-conflict-markers (d8dd421)

Open in CodSpeed

Footnotes

  1. 54 benchmarks were skipped, so the baseline results were used instead. If they were deleted from the codebase, click here and archive them to remove them from the performance reports. ↩

@charliermarsh
charliermarsh removed this pull request from stack #22609 October 15, 2026 20:31
@charliermarsh
charliermarsh force-pushed the charlie/marker-dnf-bitsets branch from b5bea7f to fb20068 Compare October 15, 2026 20:31
Base automatically changed from charlie/bench-conflicting-extras to charlie/direct-marker-transform October 15, 2026 20:31
@charliermarsh
charliermarsh force-pushed the charlie/marker-dnf-bitsets branch from fb20068 to 0a1ee00 Compare October 15, 2026 20:40
@charliermarsh
charliermarsh force-pushed the charlie/marker-dnf-bitsets branch from 0a1ee00 to a36e070 Compare October 15, 2026 20:43
@charliermarsh
charliermarsh removed this pull request from stack #22611 October 15, 2026 20:58
@charliermarsh
charliermarsh changed the base branch from charlie/direct-marker-transform to charlie/compact-conflict-markers October 15, 2026 20:58
@charliermarsh
charliermarsh force-pushed the charlie/marker-dnf-bitsets branch from a36e070 to e84dcae Compare October 15, 2026 20:58
@charliermarsh
charliermarsh force-pushed the charlie/marker-dnf-bitsets branch from e84dcae to 4e0719b Compare October 15, 2026 21:06
@charliermarsh
charliermarsh force-pushed the charlie/marker-dnf-bitsets branch from 4e0719b to 06c4df1 Compare October 15, 2026 21:18
@charliermarsh
charliermarsh force-pushed the charlie/marker-dnf-bitsets branch from 06c4df1 to 74ec208 Compare October 15, 2026 22:57
@charliermarsh
charliermarsh removed this pull request from stack #22616 October 16, 2026 01:15
@charliermarsh
charliermarsh force-pushed the charlie/marker-dnf-bitsets branch from 74ec208 to b59f827 Compare October 16, 2026 15:10
@charliermarsh
charliermarsh merged commit b59f827 into main Oct 16, 2026
@charliermarsh
charliermarsh deleted the charlie/marker-dnf-bitsets branch October 16, 2026 15:10
@charliermarsh
charliermarsh restored the charlie/marker-dnf-bitsets branch October 16, 2026 15:11

This branch was previously deployed

1 inactive deployment
automations — b59f8275 Deployed Oct 16, 2026 by charliermarsh via review / security review #51352
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant