Repository navigation
Conversation
Merging this PR will improve performance by 30.33%
|
| Mode | Benchmark | BASE |
HEAD |
Efficiency | |
|---|---|---|---|---|---|
| ⚡ | Simulation | load_python_download_catalog |
452.7 ms | 347.3 ms | +30.33% |
| 🆕 | WallTime | uv_lock_no_lockfile_warm_metadata_cache_16_conflicting_extras |
N/A | 2.8 ms | N/A |
| 🆕 | WallTime | uv_lock_no_lockfile_warm_metadata_cache_32_conflicting_extras |
N/A | 9 ms | N/A |
| 🆕 | WallTime | uv_lock_no_lockfile_warm_metadata_cache_4_conflicting_extras |
N/A | 1.1 ms | N/A |
| 🆕 |
Simulation | uv_lock_no_lockfile_warm_metadata_cache_16_conflicting_extras |
N/A | 32.8 ms | N/A |
| 🆕 |
Simulation | uv_lock_no_lockfile_warm_metadata_cache_32_conflicting_extras |
N/A | 166.2 ms | N/A |
| 🆕 |
Simulation | uv_lock_no_lockfile_warm_metadata_cache_4_conflicting_extras |
N/A | 9.9 ms | N/A |
Tip
Curious why performance improved? Comment @codspeedbot explain why performance improved on this PR, or directly use the CodSpeed MCP with your agent.
Comparing zb/index-marker-dnf (ec10f8a) with zb/bench-large-marker-dnf (3363300)2
Footnotes
-
41 benchmarks were skipped, so the baseline results were used instead. If they were deleted from the codebase, click here and archive them to remove them from the performance reports. ↩
-
No successful run was found on
zb/bench-large-marker-dnf(11dd3da) during the generation of this report, so bf47876 was used instead as the comparison base. There might be some changes unrelated to this pull request in this report. ↩
d0777ec to
bb74dfc
Compare
| let words = expressions.len().div_ceil(64); | ||
| // A sparse expression can have far more distinct terms than terms per clause. | ||
| // Bound the dense index to 32 MiB and use the linear algorithm beyond that. | ||
| if dnf.len().saturating_mul(words) > MAX_INDEX_WORDS { |
There was a problem hiding this comment.
[P2] Fall back to linear comparisons for sparse DNFs
The memory cap still admits sparse inputs that add substantial comparison work. collect_dnf produces singleton clauses for disjunctions such as platform_machine == 'arch-0' or platform_machine == 'arch-1' or .... Both indexed passes scan leading zero words before finding each other clause's term, adding a factor proportional to the number of distinct terms. At 16,384 singleton clauses, the index fits exactly within 32 MiB but requires approximately 69 billion word visits; the linear algorithm performs constant work per clause pair. Include clause density in the fallback decision, or skip unpopulated words, and cover a sparse input below the storage limit.
| for _ in 0..next() % 30 { | ||
| let mut clause = Vec::new(); | ||
| for _ in 0..next() % 12 { | ||
| let term = expressions[next() % expressions.len()].clone(); | ||
| if case % 2 == 0 || !clause.contains(&term) { |
There was a problem hiding this comment.
[P3] Exercise negation in the sparse comparison tests
Replaying this fixed generator produces at most 119 distinct terms, so even the many_expressions pass never reaches sparse_comparisons, which requires more than 128. The dedicated sparse fixtures contain no complementary terms, while the multiword fixtures use dense comparisons. Consequently, the new sparse path’s acceptance of a negated term lacks equivalence coverage. Add a deterministic fixture with more than 128 distinct terms and short complementary clauses, such as (A and B) and (not A and B), and compare its ordered result with simplify_linear.
| /// Both [`MarkerTree::TRUE`] and [`MarkerTree::FALSE`] return an empty list. Callers that | ||
| /// need to distinguish these constants must handle them before converting to DNF. | ||
| pub fn to_unsimplified_dnf(self) -> Vec<Vec<MarkerExpression>> { | ||
| simplify::to_unsimplified_dnf(self) |
There was a problem hiding this comment.
[P3] Preserve constant truth values in the new DNF API
Returning [] for both constants makes this Boolean-transformation API lossy: the reconstruction fold in unsimplified_dnf_reconstructs_marker would turn TRUE into FALSE. Both production callers guard constants correctly, but every future consumer must remember this extra precondition. Return one empty conjunction for TRUE and no conjunctions for FALSE in the new method, leaving the existing to_dnf contract unchanged, and include a tautology in the reconstruction test.
Large conflict sets can make
uv lockslow, as reported in astral-sh#21954. Mutually exclusive extras can produce large marker expressions: converting them to disjunctive normal form (DNF) can yield many clauses, and simplification repeatedly compares their marker terms. This also adds work touv export, which transforms marker trees while constructing output.Intern marker expressions and represent clauses as bit sets to make those comparisons cheaper. For large DNFs whose clauses reuse relatively few terms, count missing terms before enumerating bit-set differences to discard candidates that cannot justify removing a term. Cache the remaining differences for reuse while simplifying the clause.
Candidate caching is limited to expressions where its setup can pay off; smaller or sparser expressions compare directly, and short clauses use indexed term lookups rather than scanning mostly empty bit sets. Cap the dense bit-set index at 32 MiB and fall back to linear comparisons for larger indexes or repeated terms within a clause. Terms and clauses are still eliminated in the same order, so earlier removals affect later comparisons and the simplified marker output remains consistent.
Skip DNF simplification when resolving activated extras and checking marker reachability, where the Boolean result does not depend on removing redundant clauses. Public marker output and conflict diagnostics continue to use simplified DNF, keeping marker text compact and avoiding repeated warnings from redundant expressions.