Repository navigation
Shorten marker benchmark sampling - #22658
charliermarsh wants to merge 9 commits into
Conversation
Benchmark marker DNF conversion, extra substitution, and conflict simplification directly, with marker construction outside the timed loop. DNF conversion uses conflict guards with 4, 16, and 32 extras, producing 7, 121, and 497 clauses. Extra substitution uses compact diagrams with 4, 16, and 256 paths that all reduce to the same platform condition. These cases measure both clause simplification and the cost of expanding shared subgraphs. Conflict simplification measures `UniversalMarker::imbibe` alone and followed by formatting, using 4, 16, and 32 mutually exclusive extras. Environment-only markers cover redundant conflict guards; conflict-dependent markers and a no-conflict case provide controls. Each iteration starts from the original marker, so formatting measures the cost of the guards that remain after simplification. The benchmarks reuse the global marker interner across iterations and run in both CodSpeed modes. Related to #21954.
Merging this PR will improve performance by 12.68%
|
9f8a992 to
26a98ed
Compare
We currently repopulate the warm Git cache each time Criterion invokes the benchmark callback. Populate it once per selected benchmark and reuse it across warmup and samples. | PR | Change | Build + run | vs baseline | vs previous | Elapsed | |---|---|---:|---:|---:|---:| | [#22651](#22651) | Baseline | 35m28s | — | — | 40m53s | | **[#22657](#22657 | **Warm Git cache fix** | **24m48s** | **−10m40s** | **−10m40s** | **25m47s** | | [#22658](#22658) | Marker sampling | 23m38s | −11m50s | −1m10s | 25m07s | | [#22659](#22659) | Larger build runners | 20m01s | −15m27s | −3m37s | 21m14s | | [#22660](#22660) | Zstd artifacts | 17m25s | −18m03s | −2m36s | 18m39s | | [#22656](#22656) | Sharding | 14m58s | −20m30s | −2m27s | 15m51s | Rows are cumulative measurements from the original marker-based stack, including the removed marker-sampling layer, before rebasing onto `main`; this PR was measured at `9f8a992`. Build + run sums the build job and longest wall-time job; elapsed includes queueing. These are single runs, so build and network variation affect the differences ([CI run](https://github.com/astral-sh/uv/actions/runs/38958624816)).
Reduce marker benchmark warmup from 3s to 1s and measurement time from 5s to 2s. Keep all 25 cases and 100 samples per case.
Rows are cumulative. Build + run sums the build job and longest wall-time job; elapsed includes queueing. These are single runs, so build and network variation affect the differences (CI run).