Skip to content

Experiment with bulk Ruff cache metadata on macOS - #28300

Closed
charliermarsh wants to merge 1 commit into
mainfrom
charlie/ruff-bulk-cache-metadata
Closed

charliermarsh wants to merge 1 commit into
mainfrom
charlie/ruff-bulk-cache-metadata

Conversation

@charliermarsh

Copy link
Copy Markdown
Member

Summary

Explore reading the modification times and permissions needed for Ruff's file-cache keys with macOS getattrlistbulk, following astral-sh/uv#21344. We group selected files by parent directory after discovery and prefetch keys for directories with multiple selected files. Symlinks, unsupported attributes, and failed bulk reads retain individual metadata lookups. Prefetched keys are consumed once, and formatting still fetches fresh metadata after writing a file.

This implementation has not demonstrated a performance improvement. It adds a directory scan after the ignore walk, plus maps for the prefetched keys. A local paired run produced these median full-command times:

Synthetic tree Command Main This change
100 directories × 100 Python files Cached check 27.5 ms 32.1 ms
100 directories × 100 Python files Cached format check 29.8 ms 32.2 ms
1,000 directories × 5 Python files Cached check 39.8 ms 52.7 ms
1,000 directories × 5 Python files Cached format check 40.1 ms 53.0 ms

These are synthetic measurements on macOS 26.6.2 using optimized development builds, with ten measured runs per binary in ABBA order after three warmups. Each file contains value = 1; configuration discovery is disabled to isolate cache handling, and command output matches. The machine was shared and not idle, so the magnitudes are provisional. The comparison base is 6077dc3d7d.

The extra scan is a likely reason the reduced metadata-call count does not translate into a win here. Integrating metadata collection with the existing directory enumeration needs investigation before considering this a performance improvement.

@charliermarsh charliermarsh added the performance Potential performance improvement label Sep 4, 2026
@astral-sh-bot

astral-sh-bot Bot commented Sep 4, 2026

Copy link
Copy Markdown

Typing conformance results

No changes detected ✅

Current numbers
The percentage of diagnostics emitted that were expected errors held steady at 97.84%. The percentage of expected errors that received a diagnostic held steady at 96.45%. The number of fully passing files held steady at 128/145.

@astral-sh-bot

astral-sh-bot Bot commented Sep 4, 2026

Copy link
Copy Markdown

Memory usage report

Memory usage unchanged ✅

@astral-sh-bot

astral-sh-bot Bot commented Sep 4, 2026

Copy link
Copy Markdown

ecosystem-analyzer results

No diagnostic changes detected ✅

Full report with detailed diff (timing results)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

performance Potential performance improvement

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant