Repository navigation
perf(fuse): set FOPEN_KEEP_CACHE on read-only mounts - #130
Merged
Merged
Conversation
By default, the FUSE kernel module invalidates the page cache for a file on every open() — a conservative behavior because the FUSE server might mutate the file out-of-band. For our read-only mount the underlying view is stable for the lifetime of the session, so we can ask the kernel to retain cached pages with FOPEN_KEEP_CACHE. Without it, every from_pretrained re-walks the file via FUSE READ calls, deserialising xorbs from the local cache for each tensor — even when the bytes were already resident from the previous load. Bench (m6i.xlarge, gemma-4-E2B-it 9.6 GiB safetensors, transformers real load with low_cpu_mem_usage=False): | scenario | before | after | |-------------------|---------|---------| | cold model load | 28.28s | 23.07s | | warm model load | 4.30s | 0.81s | | peak RSS | 1060 MB | 1060 MB | Warm reload is now within ~150 ms of pure-disk mmap (0.69s on a local copy of the same model). The cold path also improves: pages fetched during the initial walk stay resident for any re-touch within the load. Read-only check ensures we don't enable this on writable mounts where out-of-band mutation is theoretically possible (the writable path goes through staging anyway, but explicit is safer). No new tests: behavior is observable end-to-end via the open path and was verified on EC2 with a real transformers from_pretrained.
Contributor
Benchmark Results |
Contributor
POSIX Compliance (pjdfstest) |
XciD
force-pushed
the
perf/keep-cache
branch
2 times, most recently
from
April 26, 2026 06:43
479dc77 to
38eb3fc
Compare
Request FUSE_AUTO_INVAL_DATA in init() and gate FOPEN_KEEP_CACHE on either read-only or auto_inval-supported. Read-only mounts already get the cache benefit; writable mounts now get it too on kernels that support the capability (Linux 3.15+), with the kernel auto-invalidating cached pages whenever a file's mtime/size changes. If the kernel lacks AUTO_INVAL_DATA on a writable mount we skip KEEP_CACHE there (current main behavior), so worst case is a no-op.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Two-commit branch making the FUSE kernel page cache survive across opens:
FOPEN_KEEP_CACHEper-file flag: tells the kernel to retain cached pages on open. Without it, everyopen()invalidates the file's pages and forces a full FUSE READ traversal even when bytes are already resident.FUSE_AUTO_INVAL_DATAcapability: lets the kernel auto-invalidate cached pages on mtime/size changes, making KEEP_CACHE safe on writable mounts too.Behavior
--read-only--direct-ioBench
m6i.xlarge, transformers
from_pretrainedon a 9.6 GiB safetensors mount, with the kernel page cache hot from a prior load:The first cold load is unaffected — KEEP_CACHE only helps subsequent reads while the page cache survives. When the kernel cache is dropped (drop_caches, memory pressure, process restart), behavior reverts to the cold path.
Notes
auto_inval_supportedflag is a one-shot atomic written during the FUSE init handshake, read on every open. No hot-path overhead.