New experimental cache does not seem to cache negative results

[jrudolph]

I'm trying out the new experimental CacheStore and am wondering if it supports caching negative results? We are trying to use it against sparse arrays written with write_empty_chunks: False. From a cursory look at the CacheStore implementation it seems that a negative (None) result might remove a cache entry instead of caching an empty/negative result.

When using fixed sparse zarr arrays caching negative entries would be very helpful. There are dynamic use cases where arrays might still change where you'd like to have different behaviors but the default should not have worst performance is sparse regions.

[d-v-b]

this is a great point -- what API would you like to see here? What makes this kind of interesting is that the "cache the absence of a key" behavior is specific to chunk keys: we probably do not want to cache the absence of a metadata document.

[maxrjones]

this is a great point -- what API would you like to see here? What makes this kind of interesting is that the "cache the absence of a key" behavior is specific to chunk keys: we probably do not want to cache the absence of a metadata document.

🤔 caching the absence of a metadata document could help for applications that try to get consolidated metadata and fall back to non-consolidated, or try and fallback for zarr V3 vs V2 metadata (e.g., Xarray).