Add async client-side caching support - #4220
Conversation
![chatgpt-codex-connector[bot]](https://avatars.githubusercontent.com/in/1144995?s=60&v=4){kind=small}
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: e77bfb8c07
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
| if is_cacheable and kwargs.get("keys") is None: | ||
| raise ValueError("Cannot create cache key.") |
There was a problem hiding this comment.
With the normal async client path, redis/asyncio/client.py::_send_command_parse_response still calls conn.send_command(*args) and drops **options, so high-level cacheable commands such as await r.get("k") never pass the keys=[...] metadata added by the command mixins. Because GET is cacheable by default, this new check sees no keys and raises ValueError("Cannot create cache key.") before any request is sent, making Redis(cache_config=...) fail for ordinary reads; the sync client forwards **options to send_command, so the async path needs the same behavior.
AGENTS.md reference: AGENTS.md:L145-L148
Useful? React with 👍 / 👎.
There was a problem hiding this comment.
Addressed in 00fd0ac. Async _send_command_parse_response now forwards command options, including cache keys, to the connection. Added a direct forwarding regression test; the asyncio connection/cache suites pass (106 passed, 41 skipped).
| if self._cache.get(self._current_command_cache_key) is not None: | ||
| return |
There was a problem hiding this comment.
With concurrent cache misses for the same key on different pooled connections, the first request leaves an IN_PROGRESS entry. This branch treats any surviving entry as usable and returns without sending the second command; if the first connection already has its reply, the drain above can even consume that response as a pending invalidation. The second caller then awaits read_response() on a connection that never sent a command, and the first caller may have lost its reply, so ordinary concurrent cached reads can time out or leave the entry stuck in progress.
Useful? React with 👍 / 👎.
There was a problem hiding this comment.
Addressed in 00fd0ac. IN_PROGRESS cache entries now expose a completion event. Concurrent async cache misses wait for the original fill, then reuse its value; failed fills delete and signal the entry so waiters retry instead of hanging. Added success and failure concurrency tests.
| if data[1] is None: | ||
| self._cache.flush() | ||
| else: | ||
| self._cache.delete_by_redis_keys(data[1]) |
There was a problem hiding this comment.
When an async invalidation removes cached entries, this branch discards the delete_by_redis_keys() result and never records a client-side-cache eviction, while the sync proxy records invalidation evictions after deleting keys. Async clients will report hits, misses, network savings, and cache size, but invalidation-driven evictions are missing from metrics, making cache observability inaccurate for production async users.
AGENTS.md reference: AGENTS.md:L145-L148
Useful? React with 👍 / 👎.
There was a problem hiding this comment.
Addressed in 00fd0ac. Async invalidation now records CSCReason.INVALIDATION with the number of removed entries, matching the sync proxy behavior. Added a metric regression test.
|
Please make sure resolve Codex comments first |
00fd0ac to
468e0ca
Compare
![cursor[bot]](https://avatars.githubusercontent.com/in/1210556?s=60&v=4){kind=small}
There was a problem hiding this comment.
Cursor Bugbot has reviewed your changes using default effort and found 1 potential issue.
Reviewed by Cursor Bugbot for commit 468e0ca. Configure here.
| await self._conn.disconnect(*args, **kwargs) | ||
|
|
||
| async def send_packed_command(self, command, check_health: bool = True) -> None: | ||
| await self._conn.send_packed_command(command, check_health=check_health) |
There was a problem hiding this comment.
Missing cache key reset in send_packed_command
High Severity
AsyncCacheProxyConnection.send_packed_command does not reset self._current_command_cache_key to None before forwarding the packed command. The sync CacheProxyConnection.send_packed_command explicitly sets self._current_command_cache_key = None (with a comment explaining pre-packed commands are not individually cacheable). Without this reset, a subsequent read_response call could cache the packed command's response under a stale cache key from an earlier send_command, corrupting the cache with wrong data.
Reviewed by Cursor Bugbot for commit 468e0ca. Configure here.
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 468e0caf96
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
| async with self._pool_lock: | ||
| while await cached_entry.connection_ref.can_read(): | ||
| try: | ||
| await cached_entry.connection_ref.read_response( | ||
| push_request=True, |
There was a problem hiding this comment.
In the current code this drain still runs for a VALID cache entry by reading from cached_entry.connection_ref, which can be a different raw connection that the pool has since handed to another coroutine; the pool lock is not held while borrowed connections wait for their own replies. If that coroutine's reply is buffered when a second connection serves this cache hit, this read_response(push_request=True, ...) consumes and discards the normal command reply as if it were an invalidation, leaving the real owner to time out or see a corrupted stream. Only drain the entry's connection when you can prove it is idle/currently owned, or process invalidations on the borrowed connection instead.
Useful? React with 👍 / 👎.
| ) | ||
| ) | ||
|
|
||
| await self._conn.send_command(*args, **kwargs) |
There was a problem hiding this comment.
After setting the shared cache entry to IN_PROGRESS, this await can exit via asyncio.CancelledError while the underlying connection disconnects in its BaseException handler; because the proxy only deletes/signals the entry in read_response(), no completion event is set for this key. A later request for the same cache key then waits forever at the in-flight fill path instead of retrying, which is easy to hit when callers wrap Redis operations in asyncio.wait_for or cancel tasks.
Useful? React with 👍 / 👎.
| self._current_command_cache_key = CacheKey( | ||
| command=args[0], | ||
| redis_keys=tuple(kwargs["keys"]), | ||
| redis_args=args, |
There was a problem hiding this comment.
When a cacheable command passes a single key as a scalar, this conversion splits it into characters/integers instead of storing the Redis key; for example zrevrange() sets options["keys"] = name, so name="myzset" becomes ('m', 'y', ...) (or byte integers). Redis invalidations arrive for b"myzset", which will not match that tuple, so async caching can keep returning stale ZREVRANGE results after the sorted set is modified. Normalize scalar str/bytes keys to a one-item tuple before building CacheKey.
AGENTS.md reference: AGENTS.md:L158-L162
Useful? React with 👍 / 👎.
| if (cache_config or cache) and check_protocol_version(protocol, 3): | ||
| kwargs.update( | ||
| { | ||
| "cache": cache, | ||
| "cache_config": cache_config, | ||
| } | ||
| ) |
There was a problem hiding this comment.
Because this cache wiring is nested only under the TCP branch, Redis(unix_socket_path=..., protocol=3, cache_config=...) silently constructs a pool with cache is None even though the user explicitly requested client-side caching and the RESP3 validation below still succeeds. Direct async ConnectionPool(path=..., connection_class=UnixDomainSocketConnection, cache_config=...) does create the proxy, so the high-level async constructor should pass the cache/cache_config kwargs for Unix sockets as well or reject the combination instead of disabling caching.
Useful? React with 👍 / 👎.
| await self._conn.read_response( | ||
| push_request=True, | ||
| timeout=0, | ||
| disconnect_on_error=False, |
There was a problem hiding this comment.
When a pending invalidation frame is only partially buffered, async read_response(timeout=0) returns None on the user-requested timeout rather than raising TimeoutError, but this loop only breaks on the exception. Because can_read() remains true while the partial bytes stay buffered, a cached async client can spin here and starve the event loop before the rest of the frame arrives; treat a None result like no complete push was available.
AGENTS.md reference: AGENTS.md:L145-L148
Useful? React with 👍 / 👎.
2 participants
Fixes #3916
What changed
The synchronous client already supports RESP3 client-side caching, but the async standalone client did not expose cache or cache_config and its async connection pool could not wrap connections for tracking and invalidation handling.
This change:
Verification
Note
Medium Risk
New async connection-layer caching affects all command I/O on pooled connections (tracking, invalidation, locking, and in-flight deduplication); mistakes could cause stale reads or connection desync, though parity with the existing sync proxy and broad unit/integration tests reduce risk.
Overview
Adds client-side caching (CSC) to the async standalone
Redisclient, matching behavior already available on the sync client.redis.asyncio.Redisnow acceptscacheandcache_config, enforces RESP3-only configuration, and exposesget_cache(). The asyncConnectionPoolbuilds the shared cache (custom implementation, factory, orCacheConfig), registers CSC observability callbacks, and wraps each connection in a newAsyncCacheProxyConnectionthat enablesCLIENT TRACKING, serves cache hits/misses, coordinates in-flight fills viacompletion_eventonCacheEntry, and processes invalidation push messages asynchronously.Command path:
_send_command_parse_responseforwards**options(includingkeysfor cacheable commands) tosend_command. Connections storehandshake_metadatafrom HELLO for Redis 7.4+ validation on connect.Docs in
resp3_features.rstdocument async CSC usage and clarify supported client types (async + sync cluster/sentinel, not async cluster). Tests cover pool wiring, RESP2 rejection, proxy hit/miss/invalidation/in-flight behavior, and an integration round-trip on Redis ≥ 7.4.Reviewed by Cursor Bugbot for commit 468e0ca. Bugbot is set up for automated code reviews on this repo. Configure here.