feat(redis): add Redis Sentinel support#2895
Conversation
|
Hi, Thanks for the hard work. Seeing the PR for redis cluster, I wonder if it would not be "better" to create a RedisSentinelBroker than to add parameters to the generic RedisBroker. It would prevent these param to be available in redisCluster as well, and it would somehow keep the same logic where you just switch from one broker to the other one when changing technology. What do you think ? |
Add connection-level Redis Sentinel support to RedisBroker: - New broker params: sentinels, sentinel_master_name, sentinel_kwargs. - RedisConnectionState builds the client via Sentinel.master_for() when configured, so the SentinelConnectionPool re-discovers the current master on reconnect. Publishers and stream consumers fail over for free, since both go through connection.client. - Stream subscriber now backs off on fetch errors like the base channel/list loop (shared CONSUME_ERROR_BACKOFF_SECONDS), avoiding a busy-loop during a master failover. - Unit tests (no running Sentinel needed) and a redis_sentinel marker. Refs ag2ai#1812
RedisRouter (FastAPI integration) now accepts sentinels / sentinel_master_name / sentinel_kwargs and forwards them to the underlying RedisBroker, so FastAPI + FastStream applications get Sentinel support too (previously only the standalone RedisBroker did). Adds unit tests asserting the router forwards config to broker.
…Broker Per review feedback on ag2ai#2895, Sentinel support now lives in its own RedisSentinelBroker / RedisSentinelRouter instead of extra parameters on the generic RedisBroker — mirroring RedisClusterBroker (ag2ai#2854), so each topology is a distinct broker class and Sentinel params no longer leak into RedisBroker / RedisClusterBroker. - RedisSentinelConnectionState builds the client via Sentinel.master_for(); the generic RedisBroker / RedisRouter keep their original API. - RedisSentinelParams holds sentinels / sentinel_master_name / sentinel_kwargs. - RedisSentinelRouter adds FastAPI Sentinel support (which RedisClusterBroker does not provide). - Docs: docs/en/redis/sentinel.md + navigation entry. Refs ag2ai#1812
cca8a33 to
1a7226b
Compare
|
Thanks for the suggestion — agreed. I've reworked it into a dedicated |
Summary
Adds connection-level Redis Sentinel support to
RedisBroker, addressing #1812.What changed
RedisConnectionParams):sentinels,sentinel_master_name,sentinel_kwargs. Extracted inRedisBroker.__init__; a missingsentinel_master_nameraisesSetupError. Excluded from cluster params.RedisConnectionStategains an optionalSentinelConfig. When set,connect()builds the client viaredis.asyncio.sentinel.Sentinel(...).master_for(...)instead of a plainConnectionPool. The resultingSentinelConnectionPoolre-discovers the current master on every reconnect, so publishers and stream (XREADGROUP) consumers fail over for free — both already go throughconnection.client._consumeloop had no backoff in itsexcept(unlike the base channel/list loop), so it would hot-loop on a persistentConnectionError(e.g. during a master failover). Extracted the existing5s backoff into a sharedCONSUME_ERROR_BACKOFF_SECONDSand applied it in both loops.SentinelConfig, that direct mode is unaffected, validation, and thatconnect()actually yields aSentinelConnectionPool. Newredis_sentinelpytest marker.mypy --strict,ruff check,ruff format --checkand the new unit tests pass locally.Scope / follow-ups
This is the connection-level MVP outlined in #1812 (publishers + simple commands + streams). Intentionally not in this PR, to keep it focused:
pubsub()subscription; rebuilding and resubscribing it on failover is a separate change and is left as a follow-up. (Stream consumers recover because eachxreadgroupre-acquires a connection from the Sentinel pool.)redis-clustersetup). The tests here are unit-level.Refs #1812.