Repository navigation
Conversation
We now expose XAUTOCLAIM through RedisStreamCommands and ReactiveStreamCommands with implementations for both Lettuce and Jedis. XAutoClaimOptions carries the min-idle-time, the start id and an optional COUNT. Results are returned as ClaimedRecords and ClaimedRecordIds. Both hold the next cursor alongside the claimed records so that callers can resume the scan, a cursor of 0-0 indicating that the pending entries list has been scanned completely. The reactive variants return Mono rather than Flux for the same reason. The ids of pending entries that no longer exist in the stream, the third reply element added in Redis 7.0, are not exposed. Neither Lettuce 7.7.0 nor Jedis 8.0.1 report them today. See spring-projects#3434 Signed-off-by: dae won <eodnjs01477@gmail.com>
StreamOperations and ReactiveStreamOperations now offer autoClaim and autoClaimJustId next to the existing claim methods, applying the configured serializers to keys and records the same way. Closes spring-projects#3434 Signed-off-by: dae won <eodnjs01477@gmail.com>
renechoi
left a comment
There was a problem hiding this comment.
Ran the branch against Redis 8.6.2 with the pinned drivers. The Lettuce comment on xAutoClaimJustId describes something users will hit: the two drivers return different ids once a claimed entry no longer exists in the stream.
3 entries pending for c1, middle one XDELed, then autoclaim to c2:
- Lettuce 7.7.0 ->
[a, c, b], andbis gone from the stream - Jedis 8.0.1 ->
[a, c]
redis/lettuce#3901 is merged but not in 7.7.0. Both new JUSTID tests use a single entry and no XDEL, so they stay green either way, while getIds() is documented as the ids that were claimed.
Thanks for running it, I reproduced exactly what you describe. Three entries pending for
You are right that the tests do not catch it. All four JUSTID tests use a single entry and no This cannot be fixed inside Spring Data Redis. With redis/lettuce#3901 fixes this along with adding |
Spring Data Redis exposes
XCLAIMbut notXAUTOCLAIM. Recovering unacknowledged messages from a failed consumer currently requires either issuingXPENDINGfirst and passing the ids toclaim, or unwrappingRedisConnection#getNativeConnection()to call the underlying driver directly. The latter ties the recovery logic to a specific driver and bypasses the serializers configured on the template.Both driver versions declared in
pom.xmlalready support the command, so this change is only about exposing it: Lettuce 7.7.0 throughxautoclaim(K, XAutoClaimArgs<K>)returningClaimedMessages, and Jedis 8.0.1 throughxautoclaimandxautoclaimJustId.XAutoClaimOptionsholds the min-idle-time, the start id and an optionalCOUNT, mirroring howXClaimOptionssits next toxClaim.Return type
XAUTOCLAIMreturns a cursor in addition to the claimed entries, and a cursor of0-0is what tells the caller the scan is finished. Returning a bareListorFluxthe wayclaimdoes would discard that, so the command returnsClaimedRecords(cursor plus records) andClaimedRecordIdsfor theJUSTIDvariant. Both implementStreamable, likePendingMessages. For the same reason the reactive methods returnMono<ClaimedRecords<...>>rather than aFluxof records. This was the open design question in #3434, so feedback on the shape is welcome.Out of scope
The third reply element added in Redis 7.0, the ids of pending entries that no longer exist in the stream, is not exposed. Neither driver reports it today. The Lettuce side is fixed in redis/lettuce#3901 and ships in 7.8, so I left it out rather than add an accessor that is always empty.
ClaimedRecordsandClaimedRecordIdscan take the extra list later without changing the existing signatures.Driver coverage
Both Lettuce and Jedis, imperative and reactive:
LettuceStreamCommands,LettuceReactiveStreamCommands, andJedisStreamCommands(inherited byJedisClusterStreamCommands), with conversion handled in the respectiveStreamConverters. The Jedis converter filters out nil entries, which Redis versions before 7.0 return in place of pending entries whose message was deleted.Backward compatibility
xAutoClaimandxAutoClaimJustIdare declared as abstract methods onRedisStreamCommands,ReactiveStreamCommandsandStringRedisConnection, following the pattern set byxAckDelandxDelExin #3247. Custom implementations of those interfaces will need to implement the new methods.Documentation and commits
No update to
appendix.adocis necessary, asXAUTOCLAIMis already listed there as supported, which this change makes true.Split into two commits, the connection contract first and the template API that builds on it second, to make review easier. Each commit carries its own tests and compiles on its own. Happy to squash if you prefer a single commit.
Verification
Verified locally against Redis 8.6.2 via
make startacross both drivers, standalone and cluster, imperative and reactive, including the pipeline and transaction paths. 3003 tests acrossDefaultStreamOperationsIntegrationTests,DefaultReactiveStreamOperationsIntegrationTests,LettuceReactiveStreamCommandsIntegrationTests, theAbstractConnectionIntegrationTestssubclasses, the converter unit tests and theDefaultStringRedisConnectionunit tests.The two failures observed,
xAckDelShouldAcknowledgeAndDeleteEntriesandxAckDelWithStringIdsShouldWork, reproduce on unmodifiedmainin the same environment and are unrelated to this change. New tests cover the claim itself, cursor pagination withCOUNTacross two calls until the cursor returns to0-0, theJUSTIDvariant, and the raw reply shapes of both drivers. Since this introduces new API, those tests do not compile before the change rather than failing against it.Closes #3434