Skip to content

Adds support for the new Idempotent Producers feature in Redis 8.6#3926

Merged
petyaslavova merged 9 commits intoredis:masterfrom
elena-kolevska:feat/streams-idempotency
Feb 4, 2026
Merged

Adds support for the new Idempotent Producers feature in Redis 8.6#3926
petyaslavova merged 9 commits intoredis:masterfrom
elena-kolevska:feat/streams-idempotency

Conversation

@elena-kolevska
Copy link
Contributor

@elena-kolevska elena-kolevska commented Jan 29, 2026

Description of change

Adds support for the new Idempotent Producers feature in Redis 8.6

Idempotent Producers ensures exactly-once message delivery semantics for Redis Streams. It prevents duplicate entries from being added to a stream by tracking idempotent IDs (iids) per producer ID (pid). When a duplicate is detected, Redis returns the existing entry ID instead of creating a new entry.

Changes
  1. XADD Command - New Idempotent Parameters ()
idmpauto: Producer ID for automatic idempotent ID calculation.
    Automatically calculates an idempotent ID based on entry content to prevent
    duplicate entries. Can only be used with id='*'. Creates an IDMP map if it
    doesn't exist yet. The producer ID must be unique per producer and consistent
    across restarts.
idmp: Tuple of (producer_id, idempotent_id) for explicit idempotent ID.
    Uses a specific idempotent ID to prevent duplicate entries. Can only be used
    with id='*'. The producer ID must be unique per producer and consistent across
    restarts. The idempotent ID must be unique per message and per producer.
    Shorter idempotent IDs require less memory and allow faster processing.
    Creates an IDMP map if it doesn't exist yet.
  1. New XCFGSET Command - Configures the idempotency parameters for a stream's IDMP map

Sets how long Redis remembers each idempotent ID (iid) and the maximum
number of iids to track. This command clears the existing IDMP map
(Redis forgets all previously stored iids), but only if the configuration
value actually changes.

Args:
    name: The name of the stream.
    idmp_duration: How long Redis remembers each iid in seconds.
        Default: 100 seconds (or value set by stream-idmp-duration config).
        Minimum: 1 second, Maximum: 300 seconds.
        Redis won't forget an iid for this duration (unless maxsize is reached).
        Should accommodate application crash recovery time.
    idmp_maxsize: Maximum number of iids Redis remembers per producer ID (pid).
        Default: 100 iids (or value set by stream-idmp-maxsize config).
        Minimum: 1 iid, Maximum: 1,000,000 (1M) iids.
        Should be set to: mark-delay [in msec] × (messages/msec) + margin.
        Example: 10K msgs/sec (10 msgs/msec), 80 msec mark-delay
        → maxsize = 10 × 80 + margin = 1000 iids.
  1. XINFO STREAM - New response fields
    New fields were added in the response of XINFO STREAM
  • idmp-duration (int) - Configured duration for the stream's IDMP map
  • idmp-maxsize (int) - Configured maxsize for the stream's IDMP map
  • pids-tracked (int) - Number of producer IDs currently tracked
  • iids-tracked (int) - Number of idempotent IDs currently tracked (active, not expired/evicted)
  • iids-added (int) - Cumulative counter of all idempotent entries added during stream lifetime
  • iids-duplicates (int) - Cumulative counter of all duplicate iids detected during stream lifetime

Pull Request check-list

Please make sure to review and check all of these items:

  • Do tests and lints pass with this change?
  • Do the CI tests pass with this change (enable it first in your forked repo and wait for the github action build to finish)?
  • Is the new or changed code fully tested?
  • Is a documentation update included (if this change modifies existing APIs, or introduces new ones)?
  • Is there an example added to the examples folder (if applicable)?

NOTE: these things are not required to open a PR and can be done
afterwards / while the PR is open.

@jit-ci
Copy link

jit-ci bot commented Jan 29, 2026

Hi, I’m Jit, a friendly security platform designed to help developers build secure applications from day zero with an MVS (Minimal viable security) mindset.

In case there are security findings, they will be communicated to you as a comment inside the PR.

Hope you’ll enjoy using Jit.

Questions? Comments? Want to learn more? Get in touch with us.

@elena-kolevska elena-kolevska marked this pull request as ready for review January 29, 2026 18:55
Copy link
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR adds support for Redis 8.6's new Idempotent Producers feature, which ensures exactly-once message delivery semantics for Redis Streams by preventing duplicate entries.

Changes:

  • Added idmpauto and idmp parameters to the XADD command for automatic and explicit idempotent ID handling
  • Implemented new XCFGSET command to configure idempotency parameters (duration and maxsize) for stream IDMP maps
  • Added comprehensive test coverage for the new idempotent features in both sync and async test suites

Reviewed changes

Copilot reviewed 3 out of 3 changed files in this pull request and generated 6 comments.

File Description
redis/commands/core.py Added idempotent parameters to xadd method with validation, implemented xcfgset command, reordered command argument construction
tests/test_commands.py Added comprehensive tests for XINFO STREAM idempotent fields, XADD with idmpauto/idmp, and XCFGSET configuration
tests/test_asyncio/test_commands.py Added async versions of all idempotent producer tests, mirroring sync test coverage

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Signed-off-by: Elena Kolevska <elena@kolevska.com>
Signed-off-by: Elena Kolevska <elena@kolevska.com>
Signed-off-by: Elena Kolevska <elena@kolevska.com>
Signed-off-by: Elena Kolevska <elena@kolevska.com>

Small cleanups after Copilot review

Signed-off-by: Elena Kolevska <elena@kolevska.com>

One more linter fix

Signed-off-by: Elena Kolevska <elena@kolevska.com>

Reformat

Signed-off-by: Elena Kolevska <elena@kolevska.com>
@elena-kolevska elena-kolevska force-pushed the feat/streams-idempotency branch from 4f3f28a to 759e72b Compare January 30, 2026 20:40
@petyaslavova petyaslavova added the feature New feature label Feb 2, 2026
Copy link
Collaborator

@petyaslavova petyaslavova left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM.

@petyaslavova petyaslavova merged commit 2b2f2cf into redis:master Feb 4, 2026
108 of 110 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

feature New feature

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants