Fix RuntimeError in ClusterPubSub sharded message generator#3889
Fix RuntimeError in ClusterPubSub sharded message generator#3889petyaslavova merged 4 commits intomasterfrom
Conversation
There was a problem hiding this comment.
Pull request overview
This PR fixes a RuntimeError: dictionary changed size during iteration that occurs when subscribing to multiple sharded channels across different cluster nodes. The issue arose because the generator was iterating directly over node_pubsub_mapping.values() while concurrent subscriptions could modify the dictionary.
Key changes:
- Modified
_pubsubs_generator()to create a snapshot of pubsub instances before iteration - Introduced cycle boundaries using
Nonemarkers to properly track iteration completion - Added comprehensive tests validating multi-node sharded pubsub behavior
Reviewed changes
Copilot reviewed 2 out of 2 changed files in this pull request and generated 2 comments.
| File | Description |
|---|---|
| redis/cluster.py | Fixed generator to snapshot mapping values before iteration and added cycle markers |
| tests/test_pubsub.py | Added three cluster-specific tests for multi-node sharded pubsub scenarios |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
d76f170 to
50f8ab3
Compare
There was a problem hiding this comment.
Pull request overview
Copilot reviewed 2 out of 2 changed files in this pull request and generated 3 comments.
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
Fix RuntimeError in ClusterPubSub sharded message generator
Problem:
RuntimeError: dictionary changed size during iterationwhen subscribing to multiple sharded channels on different cluster nodes.Solution: Create a snapshot of node_pubsub_mapping.values() before iteration in _pubsubs_generator() to prevent errors from concurrent modifications.
Changes:
Fixed _pubsubs_generator() in ClusterPubSub class
Added 3 cluster-only tests validating multi-node sharded pubsub and the concurrent mapping changes fix