Skip to content

Add opt-in latency-aware cluster read balancing - #4212

Draft
lh0156 wants to merge 1 commit into
redis:masterfrom
lh0156:agent/latency-based-cluster-balancing
Draft

Add opt-in latency-aware cluster read balancing#4212
lh0156 wants to merge 1 commit into
redis:masterfrom
lh0156:agent/latency-based-cluster-balancing

Conversation

@lh0156

@lh0156 lh0156 commented Jul 26, 2026

Copy link
Copy Markdown

Fixes #4185

Summary

  • Add opt-in LoadBalancingStrategy.LATENCY_BASED for cluster reads.
  • Select two eligible nodes with power-of-two choices and prefer the lower peak-sensitive EWMA score:
    peak_ewma * (in_flight + 1).
  • Keep existing round-robin and random strategies unchanged.
  • Record per-node latency and in-flight state for sync and asyncio commands.
  • Record MOVED/ASK attempts against the node that handled each attempt.
  • Record one node-level sample for grouped pipeline execution when the batch contains reads.
  • Document the strategy and add a small cluster benchmark.

Implementation notes

The latency state is implemented once in the shared LoadBalancer, so sync and asyncio clients use the same selection and update logic. The selection path is protected by the existing bounded lock; no await is introduced into asyncio selection.

Latency instrumentation is enabled only for the new strategy. Existing strategies do not pay for request timing or state updates.

Verification

  • TDD RED: node-manager tests initially failed because the candidate node list was not supplied to LoadBalancer.
  • TDD GREEN: focused sync tests: 8 passed.
  • Focused asyncio tests: 3 passed.
  • invoke linters: passed.
  • python -m compileall: passed.
  • git diff --check: passed.

The full cluster test files require local Redis cluster services. They were attempted, but this environment has no Redis cluster listening on the test ports, so connection-dependent tests could not complete.

The benchmark is available at benchmarks/cluster_latency_load_balancing.py; it requires a Redis cluster with at least one replica for the selected key slot and reports delayed-replica selection share and p99 latency against round-robin.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

ClusterClient: latency-aware read load balancing (power-of-two-choices + peak-EWMA)

1 participant