Conversation
kjnilsson
force-pushed
the
qq-vNext
branch
4 times, most recently
from
June 2, 2026 09:03
4addd39 to
b9f90f4
Compare
kjnilsson
force-pushed
the
qq-vNext
branch
4 times, most recently
from
June 2, 2026 13:16
07e3478 to
17b01fe
Compare
Contributor
There was a problem hiding this comment.
Pull request overview
This PR bumps the quorum-queue rabbit_fifo state machine to v9 and introduces per-originating-node ingress byte tracking, including a v8 compatibility module to support upgrades from existing machine versions.
Changes:
- Add
ingress_bytes_by_nodeto therabbit_fifomachine state and update enqueue/purge/overview logic to maintain and expose it. - Introduce
rabbit_fifo_v8+rabbit_fifo_v8.hrland update version routing/conversion to support v7→v8→v9 upgrades. - Update Common Test and PropEr suites to use machine version 9 and adjust assertions to query machine overview instead of older query helpers.
Reviewed changes
Copilot reviewed 7 out of 8 changed files in this pull request and generated 4 comments.
Show a summary per file
| File | Description |
|---|---|
| deps/rabbit/src/rabbit_fifo.erl | Bumps machine version to 9, adds ingress tracking + aux-rate estimation, updates conversion chain and overview output. |
| deps/rabbit/src/rabbit_fifo.hrl | Extends the #rabbit_fifo{} record with ingress_bytes_by_node. |
| deps/rabbit/src/rabbit_fifo_v8.erl | New v8 implementation preserved for upgrades and module routing. |
| deps/rabbit/src/rabbit_fifo_v8.hrl | New v8 record/type definitions to keep tuple layout stable for v8 states. |
| deps/rabbit/test/rabbit_fifo_SUITE.erl | Updates conversion tests to v9 and adds ingress-tracking unit tests. |
| deps/rabbit/test/rabbit_fifo_prop_SUITE.erl | Updates machine version to 9 and adds a property for ingress map validity. |
| deps/rabbit/test/rabbit_fifo_dlx_integration_SUITE.erl | Switches assertions to use ra:member_overview/1 machine overview. |
| deps/rabbit/test/quorum_queue_SUITE.erl | Updates purge assertion to use machine overview map. |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
kjnilsson
force-pushed
the
qq-vNext
branch
3 times, most recently
from
June 2, 2026 15:54
6ad4ad9 to
494131c
Compare
kjnilsson
force-pushed
the
qq-vNext
branch
6 times, most recently
from
July 6, 2026 13:47
810e6db to
38f834d
Compare
Implement the OSS infrastructure for ingress-dependent quorum queue leader
rebalancing. This adds:
- Per-node enqueue byte counters in the replicated state machine, updated
deterministically on every enqueue via apply_enqueue/6
- Exposure of cumulative totals via overview/1, propagated to all replicas
on every tick
- Aux state leaky integrators (via ra_li) to compute smoothed per-node
ingress rates, updated on handle_tick
- A get_ingress_rates aux query that returns bytes/second per node,
callable locally without cross-node coordination
The new field ingress_bytes_by_node defaults to #{} and is never read by
OSS, so pre-v9 members in mixed clusters are unaffected. The aux state
version is bumped to aux_v5 with an upgrade clause for rolling restarts.
Decay time is configurable via persistent_term.
Tests added to verify accumulation, snapshot persistence, v8→v9 conversion,
and rate decay behavior under controlled tick timing.
Consumers can now request previously deferred (parked) messages by listing their tokens under the rabbitmq:deferred-tokens FLOW property. rabbit_queue_type gains an optional assign_deferred/4 callback, implemented by quorum queues via rabbit_fifo_client. In rabbit_fifo, a deferral token is only honoured when the client also sets an explicit x-opt-delivery-time; the delayed-retry path never creates a deferred entry, keeping tokens purely client-driven. Advertise support via the rabbitmq:deferred-tokens link capability and document the protocol in deps/rabbit/DEFERRED.md.
Add an "x-opt-index" message annotation to every message read from a quorum queue - via basic.get, normal delivery, or dead-lettering - containing the raft log index the message currently occupies. This lets a client identify exactly which point in the log a message came from, e.g. to correlate it with the queue's own operator-facing raft index.
kjnilsson
force-pushed
the
qq-vNext
branch
2 times, most recently
from
July 15, 2026 10:15
3235634 to
5fa6c53
Compare
consumer_created/updated/deleted rows were recorded on the consuming channel's node, unlike other queue metrics which live on the Ra leader. Cross-node updates went through an erpc:cast that silently dropped on a disconnected node, leaving stale rows with no retry. Move ownership to the leader: the checkout apply clause and state_enter(leader/follower, ...) now write/clear these rows locally, since mod_call effects only ever run on the current leader and must never crash it. Also fan out a one-off cleanup on the v9 machine version bump, to remove stale rows any node may have written under the old scheme.
Cancelling a consumer with unacked messages used to delete its consumer_created row immediately, even though the consumer itself lingers in state until those messages are settled/discarded/returned. Move the delete effect into update_or_remove_con/5, the single place a cancelled consumer is actually dropped from state, so the metrics row disappears exactly when the consumer does rather than up front. down/remove still delete immediately, since those forcibly return all messages and remove the consumer right away.
Comment on lines
+2509
to
2512
| %% deferred entry so that tokens remain a purely client-driven | ||
| %% mechanism. | ||
| DeferralToken = maps:get(<<"x-opt-deferral-token">>, Anns, undefined), | ||
| {true, DeliveryTime, DeferralToken}; |
Comment on lines
+26
to
+29
| %% constants for packed msg references where both the raft index and the size | ||
| %% is packed into a single immidate term | ||
| %% | ||
| %% 59 bytes as immedate ints are signed |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
This branch lays the groundwork for RabbitMQ's next quorum queue release (rabbit_fifo v9) and adds three related features:
Details
ingress decay time is config
Testing