Skip to content

QQ: v9#16583

Draft
kjnilsson wants to merge 6 commits into
mainfrom
qq-vNext
Draft

QQ: v9#16583
kjnilsson wants to merge 6 commits into
mainfrom
qq-vNext

Conversation

@kjnilsson

@kjnilsson kjnilsson commented Jun 1, 2026

Copy link
Copy Markdown
Contributor

Summary

This branch lays the groundwork for RabbitMQ's next quorum queue release (rabbit_fifo v9) and adds three related features:

  • Raft index message annotation — every message read from a quorum queue is now stamped with an x-opt-index annotation containing the raft log index it currently occupies, so clients/operators can correlate a specific message with the queue's raft log position
  • Deferred message tokens over AMQP 1.0 — consumers can park a message under a client-chosen token (via the MODIFIED outcome's x-opt-deferral-token + x-opt-delivery-time annotations) and later pull it back on demand by listing tokens in a FLOW frame's rabbitmq:deferred-tokens property, instead of waiting on a delayed-retry timer. Support is advertised via the rabbitmq:deferred-tokens link capability and documented in deps/rabbit/DEFERRED.md
  • Per-node ingress byte tracking — rabbit_fifo now tracks enqueue bytes per node in the replicated state machine, exposes cumulative totals via overview/1, and computes smoothed per-node ingress rates (via ra_li leaky integrators) in aux state.
  • Leader-Owned Consumer Metrics — consumer_created/updated/deleted rows for quorum queue consumers are now recorded on the current Ra leader instead of the consuming channel's node, matching how other queue metrics are owned; a cancelled consumer's metrics are no longer deleted immediately if it still has unsettled messages, only once they're settled, discarded, or returned

Details

  • rabbit_fifo version is bumped to 9, with the prior v8 implementation preserved as rabbit_fifo_v8 (new module) so that v7 -> v8 conversion logic can be split out cleanly; v8 -> v9 conversion is currently a no-op
  • rabbit_queue_type gains anallback, implemented forquorum queues via rabbit_fifo_client
  • A deferral token is only hsets an explicitx-opt-delivery-time — the delayed-retry path never creates a deferred entry, keeping tokens purely client-driven
  • Aux state version bumped to aux_v5 with an upgrade clause for rolling restarts;
    ingress decay time is config

Testing

  • New/updated coverage in am_SUITE, rabbit_fifo_SUITE,rabbit_fifo_dlx_integration_SUITE, and rabbit_fifo_prop_SUITE for raft index annotation, deferral token aaccumulation/snapshotting/rate-decay under controlled tick timing

@kjnilsson
kjnilsson force-pushed the qq-vNext branch 4 times, most recently from 4addd39 to b9f90f4 Compare June 2, 2026 09:03
@kjnilsson kjnilsson changed the title QQ: v9 - module setup. QQ: v9 Jun 2, 2026
@kjnilsson
kjnilsson force-pushed the qq-vNext branch 4 times, most recently from 07e3478 to 17b01fe Compare June 2, 2026 13:16
@kjnilsson
kjnilsson requested a review from Copilot June 2, 2026 13:43

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR bumps the quorum-queue rabbit_fifo state machine to v9 and introduces per-originating-node ingress byte tracking, including a v8 compatibility module to support upgrades from existing machine versions.

Changes:

  • Add ingress_bytes_by_node to the rabbit_fifo machine state and update enqueue/purge/overview logic to maintain and expose it.
  • Introduce rabbit_fifo_v8 + rabbit_fifo_v8.hrl and update version routing/conversion to support v7→v8→v9 upgrades.
  • Update Common Test and PropEr suites to use machine version 9 and adjust assertions to query machine overview instead of older query helpers.

Reviewed changes

Copilot reviewed 7 out of 8 changed files in this pull request and generated 4 comments.

Show a summary per file
File Description
deps/rabbit/src/rabbit_fifo.erl Bumps machine version to 9, adds ingress tracking + aux-rate estimation, updates conversion chain and overview output.
deps/rabbit/src/rabbit_fifo.hrl Extends the #rabbit_fifo{} record with ingress_bytes_by_node.
deps/rabbit/src/rabbit_fifo_v8.erl New v8 implementation preserved for upgrades and module routing.
deps/rabbit/src/rabbit_fifo_v8.hrl New v8 record/type definitions to keep tuple layout stable for v8 states.
deps/rabbit/test/rabbit_fifo_SUITE.erl Updates conversion tests to v9 and adds ingress-tracking unit tests.
deps/rabbit/test/rabbit_fifo_prop_SUITE.erl Updates machine version to 9 and adds a property for ingress map validity.
deps/rabbit/test/rabbit_fifo_dlx_integration_SUITE.erl Switches assertions to use ra:member_overview/1 machine overview.
deps/rabbit/test/quorum_queue_SUITE.erl Updates purge assertion to use machine overview map.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Comment thread deps/rabbit/src/rabbit_fifo.erl Outdated
Comment thread deps/rabbit/test/rabbit_fifo_SUITE.erl
Comment thread deps/rabbit/test/rabbit_fifo_SUITE.erl
Comment thread deps/rabbit/test/rabbit_fifo_prop_SUITE.erl
kjnilsson added 4 commits July 6, 2026 14:48
Implement the OSS infrastructure for ingress-dependent quorum queue leader
rebalancing. This adds:

- Per-node enqueue byte counters in the replicated state machine, updated
  deterministically on every enqueue via apply_enqueue/6
- Exposure of cumulative totals via overview/1, propagated to all replicas
  on every tick
- Aux state leaky integrators (via ra_li) to compute smoothed per-node
  ingress rates, updated on handle_tick
- A get_ingress_rates aux query that returns bytes/second per node,
  callable locally without cross-node coordination

The new field ingress_bytes_by_node defaults to #{} and is never read by
OSS, so pre-v9 members in mixed clusters are unaffected. The aux state
version is bumped to aux_v5 with an upgrade clause for rolling restarts.

Decay time is configurable via persistent_term.

Tests added to verify accumulation, snapshot persistence, v8→v9 conversion,
and rate decay behavior under controlled tick timing.
Consumers can now request previously deferred (parked) messages by
listing their tokens under the rabbitmq:deferred-tokens FLOW property.
rabbit_queue_type gains an optional assign_deferred/4 callback,
implemented by quorum queues via rabbit_fifo_client. In rabbit_fifo,
a deferral token is only honoured when the client also sets an
explicit x-opt-delivery-time; the delayed-retry path never creates a
deferred entry, keeping tokens purely client-driven.

Advertise support via the rabbitmq:deferred-tokens link capability
and document the protocol in deps/rabbit/DEFERRED.md.
Add an "x-opt-index" message annotation to every message read from a
quorum queue - via basic.get, normal delivery, or dead-lettering -
containing the raft log index the message currently occupies. This
lets a client identify exactly which point in the log a message came
from, e.g. to correlate it with the queue's own operator-facing raft
index.
@kjnilsson
kjnilsson force-pushed the qq-vNext branch 2 times, most recently from 3235634 to 5fa6c53 Compare July 15, 2026 10:15
consumer_created/updated/deleted rows were recorded on the consuming
channel's node, unlike other queue metrics which live on the Ra
leader. Cross-node updates went through an erpc:cast that silently
dropped on a disconnected node, leaving stale rows with no retry.

Move ownership to the leader: the checkout apply clause and
state_enter(leader/follower, ...) now write/clear these rows locally,
since mod_call effects only ever run on the current leader and must
never crash it. Also fan out a one-off cleanup on the v9 machine
version bump, to remove stale rows any node may have written under
the old scheme.
Cancelling a consumer with unacked messages used to delete its
consumer_created row immediately, even though the consumer itself
lingers in state until those messages are settled/discarded/returned.

Move the delete effect into update_or_remove_con/5, the single place
a cancelled consumer is actually dropped from state, so the metrics
row disappears exactly when the consumer does rather than up front.
down/remove still delete immediately, since those forcibly return all
messages and remove the consumer right away.

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 13 out of 14 changed files in this pull request and generated 2 comments.

Comment on lines +2509 to 2512
%% deferred entry so that tokens remain a purely client-driven
%% mechanism.
DeferralToken = maps:get(<<"x-opt-deferral-token">>, Anns, undefined),
{true, DeliveryTime, DeferralToken};
Comment on lines +26 to +29
%% constants for packed msg references where both the raft index and the size
%% is packed into a single immidate term
%%
%% 59 bytes as immedate ints are signed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants