Consistent hash exchange should keep same distribution after server restart

[acogoluegnes]

See https://groups.google.com/d/msg/rabbitmq-users/qHHptwZoxj8/XQbgGa8XBgAJ to reproduce.

[acogoluegnes]

The $2 number that is matched against the hash to choose the bucket is regenerated and "re-randomized" after each start. That's why buckets aren't consistent across restarts.

[michaelklishin]

This has been the case from day one.

[acogoluegnes]

Weird nobody noticed before today.

[rai69]

I see that this issue is closed a long time ago, but with version 3.8.2 of RabbitMQ and the consistent-hash-exchange plugin I can reproduce the problem that after a restart the distribution is not always the same. Sometime it is, sometimes it isn't. Would you like me to file a new issue or provide information here?

[michaelklishin]

We would like to have a specific set of steps to reproduce. We won't investigate without one. The plugin has changed significantly since 2017, see #37 for example.

[michaelklishin]

One thing that can still affect the plugin even after #37 is binding changes between restarts or application deployments. Ultimately it's the applications that control the state of the consistent hashing ring via bindings, and changes to it can lead to changes in bucket distribution. Every binding has a "weight", for example.

Please put together an executable test with a set of steps that describes any administrative actions taken on the node (e.g. when it is restarted) and start a mailing list thread.

[rai69]

@michaelklishin I will be providing this today

[rai69]

My test is executed with a dotnet application that can be found here: https://github.com/rai69/rabbitmq-consisten-hash-test

Setup RabbitMQ

• An exchange of type topic named exchange-in
• An exchange of type x-consistent-hash named exchange-consumer connected to the exchange-in exchange
• 5 queues named consumer.0 through consumer.4 with a binding to exchange-consumer an

Test
To execute the test, following the following steps multiple times:

1. execute dotnet run .
2. examine the number of messages in the consumer queueus and make note of the number per queue
3. on the rabbitmq instance call: rabbitmqctl stop_app
4. on the rabbitmq instance call: rabbitmqctl stop_app
5. examine the number of messages in the consumer queueus these should be all 0
6. repeat from step 1 until the values in step 2 are different for 1 or more queues

Results
Initial run

After a few test runs

Problem
After a restart it can't be garanteed that messsages will be sent to the same queue.