RabbitMQ can go down after restart

[denisenkom]

After restart of rabbitmq server it become inaccessible returning error to clients:

ConnectionError: 541: INTERNAL_ERROR

I found that it was caused by corrupted rabbit_serial file in mnesia directory.

Here is what I have in log:

=SUPERVISOR REPORT==== 28-May-2013::17:44:55 ===
     Supervisor: {local,rabbit_guid_sup}
     Context:    start_error
     Reason:     {'EXIT',
                     {{case_clause,{ok,[]}},
                      [{rabbit_guid,update_disk_serial,0,
                           [{file,"src/rabbit_guid.erl"},{line,64}]},
                       {rabbit_guid,start_link,0,
                           [{file,"src/rabbit_guid.erl"},{line,54}]},
                       {supervisor,do_start_child,2,
                           [{file,"supervisor.erl"},{line,303}]},
                       {supervisor,start_children,3,
                           [{file,"supervisor.erl"},{line,287}]},
                       {supervisor,init_children,2,
                           [{file,"supervisor.erl"},{line,253}]},
                       {gen_server,init_it,6,
                           [{file,"gen_server.erl"},{line,304}]},
                       {proc_lib,init_p_do_apply,3,
                           [{file,"proc_lib.erl"},{line,227}]}]}}
     Offender:   [{pid,undefined},
                  {name,rabbit_guid},
                  {mfargs,{rabbit_guid,start_link,[]}},
                  {restart_type,transient},
                  {shutdown,4294967295},
                  {child_type,worker}]

This error can be reproduced by replacing rabbit_serial file with empty file.

I expect rabbitmq to be able to recover after restart, maybe by replacing invalid rabbit_serial file with some good default?

[emile]

The broker can not recover from arbitrary filesystem corruption or missing files. Please post follow-up questions to the rabbitmq-discuss mailing list.

[denisenkom]

Just for the record, removing corrupted rabbit_serial file helps to recover rabbitmq server in this case