Update Debian init script to use --background flag on start-stop-daemon ...

[cgriego]

...to fork and background instead of just backgrounding.

The change from setsid in 2.7.1 to start-stop-daemon in 2.8.0 introduced a regression, removing a necessary process fork. It's difficult to recreate the scenario where this is needed, but it is reproducible if bootstrapping a RabbitMQ server using Chef and Opscode's RabbitMQ cookbook.

[bitonic]

Not having experience with Chef or Opscode, it is not clear to us what issues are you having. Could you give more details?

Besides, the suggestion is to use the "--background" flag of start-stop-daemon, but the man page warns:

WARNING: start-stop-daemon cannot check the exit status if the process fails to execute for any reason. This is a last resort, and is only meant for programs that either make no sense forking on their own, or where it's not feasible to add the code for them to do this themselves.

The erlang way to daemonise is run_erl, but that creates pipes for the pty which we don't want.

[cgriego]

@bitonic The problem isn't exclusive to Chef, but it is the easiest way I've found to recreate the issue. The problem is one of who the parent of the rabbitmq-server process is. In the Chef bootstrap case without the fix applied, the parent of the process ends up being part of the Chef chain of processes, causing rabbitmq-server to exit when Chef completes running. Here is that hierarchy of processes:

* SSH login shell
** `bash -c` subshell running a script provided as a String
*** chef-client Ruby process
**** Ruby execs a subshell
***** `/usr/sbin/invoke-rc.d rabbitmq-server`
****** Debian magic between invoke-rc.d and init.d script
******* init.d script calls `start-stop-daemon`
******** rabbitmq-server

The & in the init.d script only backgrounds the start-stop-daemon process, but since start-stop-daemon (without the --background) flag, expects the process it is starting to daemonize itself. The --backround flag replaces the & backgrounding and also adds a process fork to ensure the parent of the rabbitmq-server process ends up being init and not one of the many processes listed above. The previous setsid call was performing this fork, but start-stop-daemon needs the --background flag to know that it needs to fork on behalf of the child process.

The --background flag is not optimal, it's definitely preferred for the process to self-daemonize, but it is an improvement over the use of & and the use of rabbitmqctl wait effectively makes up for not being able to check the exit status of rabbitmq-server directly.

[emile]

@cgriego Thanks for your suggestion. We are currently looking for a way to incorporate it without losing stdout and stderr, which contains potentially crucial information. See the mercurial branch named 'bug24930' for commits relating to this.

[cgriego]

@emile Good catch and thanks for the update.

[bitonic]

In the end we "solved" it by writing stdout/err to files in /var/log/rabbitmq/. The change will be included in the next release.

[cgriego]

Excellent. Thank you @bitonic and @emile.