Compile OpenSSL & Erlang/OTP from source, use Ubuntu as the base image

[gerhard]

Sharing this early so that we can discuss the direction that this is going in. Various decisions made in the Dockerfile have been captured inline in comments, should make for a good PR discussion.

The primary goal is to upgrade the Erlang/OTP version to latest stable,
which is v21.2.2 at the time of this commit. RabbitMQ v3.7.x will stop
supporting Erlang v20.x in September 2019 (~8 months from now). RabbitMQ
v3.8.x will only support Erlang v21.x. RabbitMQ Erlang/OTP release
support policy was announced on the rabbitmq-users mailing list in
October 2018:
https://groups.google.com/forum/#!msg/rabbitmq-users/G4UJ9zbIYHs/tyt_kDoFBgAJ

The secondary goal is to only ship the required artefacts in the final
image. For example, all Erlang/OTP applications & features which are not
required by RabbitMQ are disabled. I suspect that the final Erlang/OTP
release can be shrunk further (it currently stands 130MB), but this is a
minor concern right now.

Related to the secondary goal, we enable certain features in Erlang/OTP
which are useful when debugging:

• extra microstate accounting is not known to negatively affect
  performance, this feature is exposed via the rabbitmq-diagnostics runtime_thread_stats command added in v3.7.10
• lock counting is only enabled if the Erlang VM is started in a
  specific mode, this feature doesn't impact the default beam.smp runtime

The final goal is to be explicit about the OpenSSL version that
Erlang/OTP uses. Using a shared OpenSSL might be convenient, but it
has the following drawbacks:

• depending on the base image for OpenSSL updates
• not knowing which OpenSSL version we compile against
• not knowing how OpenSSL was configured
• not being able to change OpenSSL configuration

Compiling OpenSSL adds an extra concern and definitely complicates this
Dockerfile, but one possible mitigation would be to automate version
bumps when a new OpenSSL version gets published. I am also expecting
that images will be automatically built & published from this
Dockerfile. Since OpenSSL is compiled with all defaults, I do not expect
things to stop working and become a maintenance overhead - we are not
using any advanced compilation flags.

I am including the full docker build log, I always find the information
captured in build logs to be helpful.

Resources which I found helpful while putting this Dockerfile together:

• https://github.com/rabbitmq/rabbitmq-server-boshrelease/blob/816bb377a59975c461e1af72367f187edc39ad3d/packages/erlang-21.1/packaging
• https://github.com/erlang/docker-erlang-otp/blob/e2e804aeeb6e6bc5fd49f66481be1dff829428f5/21/Dockerfile
• https://github.com/erlang/docker-erlang-example#2-build-stage-1-create-a-minimal-docker-image
• https://bugs.erlang.org/browse/ERL-823
• http://erlang.org/pipermail/erlang-questions/2019-January/097012.html
• https://github.com/lrascao/erlang-ec2-build
• https://github.com/kerl/kerl/blob/master/kerl

[gerhard]

This started as #277. People which are not on the initial issue that might be interested in this: @lukebakken @acogoluegnes @MarcialRosales @mkuratczyk

[gerhard]

I've just noticed that the Debian failing builds are related to OpenSSL & the Alpine one is related to missing https://github.com/rabbitmq/rabbitmq-server/releases/tag/v3.7.8-rc.4 files.

[lukebakken]

Based on the last two comments in #277, shouldn't Debian be the base image? Ubuntu is referenced in the title.

[gerhard]

@lukebakken the head of the Dockerfile has the answer 😉

[tianon]

I'm definitely +1 on using Ubuntu instead of Debian if it's something better supported by RabbitMQ upstream (:100:), although unfortunately, multi-stage builds are going to be a no-go here: docker-library/official-images#3383

Does this also mean you'll be planning to deprecate the Alpine-based variants? I'm in favor of that if you (as in upstream) don't want to commit to actually supporting Alpine, but do note that you'll definitely have some, ahem, dissatisfied and likely vocal users who enjoy the smaller image size. 😅

[gerhard]

I didn't understand from the issue linked above why multi-stage builds are a no-go. Everything works as expected locally (macOS 10.14.2, Docker Desktop 2.0.0.0-mac81), the final image is ~250MB currently, still working on the RabbitMQ part.

We haven't decided yet what will happen with Alpine-based variants. We are 100% behind Ubuntu Bionic, even at a company level, it makes sense to focus our efforts there. In other words, we are happy to put the proverbial stamp on the Bionic-based RabbitMQ images. Those that value this endorsement over image size will have an easy time choosing 😉

[tianon]

I didn't understand from the issue linked above why multi-stage builds are a no-go.

That'd be because we don't docker build the images here directly (except for testing) -- those go through the Docker standard "Official Images" process outlined in https://github.com/docker-library/official-images, and build on that infrastructure via a separate tool which doesn't support multi-stage builds and isn't trivial to update to support multi-stage builds.

I'm happy to help convert this into a non-multi-stage build (and still have it clean up, just within one layer so the cleanup actually takes effect) once you get something you're happy with, though. 👍

Those that value this endorsement over image size will have an easy time choosing 😉

🤘 SGTM

[gerhard]

This is now clear, thank you!

[gerhard]

@tianon the current ubuntu Dockerfile is there from the RabbitMQ/Erlang/OpenSSL perspective. It's missing gosu, docker-entrypoint, management as well as the non multi-stage build approach. The current image size stands at 189MB. It's not great, but it doesn't feel terrible either. It's something that we can improve on, for sure, but in a new context - don't want to drag this out longer than it's necessary.

I will lift the missing bits from debian/3.7 and leave the multi-stage build approach to you. Let me know if you have something else in mind.

[gerhard]

@tianon @yosifkit this is now ready for your review.

FWIW, the Erlang version that this PR installs for RabbitMQ 3.7 is not compatible with RabbitMQ 3.6, as per the official RabbitMQ Erlang Version Requirements. Since RabbitMQ 3.6 minor is EOL since May 2018, I will submit a separate PR to remove it.

[gerhard]

• Travis builds are passing
• all comments resolved
• non-multistage builds
• update https://github.com/docker-library/docs/tree/master/rabbitmq

[michaelklishin]

@tianon @yosifkit we believe this is ready for a final review :)

[michaelklishin]

@tianon @yosifkit friendly reminder that we believe this is ready to be reviewed 👋 🤓

[yosifkit]

I've been working off and on over the past few days to merge it into a single stage build. I think it is mostly ready, but now I have @tianon giving it a once over for the things I forgot.

I think my last build was 181MB, so hopefully I didn't drop something important.

[gerhard]

I think my last build was 181MB, so hopefully I didn't drop something important.

That is great news! If it helps the effort, happy to review what you have so far 😉

[yosifkit]

Commits were pushed here. Let us know what you think.

[michaelklishin]

@yosifkit this is a good starting point. I have some ideas about how to improve sanity checking of TLS/crypto. We once had them built without any EC curves in our RPM image by mistake, for example, and that wasn't obvious. I don't think those ideas should delay this or any other PRs we have open.

[gerhard]

Started reviewing f22c0b2, leaving inline comments.

It felt convenient and appropriate to use alpine:edge for the Alpine variant, which would ensure that the latest Erlang/OTP & openssl were installed, without needing to compile them from source. Happy either way, this is ready to 🚢

[yosifkit]

Yeah, from previous experience, we don't want to move images to be FROM alpine:edge.

End users should not use "edge" as their main day-to-day workstation or as productive system. Because "edge" is a development branch, many changes are not heavily tested (or tested at all) and packages in "edge" can and sometimes do break without warning.

- https://wiki.alpinelinux.org/wiki/Edge

This looks good to me; we can get the bumps via the update.sh bot.

[gerhard]

What needs to happen for this change to make it through to https://hub.docker.com/_/rabbitmq?tab=tags ?

I can see that update.sh build 3466 succeeded, but I can't figure out how/when the platform-specific builds will run, such as multiarch/amd64/rabbitmq.

[bfink13]

I think this broke rabbitmqctl wait as detailed in #162. The fix was merged in #163 (install procps) but that step was removed in this merge.

I'm not entirely sure where in the Dockerfile the procps install should go - can anyone advise?