Static compilation of rust binaries using musl

[smklein]

Hiya, this is more a platform for discussion than anything else. I wanted to post this issue here because I'm interested in contributing to it, but I wanted some guidance first.

TL;DR: I'd like to be able to statically link rust binaries by compiling with musl.

Following the instructions here: https://doc.rust-lang.org/edition-guide/rust-2018/platform-and-target-support/musl-support-for-fully-static-binaries.html , it seems the x86_64-unknown-linux-musl target is what I should be using.

I've tried compiling the examples workspace with the following:

load("@io_bazel_rules_rust//rust:repositories.bzl", "rust_repository_set") 
                                                    
rust_repository_set(                                
    name = "rust_linux_x86_64",                                                                          
    exec_triple = "x86_64-unknown-linux-musl",
    extra_target_triples = ["wasm32-unknown-unknown"],
    version = "1.44.0",                             
    edition = "2018",                                                                                    
)         

But this has resulted in output like the following:

$ cd examples && bazel build //hello_world/...
INFO: Analyzed 2 targets (0 packages loaded, 0 targets configured).
INFO: Found 2 targets...
INFO: From Compiling Rust rlib hello_lib (2 files):
process wrapper error: failed to exec the new process: No such file or directory.
ERROR: /home/smklein/repos/rules_rust/examples/hello_lib/BUILD:11:13: output 'hello_lib/libhello_lib-590739884.rlib' was not created
ERROR: /home/smklein/repos/rules_rust/examples/hello_lib/BUILD:11:13: not all outputs were created or valid
INFO: Elapsed time: 0.259s, Critical Path: 0.05s
INFO: 1 process: 1 linux-sandbox.
FAILED: Build did NOT complete successfully

Is this expected behavior? Where should I start looking if I was interested in adding support for compilation against this alternative C library?

[mfarrugi]

process wrapper error: failed to exec the new process: No such file or directory. is obfuscating the error, would be useful to change

rules_rust/util/process_wrapper/system_posix.cc

Lines 160 to 162 in 09ec54c

	execve(executable.c_str(), argv.data(), envp.data());
	std::cerr << "process wrapper error: failed to exec the new process: "
	<< std::strerror(errno) << ".\n";

to print which executable was missing.

[mfarrugi]

also bazel build -s might be more informative

[dfreese]

No such file or directory is the output of the musl rustc when musl is not installed.

sudo apt-get install musl-dev

takes care of that error, but left me with:

bazel build -c opt @raze__fragile__1_0_0//...
INFO: Analyzed target @raze__fragile__1_0_0//:fragile (1 packages loaded, 41 targets configured).
INFO: Found 1 target...
INFO: From Compiling Rust lib fragile v1.0.0 (5 files):
Error loading shared library libgcc_s.so.1: No such file or directory

[arlyon]

I'm getting the same error, seems that librustc_driver and libstd are both expecting gcc to be available.

edit:

When running ldd on the rustc binary, it complains about an invalid ELF header.

❯ ldd /home/arlyon/.cache/bazel/_bazel_arlyon/6a26623ca76fad8d00fafd66745eebda/external/rust-musl/bin/rustc
/home/arlyon/.cache/bazel/_bazel_arlyon/6a26623ca76fad8d00fafd66745eebda/external/rust-musl/bin/rustc: error while loading shared libraries: /lib64/libc.so: invalid ELF header

On Fedora 33, my libc.so is an ld script that looks in two places for static and dynamic libs, which I assume cannot be read properly for some reason. Changing this to a symlink to a 'correct' library (do not try this at home) and setting LD_LIBRARY_PATH seems to get a step closer.

❯ ldd /home/arlyon/.cache/bazel/_bazel_arlyon/6a26623ca76fad8d00fafd66745eebda/external/rust-musl/bin/rustc
        linux-vdso.so.1 (0x00007ffc90bd6000)
        librustc_driver-e94153381e4dc694.so => /home/arlyon/.cache/bazel/_bazel_arlyon/6a26623ca76fad8d00fafd66745eebda/external/rust-musl/bin/../lib/librustc_driver-e94153381e4dc694.so (0x00007fbb241fd000)
        libstd-dfa62aa5262ff4b4.so => /home/arlyon/.cache/bazel/_bazel_arlyon/6a26623ca76fad8d00fafd66745eebda/external/rust-musl/bin/../lib/libstd-dfa62aa5262ff4b4.so (0x00007fbb23e8c000)
        libc.so => /lib64/libc.so (0x00007fbb23cc1000)
        libgcc_s.so.1 => /lib64/libgcc_s.so.1 (0x00007fbb23ca6000)
        /lib/ld-musl-x86_64.so.1 => /lib64/ld-linux-x86-64.so.2 (0x00007fbb2d0f5000)

We can now at least see the libs getting linked!

❯ export LD_LIBRARY_PATH=/lib64
❯ bazelisk build services/web:rentalbook-web                   
INFO: Analyzed target //services/web:rentalbook-web (0 packages loaded, 0 targets configured).
INFO: Found 1 target...
ERROR: /home/arlyon/.cache/bazel/_bazel_arlyon/6a26623ca76fad8d00fafd66745eebda/external/io_bazel_rules_rust/util/process_wrapper/BUILD:3:10: Linking of rule '@io_bazel_rules_rust//util/process_wrapper:process_wrapper' failed (Exit 1): gcc failed: error executing command /usr/bin/gcc @bazel-out/k8-opt-exec-2B5CBBC6/bin/external/io_bazel_rules_rust/util/process_wrapper/process_wrapper-2.params

Use --sandbox_debug to see verbose messages from the sandbox gcc failed: error executing command /usr/bin/gcc @bazel-out/k8-opt-exec-2B5CBBC6/bin/external/io_bazel_rules_rust/util/process_wrapper/process_wrapper-2.params

Use --sandbox_debug to see verbose messages from the sandbox
/usr/lib/gcc/x86_64-redhat-linux/10/../../../../lib64/crt1.o:function _start: error: undefined reference to '__libc_csu_fini'
/usr/lib/gcc/x86_64-redhat-linux/10/../../../../lib64/crt1.o:function _start: error: undefined reference to '__libc_csu_init'
collect2: error: ld returned 1 exit status
Target //services/web:rentalbook-web failed to build
Use --verbose_failures to see the command lines of failed build steps.
ERROR: /home/arlyon/Programming/rentalbook/services/web/BUILD:10:12 Linking of rule '@io_bazel_rules_rust//util/process_wrapper:process_wrapper' failed (Exit 1): gcc failed: error executing command /usr/bin/gcc @bazel-out/k8-opt-exec-2B5CBBC6/bin/external/io_bazel_rules_rust/util/process_wrapper/process_wrapper-2.params

Use --sandbox_debug to see verbose messages from the sandbox gcc failed: error executing command /usr/bin/gcc @bazel-out/k8-opt-exec-2B5CBBC6/bin/external/io_bazel_rules_rust/util/process_wrapper/process_wrapper-2.params

Use --sandbox_debug to see verbose messages from the sandbox
INFO: Elapsed time: 0.199s, Critical Path: 0.03s
INFO: 2 processes: 2 internal.
FAILED: Build did NOT complete successfully

Seems that __libc_csu_fini and __libc_csu_init are missing because they come from the fallback defined in the script which is a bit of a chicken-egg thing.

[weixiao-huang]

I have the same problem, and I try https://github.com/duarten/rust-bazel-cross. But it seems that it produces a non-static link binary. So is there any method to static link by using musl?

[glindstedt]

I've been experimenting with cross-compiling to musl a bit but I reached a blocker when combining with cargo-raze.

Currently rules_rust doesn't define any constraints for what libc you're using, and so has no way of differentiating between a glibc target and a musl target: https://github.com/bazelbuild/rules_rust/blob/main/rust/platform/triple_mappings.bzl#L183

For a trivial "hello world" you can work around this with patching rules_rust or by defining the toolchain manually like this: https://github.com/duarten/rust-bazel-cross/blob/main/build/toolchains/BUILD.bazel#L42

However, when using dependencies via cargo raze which require libc you get build files like this, and a bunch of errors about a missing libc crate dependency

    deps = [
    ] + selects.with_or({
        # cfg(unix)
        (
            "@rules_rust//rust/platform:x86_64-apple-darwin",
            "@rules_rust//rust/platform:x86_64-unknown-linux-gnu",
            "@rules_rust//rust/platform:aarch64-apple-darwin",
        ): [
            "@raze__libc__0_2_107//:libc",
        ],
        "//conditions:default": [],
    }) + selects.with_or({
        # cfg(windows)
        (
            "@rules_rust//rust/platform:x86_64-pc-windows-msvc",
        ): [
            "@raze__winapi__0_3_9//:winapi",
        ],
        "//conditions:default": [],
    }),

The targets supported by cargo-raze are defined here https://github.com/google/cargo-raze/blob/main/impl/src/util.rs#L29, but since it implicitly depends on the platform existing in rules_rust it would need to be added there first: https://github.com/bazelbuild/rules_rust/blob/main/rust/platform/triple_mappings.bzl#L20

[fionera]

Is there any progress? I tried with the current version of rules_rust and still face the same issues...