Libraries not provided as arguments to the linker

[elinorbgr]

I didn't manage to narrow it down to a simple example, because I couldn't reproduce it out of my project. So the beast is here: https://github.com/metallirc/metallircd/ on the branch vberger.

When I run cargo build it fails at the linking phase, not able to find the symbols coming from the toml crate.

The problem is that, if we compare the arguments provided to rustc by cargo:

rustc src/main.rs --crate-name metallircd --crate-type bin
-g --out-dir /home/levans/dev/metallirc/metallircd/target/debug
--emit=dep-info,link -L dependency=/home/levans/dev/metallirc/metallircd/target/debug
-L dependency=/home/levans/dev/metallirc/metallircd/target/debug/deps
--extern uuid=/home/levans/dev/metallirc/metallircd/target/debug/deps/libuuid-cf857f0edb9103fd.rlib 
--extern mio=/home/levans/dev/metallirc/metallircd/target/debug/deps/libmio-a1703e31ab8c0ae9.rlib 
--extern mod_motd=/home/levans/dev/metallirc/metallircd/target/debug/deps/libmod_motd-13d38478119c3cc9.so
--extern toml=/home/levans/dev/metallirc/metallircd/target/debug/deps/libtoml-e4d393d734aaa578.rlib 
--extern time=/home/levans/dev/metallirc/metallircd/target/debug/deps/libtime-e758cbe877e9589d.rlib 
--extern log=/home/levans/dev/metallirc/metallircd/target/debug/deps/liblog-54cf393d3c69686f.rlib 
--extern mod_away=/home/levans/dev/metallirc/metallircd/target/debug/deps/libmod_away-75894f3762435d3c.so 
--extern metallirc=/home/levans/dev/metallirc/metallircd/target/debug/deps/libmetallirc-330c17c546085f24.so 
--extern getopts=/home/levans/dev/metallirc/metallircd/target/debug/deps/libgetopts-d6ecf60df5ea72e6.rlib 
-L native=/home/levans/dev/metallirc/metallircd/target/debug/build/time-e758cbe877e9589d/out

and the arguments provided by rustc to the linker:

"cc" "-Wl,--as-needed" "-m64" "-L" "/usr/lib/rust-9999/rustlib/x86_64-unknown-linux-gnu/lib" 
"-o" "/home/levans/dev/metallirc/metallircd/target/debug/metallircd"
"/home/levans/dev/metallirc/metallircd/target/debug/metallircd.o" 
"-Wl,--whole-archive" "-lmorestack" "-Wl,--no-whole-archive" "-Wl,--gc-sections" "-pie" "-nodefaultlibs" 
"/home/levans/dev/metallirc/metallircd/target/debug/deps/libgetopts-d6ecf60df5ea72e6.rlib" 
"/home/levans/dev/metallirc/metallircd/target/debug/deps/libmio-a1703e31ab8c0ae9.rlib" 
"/home/levans/dev/metallirc/metallircd/target/debug/deps/libbytes-2a073ff88fbb29db.rlib" 
"/home/levans/dev/metallirc/metallircd/target/debug/deps/libnix-a0d91ce5788b8fea.rlib" 
"/home/levans/dev/metallirc/metallircd/target/debug/deps/libbitflags-518ea12e21428edd.rlib" 
"-L" "/home/levans/dev/metallirc/metallircd/target/debug/deps" 
"-lmetallirc-330c17c546085f24" 
"-L" "/usr/lib64/rust-9999/rustlib/x86_64-unknown-linux-gnu/lib" "-lstd-gentoo-git" 
"-L" "/home/levans/dev/metallirc/metallircd/target/debug" 
"-L" "/home/levans/dev/metallirc/metallircd/target/debug/deps" 
"-L" "/home/levans/dev/metallirc/metallircd/target/debug/build/time-e758cbe877e9589d/out" 
"-L" "/usr/lib/rust-9999/rustlib/x86_64-unknown-linux-gnu/lib" 
"-L" "/home/levans/dev/metallirc/metallircd/.rust/lib/rust-9999/x86_64-unknown-linux-gnu" 
"-L" "/home/levans/dev/metallirc/metallircd/lib/rust-9999/x86_64-unknown-linux-gnu" 
"-Wl,--whole-archive" "-Wl,-Bstatic" "-Wl,--no-whole-archive" "-Wl,-Bdynamic" 
"-lrt" "-lc" "-lm" "-ldl" "-lpthread" "-lrt" "-lgcc_s" "-lpthread" "-lc" "-lm" "-lcompiler-rt"

It appears that, for some reason, libtoml-e4d393d734aaa578.rlib is not present in the second list of arguments, even if I have a extern crate toml; in my main.rs.

It could be reproduced with these versions of rustc:

rustc 1.1.0-dev (af41097b4 2015-05-13) (built 2015-05-13)
rustc 1.2.0-nightly (0cc99f9cc 2015-05-17) (built 2015-05-18)

and also on travis ( https://travis-ci.org/metallirc/metallircd/builds/63566435 )

[elinorbgr]

Also, manually running the cc command and adding the argument

"/home/levans/dev/metallirc/metallircd/target/debug/deps/libtoml-e4d393d734aaa578.rlib"

links the final executable successfully.

[Aatch]

Hmm, it's likely due to the fact that toml is actually only a dependency of libmetallircd. You shouldn't need to link in libtoml to the binary. It's possible that libmetallircd is assuming that the static dependency will be linked in by downstream users and that the binary is assuming that, because libmetallircd is dynamic, it already has the dependencies sorted.

A simple fix in the meantime is to either drop the dylib from libmetallircd, or specify that you want to dynamically link to libtoml in libmetallircd.

[Aatch]

The real question is whether this is a rustc bug or a cargo one, I'm leaning towards rustc.

[elinorbgr]

@Aatch I'm not sure I fully understand what you wrote (I have a libmetallirc and a metallircd but no libmetallircd).

However, both the library (libmetallirc) and the binary (metallircd) need to be linked to the toml crate, as they both make use of types and methods defined in it.

What currently happens is that the dynamic library is fully compiled without any problem (and ldd confirms me that all missing symbols are only related to system libraries). It's the binary which fails at compilation with the missing toml symbols.

I tried to reproduce with a toy project: a dylib depending on toml and an executable depending on toml and this dylib, but it fully compiled successfully.

However, changing libmetallirc to be a classic rlib indeed fixes the compilation.

[elinorbgr]

However, I agree It's most likely a rustc bug, as toml is in the arguments provided to rustc by cargo, but not in the arguments provided by rustc to the linker.

[elinorbgr]

Actually, if I understand correctly

rust/src/librustc_trans/back/link.rs

Lines 1100 to 1102 in 43cf733

	// We may not pass all crates through to the linker. Some crates may
	// appear statically in an existing dylib, meaning we'll pick up all the
	// symbols from the dylib.

it means that rustc does not link the executable to toml because it expects the symbols of toml to be in the dylib, but they actually are not there.

To test this, I added a pub use toml as toml_reexport; in the dylib, and changed the executable to only use this reexport (and thus removed the extern crate toml; from it), and I get the exact same error.

[elinorbgr]

Actually, it's certainly linked to #14344 (possibly a dupe?).

[alexcrichton]

@vberger hm the usage of toml in libmetalirc makes me hesitant to close this in favor of #14344, but I agree that it seems quite suspicious. Do you have a branch I can check out and poke around at?

[elinorbgr]

@alexcrichton I've let everything on the vberger branch in the state of the linking error, feel free to use it as an experiment material.

[alexcrichton]

Thanks! After some more investigation I have confirmed that this is a dupe of #14344.