Build failure on x86_64, rust 1.25.0

[therealbstern]

~/html5ever-html5ever-v0.22.2$ cargo build -vv
   Compiling html5ever v0.22.2 (file://[...]/html5ever-html5ever-v0.22.2/html5ever)
     Running `rustc --crate-name build_script_build html5ever/build.rs --crate-type bin --emit=dep-info,link -C metadata=8e14a6802a9b2fd9 -C extra-filename=-8e14a6802a9b2fd9 --out-dir [...]/html5ever-html5ever-v0.22.2/target/debug/build/html5ever-8e14a6802a9b2fd9 -C incremental=[...]/html5ever-html5ever-v0.22.2/target/debug/incremental -L dependency=[...]/html5ever-html5ever-v0.22.2/target/debug/deps --extern quote=[...]/html5ever-html5ever-v0.22.2/target/debug/deps/libquote-e1fc9be4bd78c8fa.rlib --extern proc_macro2=[...]/html5ever-html5ever-v0.22.2/target/debug/deps/libproc_macro2-a655b102e45a5668.rlib --extern syn=[...]/html5ever-html5ever-v0.22.2/target/debug/deps/libsyn-99c1aa14b66cb52e.rlib`
       Fresh serde_derive_internals v0.23.1
       Fresh tendril v0.4.0
   Compiling string_cache v0.7.2
     Running `rustc --crate-name build_script_build [...]/.cargo/registry/src/github.1485827954.workers.dev-1ecc6299db9ec823/string_cache-0.7.2/build.rs --crate-type bin --emit=dep-info,link -C metadata=9290c0ac062e9707 -C extra-filename=-9290c0ac062e9707 --out-dir [...]/html5ever-html5ever-v0.22.2/target/debug/build/string_cache-9290c0ac062e9707 -L dependency=[...]/html5ever-html5ever-v0.22.2/target/debug/deps --extern string_cache_codegen=[...]/html5ever-html5ever-v0.22.2/target/debug/deps/libstring_cache_codegen-463d5e2eed3883c8.rlib --cap-lints warn`
     Running `[...]/html5ever-html5ever-v0.22.2/target/debug/build/html5ever-8e14a6802a9b2fd9/build-script-build`
[...]/html5ever-html5ever-v0.22.2/target/debug/build/html5ever-8e14a6802a9b2fd9/build-script-build: error while loading shared libraries: libproc_macro-7d531857c463fdf1.so: cannot open shared object file: No such file or directory
error: failed to run custom build command for `html5ever v0.22.2 (file://[...]/html5ever-html5ever-v0.22.2/html5ever)`
process didn't exit successfully: `[...]/html5ever-html5ever-v0.22.2/target/debug/build/html5ever-8e14a6802a9b2fd9/build-script-build` (exit code: 127)
--- stderr
[...]/html5ever-html5ever-v0.22.2/target/debug/build/html5ever-8e14a6802a9b2fd9/build-script-build: error while loading shared libraries: libproc_macro-7d531857c463fdf1.so: cannot open shared object file: No such file or directory

Note that the following build command works:

LD_LIBRARY_PATH=/usr/local/lib/rustlib/x86_64-unknown-linux-gnu/lib cargo build -vv

$ rustc --version -v 
rustc 1.25.0 (84203cac6 2018-03-25)
binary: rustc
commit-hash: 84203cac67e65ca8640b8392348411098c856985
commit-date: 2018-03-25
host: x86_64-unknown-linux-gnu
release: 1.25.0
LLVM version: 6.0
$ uname -a
Linux darkstar 4.9.50 #1 SMP Wed Sep 20 02:39:00 -00 2017 x86_64 Intel(R) Xeon(R) CPU E5-2676 v3 @ 2.40GHz GenuineIntel GNU/Linux

[SimonSapin]

This looks like not a bug in html5ever’s source code but rather in your Rust/Cargo install. How did you install them? Does the same happen if you install through https://rustup.rs/ ?

[therealbstern]

I used the manual install, because I wanted to check the PGP signature before deploying. I'll follow up with more results from a throwaway VM.

[SimonSapin]

Ok. It look like rustup does set LD_LIBRARY_PATH: https://github.com/rust-lang-nursery/rustup.rs/blob/1.11.0/src/rustup/toolchain.rs#L404-L431. I’m fairly sure this is a compiler install issue and not at all an html5ever bug.

[therealbstern]

Rustup may set it for its own use, but I haven't needed it set for any other crate's build. (It looks to me like this was added to rustup in response to rust-lang/rustup#932). In particular, rust-lang/rustup#932 (comment) has some discussion of this.

Another path that works is adding -C rpath to the rustc invocation for the build script, but other than adding it to RUSTFLAGS or ~/.cargo/config, I can't figure out how to make this work for generating the build script. Ideally, rpath wouldn't be needed for the final binary, since only the build script needs it (for some weird reason).

[therealbstern]

This is a rust problem, commenting over there.

[SimonSapin]

This build script (indirectly) uses the proc_macro crate, which apparently is only distributed as a .so file, no .rlib (unlike std for example which has both). I think this is related to the fact that procedural macros are usually loaded by the compiler through dynamic linking, but maybe the .rlib file should be distributed anyway.

[therealbstern]

I concur with your analysis, including that it's not a Servo problem, even though only Servo has triggered it in my experience. Clearly, other packages that do the same thing with proc_macro could (and probably do) have this problem.

I referenced this issue as noted, since the Cargo people need to be aware. Please note that this problem is triggered in the following Servo crates: html5ever, markup5ever, and string_cache.