Handling of rustc-sysroot .so files differs when sysroots do not come with their own cargo

[RalfJung]

When I install miri (cargo +nightly install --git https://github.com/solson/miri/ miri) with the nightly toolchain, I can later execute cargo +nightly miri --help just fine. However, when I do the same thing with a toolchain obtained via rustup toolchain link (a locally-compiled stage 2 sysroot), that does not work: After replacing +nightly by +stage2, all I get is

/home/r/.cargo/bin/cargo-miri: error while loading shared libraries: librustc_mir-2d8f89b6ce1d8c91.so: cannot open shared object file: No such file or directory

Interestingly, toolchains installed via https://github.com/kennytm/rustup-toolchain-install-master behave like linked ones: They also result in the above error.

What both of these toolchains (linked ones and rustup-toolchain-install-master) have in common is that they don't come with their own cargo.

I don't know how the responsibility of setting LD_LIBRARY_PATH (assuming that is what is missing here) is split between rustup and cargo, and I have no idea which cargo rustup is even using when the toolchain does not have its own. But somewhere, something doesn't quite fit together.

(Calling the cargo-miri binary directly shows such an error in both cases, but when I am using nightly, it seems that LD_LIBRARY_PATH is set properly when using cargo miri.)

[RalfJung]

I can confirm now that it is about cargo; when I install the "cargo" component with rustup-toolchain-install-master, the created binaries work fine in rustup.

[RalfJung]

This issue affects not only Linux but also Windows: installing a toolchain without cargo, and then with that toolchain building and running a program that uses rustc_private crates, does not work.

[kinnison]

I am unsure of how we ought to try and deal with this? What are you hoping we could do in rustup to resolve your problem?

[RalfJung]

It could set up whatever are the right environment variables to make the program it runs actually find shared libraries of the current toolchain.

When I wrote this issue I was not aware that this setup is typically done by cargo, not rustup. But given the problems it causes to not do this (and given how hard they are to debug!), maybe that would be worth doing in rustup as well?

An alternative might be to somehow warn that cargo-less toolchains are generate, and point to docs explaining more.

[kinnison]

Would setting up those variables cause cargo any issues? If not then I agree that'd be a good fix. if it might upset cargo then perhaps warn if any cargo-* proxy is used when the cargo component is not installed?

[RalfJung]

Would setting up those variables cause cargo any issues?

I do not know. Maybe @ehuss can help?

[ehuss]

So just to summarize:

When using a toolchain that is missing cargo, rustup has a fallback mechanism to use other toolchains. When this happens, the LD_LIBRARY_PATH points to the fallback toolchain, which is probably not what you want.

Can you try changing this line to primary_toolchain.set_env(&mut cmd);? That will set the LD_LIBRARY_PATH to point to the cargo-less toolchain instead of the fallback one. I don't think that will cause any problems with Cargo, since it doesn't dynamically link anything itself. And since it is running rustc from the "primary" toolchain, you probably want the LD_LIBRARY_PATH to match the primary one.

[kinnison]

Thanks for that @ehuss - @RalfJung are you in a position to try a rustup build with the patch suggested or do you need me to prepare one for you to try?

[RalfJung]

Yeah I can try that, though probably not before the weekend.

[RalfJung]

Yes, that seems to work exactly as I hoped. :) Thanks a lot! I will prepare a PR.