`rust_doc_test` Failed to link crates when proc-macro is used in the dependencies

[vbkaisetsu]

To describe the situation, I created the following small repository:
https://github.com/vbkaisetsu/rules-rust-link-fails-repro

Steps to reproduce

1. Clone the repository.

   git clone https://github.com/vbkaisetsu/rules-rust-link-fails-repro.git
   

2. Run the test.

   cd rules-rust-link-fails-repro
   bazel run //repro:my_lib_doc_test
   

3. The following error occurs:

   exec ${PAGER:-/usr/bin/less} "$0" || exit 1
   Executing tests from //repro:my_lib_doc_test
   -----------------------------------------------------------------------------
   error[E0460]: found possibly newer version of crate `fnv` which `tonic` depends on
    --> repro/my_lib.rs:3:5
     |
   3 | use tonic::Status;
     |     ^^^^^
     |
     = note: perhaps that crate needs to be recompiled?
     = note: the following crate versions were found:
             crate `fnv`: /home/koichi.akabe/.cache/bazel/_bazel_koichi.akabe/65e8d7f25191e794777d18f078967787/execroot/link_fail_repro/bazel-out/k8-opt-exec-2B5CBBC6/bin/external/raze__fnv__1_0_7/libfnv--1978688209.rlib
             crate `tonic`: /home/koichi.akabe/.cache/bazel/_bazel_koichi.akabe/65e8d7f25191e794777d18f078967787/execroot/link_fail_repro/bazel-out/k8-fastbuild/bin/external/raze__tonic__0_5_2/libtonic--506703014.rlib
   
   error: aborting due to previous error
   

Workarounds

• Run the test with optimization.

  bazel run -c opt //repro:my_lib_doc_test
  

• Change cfg of proc_macro_deps from exec to target.
  

  rules_rust/rust/private/rust.bzl

  Line 575 in 82b650d

  cfg = "exec",

[illicitonion]

Thanks for the repro, it's really useful for debugging!

My guess having only briefly delved in is that the root of the problem is:

1. A dependency needs to be reachable both in the target config and via a proc macro
2. We currently add both of these to the library search path, in an unspecified order
3. When the order happens to be "via proc macro" before "in target config", we find the wrong one at runtime and error

I suspect the fix for this will be to stop expanding transitive deps via proc macro deps when doing runtime execution of binaries. We should probably only be adding -L directories for runtime deps to things in the target config (but letting the rustc invocations for proc macro using compiles still see both versions)...

@hlopko does that sound about right to you?

[illicitonion]

I did a little more debugging, it looks like though we are adding multiple deps on fnv it's always for the exec version.

This comment:

rules_rust/rust/private/rustdoc_test.bzl

Line 50 in 25f396f

# The test script compiles the crate and runs it, so it needs both compile and runtime inputs.

Suggests that maybe this will be a hard problem to fix...

[jyn514]

I don't know whether this is related at all, but I remember seeing rust-lang/rust#13983 a while ago.

[UebelAndre]

It seems #1206 may have addressed this. I'll try to find some time to write a regression test for this to better understand what happened.