cargo:rustc-link-args not applied to doctest builds

[davepacheco]

I'm not sure if this is a bug or not, but it doesn't seem like desired behavior.

The docs for cargo:rustc-link-arg say:

The rustc-link-arg instruction tells Cargo to pass the -C link-arg=FLAG option to the compiler, but only when building supported targets (benchmarks, binaries, cdylib crates, examples, and tests).

Are doctest binaries intended to be supported or not? They seem to fall under "tests". Empirically, it looks like the requested linker flags are not passed to the compiler when building doctests. The rest of this comment shows why I think this.

---

To show this, I created this repo. This is a library crate that depends on pq-sys (which just happens to be the crate I discovered this with). This test crate has one function that uses something in the native libpq, wrapped by pq-sys. There's a unit test and a doctest that use this function.

My libpq is installed in "/opt/ooce/pgsql-13/lib/amd64", which is not on ld.so's default search path, so I need to build with -R/opt/ooce/pgsql-13/lib/amd64, which means passing -Wl,-R/opt/ooce/pgsql-13/lib/amd64 to gcc. To achieve this, my test crate has a build.rs that uses cargo:rustc-link-arg. To make it easier to show the problem, the build.rs script looks at the environment variable TEST_LIBPQ_DIR. If it's set, it uses cargo:rustc-link-arg to set the corresponding linker flag. If the environment variable is not set, the build.rs does nothing.

I'm using:

$ cargo +nightly version
cargo 1.56.0-nightly (18751dd3f 2021-09-01)

If I just run cargo +nightly test, the unit test fails because ld.so can't find libpq:

$ cargo +nightly test
   Compiling pq-sys v0.4.6
   Compiling link-args-doctest v0.1.0 (/home/dap/link-args-doctest)
warning: not specifying libpq dir
    Finished test [unoptimized + debuginfo] target(s) in 3.19s
     Running unittests (target/debug/deps/link_args_doctest-1d3464329f24b2d7)
ld.so.1: link_args_doctest-1d3464329f24b2d7: fatal: libpq.so.5: open failed: No such file or directory
error: test failed, to rerun pass '--lib'

Caused by:
  process didn't exit successfully: `/home/dap/link-args-doctest/target/debug/deps/link_args_doctest-1d3464329f24b2d7` (signal: 9, SIGKILL: kill)

This means our linker flag was not passed -- that's expected.

Now, if I use RUSTFLAGS and RUSTDOCFLAGS to specify the linker args, everything works:

$ RUSTFLAGS="-Clink-args=-R/opt/ooce/pgsql-13/lib/amd64" RUSTDOCFLAGS="-Clink-args=-R/opt/ooce/pgsql-13/lib/amd64" cargo +nightly test
warning: not specifying libpq dir
    Finished test [unoptimized + debuginfo] target(s) in 0.01s
     Running unittests (target/debug/deps/link_args_doctest-1d3464329f24b2d7)

running 1 test
test test_foo ... ok

test result: ok. 1 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out; finished in 0.00s

   Doc-tests link-args-doctest

running 1 test
test src/lib.rs - foo (line 1) ... ok

test result: ok. 1 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out; finished in 0.33s

This is also expected -- we forced rustc and rustdoc to pass gcc the right linker arg so that it would find the library.

What's surprising is that if I set my environment variable so that build.rs uses cargo:rustc-link-args, then the unit test works, but the doctest doesn't:

$ TEST_LIBPQ_DIR=/opt/ooce/pgsql-13/lib/amd64 cargo +nightly test
   Compiling pq-sys v0.4.6
   Compiling link-args-doctest v0.1.0 (/home/dap/link-args-doctest)
warning: using libpq dir from environment: /opt/ooce/pgsql-13/lib/amd64
    Finished test [unoptimized + debuginfo] target(s) in 2.02s
     Running unittests (target/debug/deps/link_args_doctest-1d3464329f24b2d7)

running 1 test
test test_foo ... ok

test result: ok. 1 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out; finished in 0.00s

   Doc-tests link-args-doctest

running 1 test
test src/lib.rs - foo (line 1) ... FAILED

failures:

---- src/lib.rs - foo (line 1) stdout ----
Test executable failed (terminated by signal).

stderr:
ld.so.1: rust_out: fatal: libpq.so.5: open failed: No such file or directory



failures:
    src/lib.rs - foo (line 1)

test result: FAILED. 0 passed; 1 failed; 0 ignored; 0 measured; 0 filtered out; finished in 0.47s

error: test failed, to rerun pass '--doc'

Since the unit test passed, we know the linker arg was passed correctly for that. We can also see that the test binary has the right directory on the RUNPATH:

$ readelf -d /home/dap/link-args-doctest/target/debug/deps/link_args_doctest-1d3464329f24b2d7 | grep RUNPATH
 0x000000000000001d (RUNPATH)            Library runpath: [/usr/gcc/10/lib/amd64:/opt/ooce/pgsql-13/lib/amd64]

But the doctest build doesn't.

To be really sure, here's the very verbose output:

$ TEST_LIBPQ_DIR=/opt/ooce/pgsql-13/lib/amd64 cargo +nightly test -vv
   Compiling pq-sys v0.4.6
   Compiling link-args-doctest v0.1.0 (/home/dap/link-args-doctest)
     Running `CARGO=/home/dap/.rustup/toolchains/nightly-x86_64-unknown-illumos/bin/cargo CARGO_CRATE_NAME=build_script_build CARGO_MANIFEST_DIR=/home/dap/.cargo/registry/src/github.1485827954.workers.dev-1ecc6299db9ec823/pq-sys-0.4.6 CARGO_PKG_AUTHORS='Sean Griffin <sean@seantheprogrammer.com>' CARGO_PKG_DESCRIPTION='Auto-generated rust bindings for libpq' CARGO_PKG_HOMEPAGE='' CARGO_PKG_LICENSE='MIT OR Apache-2.0' CARGO_PKG_LICENSE_FILE='' CARGO_PKG_NAME=pq-sys CARGO_PKG_REPOSITORY='https://github.com/sgrif/pq-sys' CARGO_PKG_VERSION=0.4.6 CARGO_PKG_VERSION_MAJOR=0 CARGO_PKG_VERSION_MINOR=4 CARGO_PKG_VERSION_PATCH=6 CARGO_PKG_VERSION_PRE='' LD_LIBRARY_PATH='/home/dap/link-args-doctest/target/debug/deps:/home/dap/.rustup/toolchains/nightly-x86_64-unknown-illumos/lib:/home/dap/.rustup/toolchains/nightly-x86_64-unknown-illumos/lib' rustc --crate-name build_script_build /home/dap/.cargo/registry/src/github.1485827954.workers.dev-1ecc6299db9ec823/pq-sys-0.4.6/build.rs --error-format=json --json=diagnostic-rendered-ansi --crate-type bin --emit=dep-info,link -C embed-bitcode=no -C debuginfo=2 -C metadata=066bbe521cca295f -C extra-filename=-066bbe521cca295f --out-dir /home/dap/link-args-doctest/target/debug/build/pq-sys-066bbe521cca295f -L dependency=/home/dap/link-args-doctest/target/debug/deps --cap-lints warn`
     Running `CARGO=/home/dap/.rustup/toolchains/nightly-x86_64-unknown-illumos/bin/cargo CARGO_CRATE_NAME=build_script_build CARGO_MANIFEST_DIR=/home/dap/link-args-doctest CARGO_PKG_AUTHORS='' CARGO_PKG_DESCRIPTION='' CARGO_PKG_HOMEPAGE='' CARGO_PKG_LICENSE='' CARGO_PKG_LICENSE_FILE='' CARGO_PKG_NAME=link-args-doctest CARGO_PKG_REPOSITORY='' CARGO_PKG_VERSION=0.1.0 CARGO_PKG_VERSION_MAJOR=0 CARGO_PKG_VERSION_MINOR=1 CARGO_PKG_VERSION_PATCH=0 CARGO_PKG_VERSION_PRE='' CARGO_PRIMARY_PACKAGE=1 LD_LIBRARY_PATH='/home/dap/link-args-doctest/target/debug/deps:/home/dap/.rustup/toolchains/nightly-x86_64-unknown-illumos/lib:/home/dap/.rustup/toolchains/nightly-x86_64-unknown-illumos/lib' rustc --crate-name build_script_build --edition=2018 build.rs --error-format=json --json=diagnostic-rendered-ansi --crate-type bin --emit=dep-info,link -C embed-bitcode=no -C debuginfo=2 -C metadata=ae900bc3a3724b07 -C extra-filename=-ae900bc3a3724b07 --out-dir /home/dap/link-args-doctest/target/debug/build/link-args-doctest-ae900bc3a3724b07 -C incremental=/home/dap/link-args-doctest/target/debug/incremental -L dependency=/home/dap/link-args-doctest/target/debug/deps`
     Running `/home/dap/link-args-doctest/target/debug/build/pq-sys-066bbe521cca295f/build-script-build`
[pq-sys 0.4.6] cargo:rerun-if-env-changed=PQ_LIB_DIR
[pq-sys 0.4.6] cargo:rerun-if-env-changed=PQ_LIB_STATIC
[pq-sys 0.4.6] cargo:rerun-if-env-changed=TARGET
[pq-sys 0.4.6] cargo:rerun-if-env-changed=PG_CONFIG_X86_64_UNKNOWN_ILLUMOS
[pq-sys 0.4.6] cargo:rustc-link-search=native=/opt/ooce/pgsql-13/lib/amd64
[pq-sys 0.4.6] cargo:rerun-if-env-changed=PQ_LIB_STATIC_X86_64_UNKNOWN_ILLUMOS
[pq-sys 0.4.6] cargo:rustc-link-lib=pq
     Running `/home/dap/link-args-doctest/target/debug/build/link-args-doctest-ae900bc3a3724b07/build-script-build`
     Running `CARGO=/home/dap/.rustup/toolchains/nightly-x86_64-unknown-illumos/bin/cargo CARGO_CRATE_NAME=pq_sys CARGO_MANIFEST_DIR=/home/dap/.cargo/registry/src/github.1485827954.workers.dev-1ecc6299db9ec823/pq-sys-0.4.6 CARGO_PKG_AUTHORS='Sean Griffin <sean@seantheprogrammer.com>' CARGO_PKG_DESCRIPTION='Auto-generated rust bindings for libpq' CARGO_PKG_HOMEPAGE='' CARGO_PKG_LICENSE='MIT OR Apache-2.0' CARGO_PKG_LICENSE_FILE='' CARGO_PKG_NAME=pq-sys CARGO_PKG_REPOSITORY='https://github.com/sgrif/pq-sys' CARGO_PKG_VERSION=0.4.6 CARGO_PKG_VERSION_MAJOR=0 CARGO_PKG_VERSION_MINOR=4 CARGO_PKG_VERSION_PATCH=6 CARGO_PKG_VERSION_PRE='' LD_LIBRARY_PATH='/home/dap/link-args-doctest/target/debug/deps:/home/dap/.rustup/toolchains/nightly-x86_64-unknown-illumos/lib:/home/dap/.rustup/toolchains/nightly-x86_64-unknown-illumos/lib' OUT_DIR=/home/dap/link-args-doctest/target/debug/build/pq-sys-3988470aaaa1871a/out rustc --crate-name pq_sys /home/dap/.cargo/registry/src/github.1485827954.workers.dev-1ecc6299db9ec823/pq-sys-0.4.6/src/lib.rs --error-format=json --json=diagnostic-rendered-ansi,artifacts --crate-type lib --emit=dep-info,metadata,link -C embed-bitcode=no -C debuginfo=2 -C metadata=92e5a3b5ec0e85a5 -C extra-filename=-92e5a3b5ec0e85a5 --out-dir /home/dap/link-args-doctest/target/debug/deps -L dependency=/home/dap/link-args-doctest/target/debug/deps --cap-lints warn -L native=/opt/ooce/pgsql-13/lib/amd64 -l pq`
[link-args-doctest 0.1.0] cargo:rerun-if-env-changed=TEST_LIBPQ_DIR
[link-args-doctest 0.1.0] cargo:warning=using libpq dir from environment: /opt/ooce/pgsql-13/lib/amd64
[link-args-doctest 0.1.0] cargo:rustc-link-arg=-Wl,-R/opt/ooce/pgsql-13/lib/amd64
warning: using libpq dir from environment: /opt/ooce/pgsql-13/lib/amd64
     Running `CARGO=/home/dap/.rustup/toolchains/nightly-x86_64-unknown-illumos/bin/cargo CARGO_CRATE_NAME=link_args_doctest CARGO_MANIFEST_DIR=/home/dap/link-args-doctest CARGO_PKG_AUTHORS='' CARGO_PKG_DESCRIPTION='' CARGO_PKG_HOMEPAGE='' CARGO_PKG_LICENSE='' CARGO_PKG_LICENSE_FILE='' CARGO_PKG_NAME=link-args-doctest CARGO_PKG_REPOSITORY='' CARGO_PKG_VERSION=0.1.0 CARGO_PKG_VERSION_MAJOR=0 CARGO_PKG_VERSION_MINOR=1 CARGO_PKG_VERSION_PATCH=0 CARGO_PKG_VERSION_PRE='' CARGO_PRIMARY_PACKAGE=1 LD_LIBRARY_PATH='/home/dap/link-args-doctest/target/debug/deps:/home/dap/.rustup/toolchains/nightly-x86_64-unknown-illumos/lib:/home/dap/.rustup/toolchains/nightly-x86_64-unknown-illumos/lib' OUT_DIR=/home/dap/link-args-doctest/target/debug/build/link-args-doctest-d6beefd3d9f74ce9/out rustc --crate-name link_args_doctest --edition=2018 src/lib.rs --error-format=json --json=diagnostic-rendered-ansi --crate-type lib --emit=dep-info,metadata,link -C embed-bitcode=no -C debuginfo=2 -C metadata=5f3b72e47d45b795 -C extra-filename=-5f3b72e47d45b795 --out-dir /home/dap/link-args-doctest/target/debug/deps -C incremental=/home/dap/link-args-doctest/target/debug/incremental -L dependency=/home/dap/link-args-doctest/target/debug/deps --extern pq_sys=/home/dap/link-args-doctest/target/debug/deps/libpq_sys-92e5a3b5ec0e85a5.rmeta -C link-arg=-Wl,-R/opt/ooce/pgsql-13/lib/amd64 -L native=/opt/ooce/pgsql-13/lib/amd64`
     Running `CARGO=/home/dap/.rustup/toolchains/nightly-x86_64-unknown-illumos/bin/cargo CARGO_CRATE_NAME=link_args_doctest CARGO_MANIFEST_DIR=/home/dap/link-args-doctest CARGO_PKG_AUTHORS='' CARGO_PKG_DESCRIPTION='' CARGO_PKG_HOMEPAGE='' CARGO_PKG_LICENSE='' CARGO_PKG_LICENSE_FILE='' CARGO_PKG_NAME=link-args-doctest CARGO_PKG_REPOSITORY='' CARGO_PKG_VERSION=0.1.0 CARGO_PKG_VERSION_MAJOR=0 CARGO_PKG_VERSION_MINOR=1 CARGO_PKG_VERSION_PATCH=0 CARGO_PKG_VERSION_PRE='' CARGO_PRIMARY_PACKAGE=1 LD_LIBRARY_PATH='/home/dap/link-args-doctest/target/debug/deps:/home/dap/.rustup/toolchains/nightly-x86_64-unknown-illumos/lib:/home/dap/.rustup/toolchains/nightly-x86_64-unknown-illumos/lib' OUT_DIR=/home/dap/link-args-doctest/target/debug/build/link-args-doctest-d6beefd3d9f74ce9/out rustc --crate-name link_args_doctest --edition=2018 src/lib.rs --error-format=json --json=diagnostic-rendered-ansi --emit=dep-info,link -C embed-bitcode=no -C debuginfo=2 --test -C metadata=1d3464329f24b2d7 -C extra-filename=-1d3464329f24b2d7 --out-dir /home/dap/link-args-doctest/target/debug/deps -C incremental=/home/dap/link-args-doctest/target/debug/incremental -L dependency=/home/dap/link-args-doctest/target/debug/deps --extern pq_sys=/home/dap/link-args-doctest/target/debug/deps/libpq_sys-92e5a3b5ec0e85a5.rlib -C link-arg=-Wl,-R/opt/ooce/pgsql-13/lib/amd64 -L native=/opt/ooce/pgsql-13/lib/amd64`
    Finished test [unoptimized + debuginfo] target(s) in 3.17s
     Running `/home/dap/link-args-doctest/target/debug/deps/link_args_doctest-1d3464329f24b2d7`

running 1 test
test test_foo ... ok

test result: ok. 1 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out; finished in 0.00s

   Doc-tests link-args-doctest
     Running `rustdoc --edition=2018 --crate-type lib --crate-name link_args_doctest --test /home/dap/link-args-doctest/src/lib.rs -L dependency=/home/dap/link-args-doctest/target/debug/deps -L dependency=/home/dap/link-args-doctest/target/debug/deps -L native=/opt/ooce/pgsql-13/lib/amd64 --extern link_args_doctest=/home/dap/link-args-doctest/target/debug/deps/liblink_args_doctest-5f3b72e47d45b795.rlib --extern pq_sys=/home/dap/link-args-doctest/target/debug/deps/libpq_sys-92e5a3b5ec0e85a5.rlib -C embed-bitcode=no --error-format human`

running 1 test
test src/lib.rs - foo (line 1) ... FAILED

failures:

---- src/lib.rs - foo (line 1) stdout ----
Test executable failed (terminated by signal).

stderr:
ld.so.1: rust_out: fatal: libpq.so.5: open failed: No such file or directory



failures:
    src/lib.rs - foo (line 1)

test result: FAILED. 0 passed; 1 failed; 0 ignored; 0 measured; 0 filtered out; finished in 0.48s

error: test failed, to rerun pass '--doc'

The rustc invocation for building the unit test includes -C link-arg=-Wl,-R/opt/ooce/pgsql-13/lib/amd64 but the one for rustdoc does not. If I take the rustdoc invocation, add "+nightly" and -C link-arg=-Wl,-R/opt/ooce/pgsql-13/lib/amd64 by hand, then the rustdoc test works:

$ rustdoc +nightly --edition=2018 --crate-type lib --crate-name link_args_doctest --test /home/dap/link-args-doctest/src/lib.rs -L dependency=/home/dap/link-args-doctest/target/debug/deps -L dependency=/home/dap/link-args-doctest/target/debug/deps -L native=/opt/ooce/pgsql-13/lib/amd64 --extern link_args_doctest=/home/dap/link-args-doctest/target/debug/deps/liblink_args_doctest-5f3b72e47d45b795.rlib --extern pq_sys=/home/dap/link-args-doctest/target/debug/deps/libpq_sys-92e5a3b5ec0e85a5.rlib -C embed-bitcode=no --error-format human -C link-arg=-Wl,-R/opt/ooce/pgsql-13/lib/amd64

running 1 test
test src/lib.rs - foo (line 1) ... ok

test result: ok. 1 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out; finished in 0.33s

Anyway, I expected that the link args from "cargo:rustc-link-args" would apply to rustdoc tests, but they don't seem to be. Is this a bug?

[ehuss]

Oh my, doc tests do tend to get left in the cold when new flags are added. I'm not sure if that can be solved generally, but to solve this problem specifically, I think roughly the following changes would be needed:

Add the appropriate -C link-arg arguments to the Doctest.args around here. I believe output.linker_args contains the args, and it needs to check if they "apply" (see here).

[chansuke]

@rustbot claim

[chansuke]

@ehuss Thank you for your comment. In order to reproduce this case in tests, I think /opt/ooce/pgsql-13/lib/amd64 needs to be created at cargo test but I don't know how to achieve this...

[ehuss]

There are several tests in build_script_extra_link_arg.rs where it just passes a bogus argument, and then checks that it is passed with the -v flag. So roughly:

• Have a lib.rs with a doc test.
• Have a build.rs which passes the bogus flag.
• Run cargo test --doc -v and check that it passes the flag.