Add support for doctests

Currently, nextest doesn't support Rust doctests. This is because doctests are not exposed in stable Rust the way regular test binaries are, and are instead treated as special by cargo test.

One nightly-only fix that might work is to collect doctest executables through -Z unstable-options --persist-doctests. However, this isn't a stable approach so it must be used with care.

Note: You can run cargo test --doc as a separate step from cargo nextest run. This will not incur a performance penalty: cargo nextest run && cargo test --doc will not cause any more builds than a plain cargo test that runs doctests.

So I use the --persist-doctests attribute and can share some of the issues I've faced with it, (hopefully more people using it will lead to stabilisation and improvements). So if we have a doctest with the should_panic attribute cargo test compiles and runs it and checks if the return code is non-zero, and the test passes only if it's non-zero.

However, the doctest file name is made from a normalisation of the path (all path separators replaced with _, a line number and a number to ensure the files are unique. This is because a doctest on the same line in src/foo/bar.rs and src/foo_bar.rs would otherwise generate the same filename. The line numbers are also done after macro expansion so things like include_str can make it harder to match line numbers with attributes.

I made an issue in rust about bundling attribute information into the file name to make it easier for people writing tools using this feature rust-lang/rust#92597 and if you're interested in seeing my code it's in https://github.com/xd009642/tarpaulin in src/cargo.rs, it's reasonably stable the issues mentioned above are the only parts it can fail. I've had this since the nightly flag was created and the naming convention has only changed once so I've only had one completely unexpected issue that affected all users that cause persist-doctests to be added.

Hope this is helpful, nextest looks pretty nifty 🎉

There are also --test-builder/--runtool unstable flags that are used by miri to run doc tests

Can I suggest that until this is added this is called out in the docs in a visible way? I recently was made aware how tricky doctests are to handle, but other users might not be aware. Your crate seems quite nice based on my limited usage, and it would be a shame if they didn't give it a fair chance because they thought it was broken and "skipping tests".

@nu11ptr

@nu11ptr good idea -- I've added a note to the homepage along with expanding the notes on the listing and running pages f4043b1

ci(tests): run tests separately from `nextest`

[Makefile] Run doctests separately using `cargo test`.

refact: update nextest to also run doc tests

Run doc tests in CI (#1253)

Run doc tests again

ci: Run doc tests with cargo test --workspace --doc (#5459)

fix: Fix doctests and start testing them again

fix: Fix doctests and start testing them again (#767)

Rustdoc may gain the unstable flag --extract-doctests which will provide a way to list doctests with their attributes and contents. Is this something that nextest could make use of? rust-lang/rust#134531

Separately, would it be possible for nextest to just run cargo test --doc itself after running other tests? Even without the nice UI it would eliminate the footgun, as well being less to remember/type when running locally.

Of course --extract-doctests or similar would make this unnecessary, but it seems like an easy stopgap until that is usable on stable. (For the record, I would still be quite happy if nextest can run doctests directly only on nightly).

Is this something that nextest could make use of? rust-lang/rust#134531

From a rustdoc perspecive, we defiantly want nextest to be able to use this, and if it can't that's a flaw with rustdoc's thing, and we'd love to hear about this. Ideally, I'd love to see nextest use this (as an experimental feature) before it gets stabilized, to give us some confidence it's broadly useful

Cargo wants to have --doc flags on every build command (cargo check --doc) to build doc tests. I'd recommend playing with --extract-doctests inside of cargo to add support for --doc so it can generate the doctest binaries for you with build caching and correct linking (which was brittle when people tried to do this for READMEs in the past).

This would be really nice to have -- I don't expect to have any time to work on it soon, but some assistance (maybe a prototype within nextest) would be greatly appreciated.

As a hackaround, run doctests with cargo test. For example: runcargo make nextest. I prefer to run it TDD style with bacon.

# Makefile.toml

[tasks.test]
clear = true
workspace = false
dependencies = ["test-tests", "test-doctests"]

[tasks.nextest]
clear = true
workspace = false
dependencies = ["nextest-tests", "test-doctests"]

[tasks.nextest-tests]
workspace = false
command = "cargo"
args = ["nextest", "run", "--workspace", "--hide-progress-bar", "--failure-output", "final"]

[tasks.test-tests]
workspace = false
command = "cargo"
args = ["test", "--all-targets", "--all-features", "--workspace"]

[tasks.test-doctests]
workspace = false
command = "cargo"
args = ["test", "--all-features", "--doc", "--workspace"]

Run doctests as well.

Just a heads up rust-lang/rust#134531 was completed in June and is in unstable so some aspiring soul might want to take on this project and see if that information is enough.

Thanks. Is the idea supposed to be that nextest would have to:

write each doctest out to a file
build up the appropriate rustc invocation (which can be tricky)
compile it (or compile all the doctests together as a single compilation unit)
run each test separately?

That seems to me to be quite far outside nextest's remit as a test runner. I think cargo test's build phase (ie --no-run) should be building these — don't you?

Fix and enable doctests (servo#44851)

build: add justfile + cargo-nextest as the single-source CI gate

test(coverage): add coverage + doctest subcommands, upload to Codecov

Reorganize the top-level project docs, and lint all features in CI (b…

can you guys just change cargo nextest to fork to cargo tests --doc until doctests are supported?

the CLI will work and it can silently change over when a real fix is found.

No, that will not run doctests to nextest's quality standards.

Navigation Menu

Repository navigation

Add support for doctests #16

1 remaining item

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

Add support for doctests #16

Description

Activity

xd009642 commented on Feb 14, 2022

WaffleLapkin commented on Feb 15, 2022

nu11ptr commented on Apr 22, 2022

sunshowers commented on Apr 26, 2022

1 remaining item

tgross35 commented on Dec 28, 2024

tgross35 commented on Dec 28, 2024

aDotInTheVoid commented on Jan 9, 2025

epage commented on Apr 12, 2025

sunshowers commented on Apr 15, 2025

skull-squadron commented on Jun 29, 2025

tv42 commented on Nov 12, 2025

sunshowers commented on Nov 29, 2025

thawkins commented on Oct 26, 2026

sunshowers commented on Oct 26, 2026

Metadata

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

Issue actions