Skip to content

Commit a66d733

Browse files
ojedashuahkh
authored andcommitted
rust: support running Rust documentation tests as KUnit ones
Rust has documentation tests: these are typically examples of usage of any item (e.g. function, struct, module...). They are very convenient because they are just written alongside the documentation. For instance: /// Sums two numbers. /// /// ``` /// assert_eq!(mymod::f(10, 20), 30); /// ``` pub fn f(a: i32, b: i32) -> i32 { a + b } In userspace, the tests are collected and run via `rustdoc`. Using the tool as-is would be useful already, since it allows to compile-test most tests (thus enforcing they are kept in sync with the code they document) and run those that do not depend on in-kernel APIs. However, by transforming the tests into a KUnit test suite, they can also be run inside the kernel. Moreover, the tests get to be compiled as other Rust kernel objects instead of targeting userspace. On top of that, the integration with KUnit means the Rust support gets to reuse the existing testing facilities. For instance, the kernel log would look like: KTAP version 1 1..1 KTAP version 1 # Subtest: rust_doctests_kernel 1..59 # rust_doctest_kernel_build_assert_rs_0.location: rust/kernel/build_assert.rs:13 ok 1 rust_doctest_kernel_build_assert_rs_0 # rust_doctest_kernel_build_assert_rs_1.location: rust/kernel/build_assert.rs:56 ok 2 rust_doctest_kernel_build_assert_rs_1 # rust_doctest_kernel_init_rs_0.location: rust/kernel/init.rs:122 ok 3 rust_doctest_kernel_init_rs_0 ... # rust_doctest_kernel_types_rs_2.location: rust/kernel/types.rs:150 ok 59 rust_doctest_kernel_types_rs_2 # rust_doctests_kernel: pass:59 fail:0 skip:0 total:59 # Totals: pass:59 fail:0 skip:0 total:59 ok 1 rust_doctests_kernel Therefore, add support for running Rust documentation tests in KUnit. Some other notes about the current implementation and support follow. The transformation is performed by a couple scripts written as Rust hostprogs. Tests using the `?` operator are also supported as usual, e.g.: /// ``` /// # use kernel::{spawn_work_item, workqueue}; /// spawn_work_item!(workqueue::system(), || pr_info!("x"))?; /// # Ok::<(), Error>(()) /// ``` The tests are also compiled with Clippy under `CLIPPY=1`, just like normal code, thus also benefitting from extra linting. The names of the tests are currently automatically generated. This allows to reduce the burden for documentation writers, while keeping them fairly stable for bisection. This is an improvement over the `rustdoc`-generated names, which include the line number; but ideally we would like to get `rustdoc` to provide the Rust item path and a number (for multiple examples in a single documented Rust item). In order for developers to easily see from which original line a failed doctests came from, a KTAP diagnostic line is printed to the log, containing the location (file and line) of the original test (i.e. instead of the location in the generated Rust file): # rust_doctest_kernel_types_rs_2.location: rust/kernel/types.rs:150 This line follows the syntax for declaring test metadata in the proposed KTAP v2 spec [1], which may be used for the proposed KUnit test attributes API [2]. Thus hopefully this will make migration easier later on (suggested by David [3]). The original line in that test attribute is figured out by providing an anchor (suggested by Boqun [4]). The original file is found by walking the filesystem, checking directory prefixes to reduce the amount of combinations to check, and it is only done once per file. Ambiguities are detected and reported. A notable difference from KUnit C tests is that the Rust tests appear to assert using the usual `assert!` and `assert_eq!` macros from the Rust standard library (`core`). We provide a custom version that forwards the call to KUnit instead. Importantly, these macros do not require passing context, unlike the KUnit C ones (i.e. `struct kunit *`). This makes them easier to use, and readers of the documentation do not need to care about which testing framework is used. In addition, it may allow us to test third-party code more easily in the future. However, a current limitation is that KUnit does not support assertions in other tasks. Thus we presently simply print an error to the kernel log if an assertion actually failed. This should be revisited to properly fail the test, perhaps saving the context somewhere else, or letting KUnit handle it. Link: https://lore.kernel.org/lkml/[email protected]/ [1] Link: https://lore.kernel.org/linux-kselftest/[email protected]/ [2] Link: https://lore.kernel.org/rust-for-linux/CABVgOSkOLO-8v6kdAGpmYnZUb+LKOX0CtYCo-Bge7r_2YTuXDQ@mail.gmail.com/ [3] Link: https://lore.kernel.org/rust-for-linux/ZIps86MbJF%2FiGIzd@boqun-archlinux/ [4] Signed-off-by: Miguel Ojeda <[email protected]> Reviewed-by: David Gow <[email protected]> Signed-off-by: Shuah Khan <[email protected]>
1 parent ed615fb commit a66d733

11 files changed

+555
-0
lines changed

lib/Kconfig.debug

+13
Original file line numberDiff line numberDiff line change
@@ -3010,6 +3010,19 @@ config RUST_BUILD_ASSERT_ALLOW
30103010

30113011
If unsure, say N.
30123012

3013+
config RUST_KERNEL_DOCTESTS
3014+
bool "Doctests for the `kernel` crate" if !KUNIT_ALL_TESTS
3015+
depends on RUST && KUNIT=y
3016+
default KUNIT_ALL_TESTS
3017+
help
3018+
This builds the documentation tests of the `kernel` crate
3019+
as KUnit tests.
3020+
3021+
For more information on KUnit and unit tests in general,
3022+
please refer to the KUnit documentation in Documentation/dev-tools/kunit/.
3023+
3024+
If unsure, say N.
3025+
30133026
endmenu # "Rust"
30143027

30153028
endmenu # Kernel hacking

rust/.gitignore

+2
Original file line numberDiff line numberDiff line change
@@ -2,6 +2,8 @@
22

33
bindings_generated.rs
44
bindings_helpers_generated.rs
5+
doctests_kernel_generated.rs
6+
doctests_kernel_generated_kunit.c
57
uapi_generated.rs
68
exports_*_generated.h
79
doc/

rust/Makefile

+29
Original file line numberDiff line numberDiff line change
@@ -27,6 +27,12 @@ endif
2727

2828
obj-$(CONFIG_RUST) += exports.o
2929

30+
always-$(CONFIG_RUST_KERNEL_DOCTESTS) += doctests_kernel_generated.rs
31+
always-$(CONFIG_RUST_KERNEL_DOCTESTS) += doctests_kernel_generated_kunit.c
32+
33+
obj-$(CONFIG_RUST_KERNEL_DOCTESTS) += doctests_kernel_generated.o
34+
obj-$(CONFIG_RUST_KERNEL_DOCTESTS) += doctests_kernel_generated_kunit.o
35+
3036
# Avoids running `$(RUSTC)` for the sysroot when it may not be available.
3137
ifdef CONFIG_RUST
3238

@@ -39,9 +45,11 @@ ifeq ($(quiet),silent_)
3945
cargo_quiet=-q
4046
rust_test_quiet=-q
4147
rustdoc_test_quiet=--test-args -q
48+
rustdoc_test_kernel_quiet=>/dev/null
4249
else ifeq ($(quiet),quiet_)
4350
rust_test_quiet=-q
4451
rustdoc_test_quiet=--test-args -q
52+
rustdoc_test_kernel_quiet=>/dev/null
4553
else
4654
cargo_quiet=--verbose
4755
endif
@@ -157,6 +165,27 @@ quiet_cmd_rustdoc_test = RUSTDOC T $<
157165
-L$(objtree)/$(obj)/test --output $(objtree)/$(obj)/doc \
158166
--crate-name $(subst rusttest-,,$@) $<
159167

168+
quiet_cmd_rustdoc_test_kernel = RUSTDOC TK $<
169+
cmd_rustdoc_test_kernel = \
170+
rm -rf $(objtree)/$(obj)/test/doctests/kernel; \
171+
mkdir -p $(objtree)/$(obj)/test/doctests/kernel; \
172+
OBJTREE=$(abspath $(objtree)) \
173+
$(RUSTDOC) --test $(rust_flags) \
174+
@$(objtree)/include/generated/rustc_cfg \
175+
-L$(objtree)/$(obj) --extern alloc --extern kernel \
176+
--extern build_error --extern macros \
177+
--extern bindings --extern uapi \
178+
--no-run --crate-name kernel -Zunstable-options \
179+
--test-builder $(objtree)/scripts/rustdoc_test_builder \
180+
$< $(rustdoc_test_kernel_quiet); \
181+
$(objtree)/scripts/rustdoc_test_gen
182+
183+
%/doctests_kernel_generated.rs %/doctests_kernel_generated_kunit.c: \
184+
$(src)/kernel/lib.rs $(obj)/kernel.o \
185+
$(objtree)/scripts/rustdoc_test_builder \
186+
$(objtree)/scripts/rustdoc_test_gen FORCE
187+
$(call if_changed,rustdoc_test_kernel)
188+
160189
# We cannot use `-Zpanic-abort-tests` because some tests are dynamic,
161190
# so for the moment we skip `-Cpanic=abort`.
162191
quiet_cmd_rustc_test = RUSTC T $<

rust/bindings/bindings_helper.h

+1
Original file line numberDiff line numberDiff line change
@@ -6,6 +6,7 @@
66
* Sorted alphabetically.
77
*/
88

9+
#include <kunit/test.h>
910
#include <linux/errname.h>
1011
#include <linux/slab.h>
1112
#include <linux/refcount.h>

rust/helpers.c

+7
Original file line numberDiff line numberDiff line change
@@ -18,6 +18,7 @@
1818
* accidentally exposed.
1919
*/
2020

21+
#include <kunit/test-bug.h>
2122
#include <linux/bug.h>
2223
#include <linux/build_bug.h>
2324
#include <linux/err.h>
@@ -135,6 +136,12 @@ void rust_helper_put_task_struct(struct task_struct *t)
135136
}
136137
EXPORT_SYMBOL_GPL(rust_helper_put_task_struct);
137138

139+
struct kunit *rust_helper_kunit_get_current_test(void)
140+
{
141+
return kunit_get_current_test();
142+
}
143+
EXPORT_SYMBOL_GPL(rust_helper_kunit_get_current_test);
144+
138145
/*
139146
* We use `bindgen`'s `--size_t-is-usize` option to bind the C `size_t` type
140147
* as the Rust `usize` type, so we can use it in contexts where Rust

rust/kernel/kunit.rs

+163
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,163 @@
1+
// SPDX-License-Identifier: GPL-2.0
2+
3+
//! KUnit-based macros for Rust unit tests.
4+
//!
5+
//! C header: [`include/kunit/test.h`](../../../../../include/kunit/test.h)
6+
//!
7+
//! Reference: <https://docs.kernel.org/dev-tools/kunit/index.html>
8+
9+
use core::{ffi::c_void, fmt};
10+
11+
/// Prints a KUnit error-level message.
12+
///
13+
/// Public but hidden since it should only be used from KUnit generated code.
14+
#[doc(hidden)]
15+
pub fn err(args: fmt::Arguments<'_>) {
16+
// SAFETY: The format string is null-terminated and the `%pA` specifier matches the argument we
17+
// are passing.
18+
#[cfg(CONFIG_PRINTK)]
19+
unsafe {
20+
bindings::_printk(
21+
b"\x013%pA\0".as_ptr() as _,
22+
&args as *const _ as *const c_void,
23+
);
24+
}
25+
}
26+
27+
/// Prints a KUnit info-level message.
28+
///
29+
/// Public but hidden since it should only be used from KUnit generated code.
30+
#[doc(hidden)]
31+
pub fn info(args: fmt::Arguments<'_>) {
32+
// SAFETY: The format string is null-terminated and the `%pA` specifier matches the argument we
33+
// are passing.
34+
#[cfg(CONFIG_PRINTK)]
35+
unsafe {
36+
bindings::_printk(
37+
b"\x016%pA\0".as_ptr() as _,
38+
&args as *const _ as *const c_void,
39+
);
40+
}
41+
}
42+
43+
/// Asserts that a boolean expression is `true` at runtime.
44+
///
45+
/// Public but hidden since it should only be used from generated tests.
46+
///
47+
/// Unlike the one in `core`, this one does not panic; instead, it is mapped to the KUnit
48+
/// facilities. See [`assert!`] for more details.
49+
#[doc(hidden)]
50+
#[macro_export]
51+
macro_rules! kunit_assert {
52+
($name:literal, $file:literal, $diff:expr, $condition:expr $(,)?) => {
53+
'out: {
54+
// Do nothing if the condition is `true`.
55+
if $condition {
56+
break 'out;
57+
}
58+
59+
static FILE: &'static $crate::str::CStr = $crate::c_str!($file);
60+
static LINE: i32 = core::line!() as i32 - $diff;
61+
static CONDITION: &'static $crate::str::CStr = $crate::c_str!(stringify!($condition));
62+
63+
// SAFETY: FFI call without safety requirements.
64+
let kunit_test = unsafe { $crate::bindings::kunit_get_current_test() };
65+
if kunit_test.is_null() {
66+
// The assertion failed but this task is not running a KUnit test, so we cannot call
67+
// KUnit, but at least print an error to the kernel log. This may happen if this
68+
// macro is called from an spawned thread in a test (see
69+
// `scripts/rustdoc_test_gen.rs`) or if some non-test code calls this macro by
70+
// mistake (it is hidden to prevent that).
71+
//
72+
// This mimics KUnit's failed assertion format.
73+
$crate::kunit::err(format_args!(
74+
" # {}: ASSERTION FAILED at {FILE}:{LINE}\n",
75+
$name
76+
));
77+
$crate::kunit::err(format_args!(
78+
" Expected {CONDITION} to be true, but is false\n"
79+
));
80+
$crate::kunit::err(format_args!(
81+
" Failure not reported to KUnit since this is a non-KUnit task\n"
82+
));
83+
break 'out;
84+
}
85+
86+
#[repr(transparent)]
87+
struct Location($crate::bindings::kunit_loc);
88+
89+
#[repr(transparent)]
90+
struct UnaryAssert($crate::bindings::kunit_unary_assert);
91+
92+
// SAFETY: There is only a static instance and in that one the pointer field points to
93+
// an immutable C string.
94+
unsafe impl Sync for Location {}
95+
96+
// SAFETY: There is only a static instance and in that one the pointer field points to
97+
// an immutable C string.
98+
unsafe impl Sync for UnaryAssert {}
99+
100+
static LOCATION: Location = Location($crate::bindings::kunit_loc {
101+
file: FILE.as_char_ptr(),
102+
line: LINE,
103+
});
104+
static ASSERTION: UnaryAssert = UnaryAssert($crate::bindings::kunit_unary_assert {
105+
assert: $crate::bindings::kunit_assert {},
106+
condition: CONDITION.as_char_ptr(),
107+
expected_true: true,
108+
});
109+
110+
// SAFETY:
111+
// - FFI call.
112+
// - The `kunit_test` pointer is valid because we got it from
113+
// `kunit_get_current_test()` and it was not null. This means we are in a KUnit
114+
// test, and that the pointer can be passed to KUnit functions and assertions.
115+
// - The string pointers (`file` and `condition` above) point to null-terminated
116+
// strings since they are `CStr`s.
117+
// - The function pointer (`format`) points to the proper function.
118+
// - The pointers passed will remain valid since they point to `static`s.
119+
// - The format string is allowed to be null.
120+
// - There are, however, problems with this: first of all, this will end up stopping
121+
// the thread, without running destructors. While that is problematic in itself,
122+
// it is considered UB to have what is effectively a forced foreign unwind
123+
// with `extern "C"` ABI. One could observe the stack that is now gone from
124+
// another thread. We should avoid pinning stack variables to prevent library UB,
125+
// too. For the moment, given that test failures are reported immediately before the
126+
// next test runs, that test failures should be fixed and that KUnit is explicitly
127+
// documented as not suitable for production environments, we feel it is reasonable.
128+
unsafe {
129+
$crate::bindings::__kunit_do_failed_assertion(
130+
kunit_test,
131+
core::ptr::addr_of!(LOCATION.0),
132+
$crate::bindings::kunit_assert_type_KUNIT_ASSERTION,
133+
core::ptr::addr_of!(ASSERTION.0.assert),
134+
Some($crate::bindings::kunit_unary_assert_format),
135+
core::ptr::null(),
136+
);
137+
}
138+
139+
// SAFETY: FFI call; the `test` pointer is valid because this hidden macro should only
140+
// be called by the generated documentation tests which forward the test pointer given
141+
// by KUnit.
142+
unsafe {
143+
$crate::bindings::__kunit_abort(kunit_test);
144+
}
145+
}
146+
};
147+
}
148+
149+
/// Asserts that two expressions are equal to each other (using [`PartialEq`]).
150+
///
151+
/// Public but hidden since it should only be used from generated tests.
152+
///
153+
/// Unlike the one in `core`, this one does not panic; instead, it is mapped to the KUnit
154+
/// facilities. See [`assert!`] for more details.
155+
#[doc(hidden)]
156+
#[macro_export]
157+
macro_rules! kunit_assert_eq {
158+
($name:literal, $file:literal, $diff:expr, $left:expr, $right:expr $(,)?) => {{
159+
// For the moment, we just forward to the expression assert because, for binary asserts,
160+
// KUnit supports only a few types (e.g. integers).
161+
$crate::kunit_assert!($name, $file, $diff, $left == $right);
162+
}};
163+
}

rust/kernel/lib.rs

+2
Original file line numberDiff line numberDiff line change
@@ -34,6 +34,8 @@ mod build_assert;
3434
pub mod error;
3535
pub mod init;
3636
pub mod ioctl;
37+
#[cfg(CONFIG_KUNIT)]
38+
pub mod kunit;
3739
pub mod prelude;
3840
pub mod print;
3941
mod static_assert;

scripts/.gitignore

+2
Original file line numberDiff line numberDiff line change
@@ -5,6 +5,8 @@
55
/kallsyms
66
/module.lds
77
/recordmcount
8+
/rustdoc_test_builder
9+
/rustdoc_test_gen
810
/sign-file
911
/sorttable
1012
/target.json

scripts/Makefile

+4
Original file line numberDiff line numberDiff line change
@@ -9,6 +9,8 @@ hostprogs-always-$(CONFIG_BUILDTIME_TABLE_SORT) += sorttable
99
hostprogs-always-$(CONFIG_ASN1) += asn1_compiler
1010
hostprogs-always-$(CONFIG_MODULE_SIG_FORMAT) += sign-file
1111
hostprogs-always-$(CONFIG_SYSTEM_EXTRA_CERTIFICATE) += insert-sys-cert
12+
hostprogs-always-$(CONFIG_RUST_KERNEL_DOCTESTS) += rustdoc_test_builder
13+
hostprogs-always-$(CONFIG_RUST_KERNEL_DOCTESTS) += rustdoc_test_gen
1214
always-$(CONFIG_RUST) += target.json
1315

1416
filechk_rust_target = $< < include/config/auto.conf
@@ -18,6 +20,8 @@ $(obj)/target.json: scripts/generate_rust_target include/config/auto.conf FORCE
1820

1921
hostprogs += generate_rust_target
2022
generate_rust_target-rust := y
23+
rustdoc_test_builder-rust := y
24+
rustdoc_test_gen-rust := y
2125

2226
HOSTCFLAGS_sorttable.o = -I$(srctree)/tools/include
2327
HOSTLDLIBS_sorttable = -lpthread

scripts/rustdoc_test_builder.rs

+72
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,72 @@
1+
// SPDX-License-Identifier: GPL-2.0
2+
3+
//! Test builder for `rustdoc`-generated tests.
4+
//!
5+
//! This script is a hack to extract the test from `rustdoc`'s output. Ideally, `rustdoc` would
6+
//! have an option to generate this information instead, e.g. as JSON output.
7+
//!
8+
//! The `rustdoc`-generated test names look like `{file}_{line}_{number}`, e.g.
9+
//! `...path_rust_kernel_sync_arc_rs_42_0`. `number` is the "test number", needed in cases like
10+
//! a macro that expands into items with doctests is invoked several times within the same line.
11+
//!
12+
//! However, since these names are used for bisection in CI, the line number makes it not stable
13+
//! at all. In the future, we would like `rustdoc` to give us the Rust item path associated with
14+
//! the test, plus a "test number" (for cases with several examples per item) and generate a name
15+
//! from that. For the moment, we generate ourselves a new name, `{file}_{number}` instead, in
16+
//! the `gen` script (done there since we need to be aware of all the tests in a given file).
17+
18+
use std::io::Read;
19+
20+
fn main() {
21+
let mut stdin = std::io::stdin().lock();
22+
let mut body = String::new();
23+
stdin.read_to_string(&mut body).unwrap();
24+
25+
// Find the generated function name looking for the inner function inside `main()`.
26+
//
27+
// The line we are looking for looks like one of the following:
28+
//
29+
// ```
30+
// fn main() { #[allow(non_snake_case)] fn _doctest_main_rust_kernel_file_rs_28_0() {
31+
// fn main() { #[allow(non_snake_case)] fn _doctest_main_rust_kernel_file_rs_37_0() -> Result<(), impl core::fmt::Debug> {
32+
// ```
33+
//
34+
// It should be unlikely that doctest code matches such lines (when code is formatted properly).
35+
let rustdoc_function_name = body
36+
.lines()
37+
.find_map(|line| {
38+
Some(
39+
line.split_once("fn main() {")?
40+
.1
41+
.split_once("fn ")?
42+
.1
43+
.split_once("()")?
44+
.0,
45+
)
46+
.filter(|x| x.chars().all(|c| c.is_alphanumeric() || c == '_'))
47+
})
48+
.expect("No test function found in `rustdoc`'s output.");
49+
50+
// Qualify `Result` to avoid the collision with our own `Result` coming from the prelude.
51+
let body = body.replace(
52+
&format!("{rustdoc_function_name}() -> Result<(), impl core::fmt::Debug> {{"),
53+
&format!("{rustdoc_function_name}() -> core::result::Result<(), impl core::fmt::Debug> {{"),
54+
);
55+
56+
// For tests that get generated with `Result`, like above, `rustdoc` generates an `unwrap()` on
57+
// the return value to check there were no returned errors. Instead, we use our assert macro
58+
// since we want to just fail the test, not panic the kernel.
59+
//
60+
// We save the result in a variable so that the failed assertion message looks nicer.
61+
let body = body.replace(
62+
&format!("}} {rustdoc_function_name}().unwrap() }}"),
63+
&format!("}} let test_return_value = {rustdoc_function_name}(); assert!(test_return_value.is_ok()); }}"),
64+
);
65+
66+
// Figure out a smaller test name based on the generated function name.
67+
let name = rustdoc_function_name.split_once("_rust_kernel_").unwrap().1;
68+
69+
let path = format!("rust/test/doctests/kernel/{name}");
70+
71+
std::fs::write(path, body.as_bytes()).unwrap();
72+
}

0 commit comments

Comments
 (0)