Skip to content

fs: implement tokio::fs::create_dir[_all] via io_uring#7847

Open
tahmid-23 wants to merge 4 commits intotokio-rs:masterfrom
tahmid-23:tahmid-23/uring-mkdir
Open

fs: implement tokio::fs::create_dir[_all] via io_uring#7847
tahmid-23 wants to merge 4 commits intotokio-rs:masterfrom
tahmid-23:tahmid-23/uring-mkdir

Conversation

@tahmid-23
Copy link
Copy Markdown
Contributor

io_uring + mkdirat()

As opposed to std's implementation of recursive directory creation, this does not use a recursive function to create parent directories. Instead, it iterates over positions of / (the unix / linux path separator), first failing to create parent directories until success, and then creating all children.

This prevents us from allocating a boxed future for each recursive function call.
It does have the minimal overhead of needing to find the / positions an additional time in the second forward iterator. We could save this by allocating a Vec of / positions; I don't necessarily know what's better.

@ADD-SP ADD-SP added A-tokio Area: The main tokio crate M-fs Module: tokio/fs T-io-uring Topic: Linux io_uring labels Jan 10, 2026
Copy link
Copy Markdown
Member

@martin-g martin-g left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The build fails with:

error[E0277]: the `?` operator can only be applied to values that implement `std::ops::Try`
  --> tokio/src/fs/create_dir.rs:60:12
   |
60 |         if driver_handle.check_and_init(io_uring::opcode::MkDirAt::CODE)? {
   |            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ the `?` operator cannot be applied to type `impl std::future::Future<Output = std::result::Result<bool, std::io::Error>>`
   |
   = help: the trait `std::ops::Try` is not implemented for `impl std::future::Future<Output = std::result::Result<bool, std::io::Error>>`
help: consider `await`ing on the `Future`

{
let parent_bytes = &path_bytes[..separator_pos];
let parent_str = OsStr::from_bytes(parent_bytes);
let parent_path = Path::new(parent_str);
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why not use Path::parent() ?

//
// TODO: We're attempting to create all directories sequentially.
// This would benefit from batching.
for (separator_pos, _) in path_bytes
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

And here you can use Path::components()

Ok(()) => Ok(true),
Err(ref e) if e.kind() == io::ErrorKind::NotFound => Ok(false),
// TODO: replace with uring-based statx
Err(_) if crate::fs::metadata(path).await?.is_dir() => Ok(true),
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

metadata(path).await? may hide the original error (the ignored one in Err(_))

Suggested change
Err(_) if crate::fs::metadata(path).await?.is_dir() => Ok(true),
Err(ref e) if e.kind() == io::ErrorKind::AlreadyExists => {
if crate::fs::metadata(path).await?.is_dir() {
Ok(true)
} else {
Err(e)
}
}

something like this should be better

match Op::mkdir(path)?.await {
Ok(()) => Ok(()),
// TODO: replace with uring-based statx
Err(_) if crate::fs::metadata(path).await?.is_dir() => Ok(()),
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Same as above

feature = "rt",
feature = "fs",
target_os = "linux"
))]
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This cfg guard is not really needed.
This module is already conditionally used with cfg_io_uring! in mod.rs

Comment on lines +7 to +8
use std::io;
use std::io::Error;
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
use std::io;
use std::io::Error;
use std::io::{self, Error};

use tokio_test::assert_pending;
use tokio_util::task::TaskTracker;

fn multi_rt(n: usize) -> Box<dyn Fn() -> Runtime> {
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Those seem copied from one of the other tests.
Let's extract them into a helper module.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

A-tokio Area: The main tokio crate M-fs Module: tokio/fs T-io-uring Topic: Linux io_uring

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants