Rewrite WASI implemention from C to Rust

[sunfishcode]

Wasmtime's current WASI implementation is derived from what CloudABI calls its userspace emulator code. This was an expedient way to get to a relatively complete WASI implementation that we could use to do early API iteration and to prototype the rest of the ecosystem on, however it also serves as a reference implementation of sorts, as WASI and CloudABI are relatively thin on API-level documentation.

However, an implementation in Rust would be preferable, because it would be easier to maintain along with the rest of Wasmtime's Rust code, it would be safer (there will likely still be unsafe code, however that's still an improvement over all the code being effectively "unsafe"), and it'd allow us to reduce our dependencies on cmake and cc. It may even make sense to build WASI as a crate that can be independent of Wasmtime entirely, so that it can be used in other contexts.

And, since CloudABI considers this "emulation" code, it isn't as robust as we'd ideally like it to be. For example, poll_oneoff only supports one clock subscription, some obscure features aren't implemented yet, and there may be other limitations.

This is a big project, though we can implement it one step at a time, starting with simple functions and building up to more complex ones. The very first function to do is probably proc_exit, and possibly next the args_get/args_sizes_get/environ_get/environ_sizes_get functions.

I'll probably start work on this as I have time, though if anyone else wants to take a look at these, I'd be happy to mentor!

[kubkon]

I'd be happy to help out with this especially since it intersects with the areas of my interests #121.

[sunfishcode]

That'd be great! As I mentioned above, proc_exit and the args/environ functions are good ones to start with, as the implementations are fairly simple, so it'll be good for getting started.

[kubkon]

Awesome! So, I've played somewhat with the code, and to get an idea of how things are stitched together, I've had a stab at implementing proc_exit purely in Rust. You can see the changes in my fork kubkon/wasmtime.

To quickly sum them up though, I've removed wasmtime_ssp_proc_exit (and the relevant entries in posix.h), and moved the logic into separate submodule of syscalls/proc_exit.rs. I've replaced the call to wasmtime_ssp_proc_exit with ::std::process::exit so that proc_exit function is now implemented in Rust. I've left the C shims as they were however. Are we after completely removing those as well or are we going to continue storing C-style function pointers in the relevant PrimaryMaps in instantiate::instantiate_wasi? And more importantly, is this the direction you've had in mind?

[sunfishcode]

That's a good question. I don't have a specific plan in mind for how to handle that yet. I'll think about it, though if you have ideas about how best to do that, I'd be interested in hearing them too.

[sunfishcode]

Ok, here's a concrete idea. What if we leave the shim logic and the rest of syscalls.rs as is, and just replace the wasmtime_ssp_proc_exit calls right now? The logic in syscalls.rs for other system calls handles translating arguments from wasm32 to host values, including translating pointers, which is relatively mechanical, so it's nice to keep that code separate from the code that actually implements the main functionality. Does that make sense?

[kubkon]

SGTM! I'll draft it out and submit a PR, and I guess we can take it from there.

[sunfishcode]

Multiple people are interested in this, which is great! To help avoid duplicating work, here are a few rough groups of functions that people can start with, with the more complex ones toward the end.

• {environ, args}_sizes/get} (being worked on by @kubkon and/or Replace {environ,size}_{sizes_get,get} C code with Rust #126)
• prestat_*
• sched_yield
• random_get, clock_res_get, clock_time_get
• fd_sync, fd_datasync, fd_close, fd_advise (of the fd_* functions, let's do these first)
• fd_seek, fd_tell, fd_renumber,
• fd_fdstat_set_flags, fd_fdstat_set_rights
• fd_read, fd_write, fd_fdstat_get
• fd_pread, fd_pwrite,
• path_create_directory, path_remove_directory, path_unlink_file, path_filestat_get (of the path_* functions, let's do these first)
• path_filestat_get, path_filestat_set_times,
• path_rename, path_symlink
• fd_readdir
• path_open
• poll_oneoff

If you want to start one of these groups, please check here and then post about it, and we'll figure out the details as we go. As always, feel free to ask questions!

[pchickey]

Some of these already have implementations over in lucet-wasi. Lucet uses the same license as Cranelift, so you can lift the source into this project with just copyright attribution. Or you can just use them as a reference and design your own.

As a separate note, we could probably figure out a way for Lucet runtime and wasmtime to present the same API to this sort of function, so that the same crate could be used in both runtimes.

[kubkon]

@pchickey I don't mind lifting the implementation from lucet-wasi. What is your preference @sunfishcode? I also really like the idea of sharing the implementations between the runtimes!

If it's OK with everyone, I'd like to have a go at the prestat_*s; i.e., the fd_prestat_get and fd_prestat_dir_name.

EDIT: and while I'm at it, I can also have a look at sched_yield although initially probably rewriting as *nix compatible. Any guidance here (as well anywhere else in fact) will be much appreciated!