WASI Modularity

[lachlansneff]

As discussed before (in the CG meeting, offline, and several threads), WASI should be designed to be split up into several modules/namespaces, each with a particular theme.

I'd like to discuss the granularity of the modules, a naming scheme, and versioning.

1. Granularity:

How far should syscalls be split up into modules? Would it work to have a single "fs" module, or would it make sense to split into an "fs" module that works with the filesystem and a "fd" module that works with file-descriptors? How much should the the "wasi core" module contain?

2. Naming:

Is there a preference on the naming scheme used? I'm personally most comfortable with modules being named wasi/<name> or wasi:<name>.

3. Versioning:

Should the WASI modules be versioned? Would we use semver, or something different? My opinion is that there are a lot of upsides to explicit versioning, but maybe it'd be better to keep things under the wasi_unstable umbrella until WASI is standardized.

[sunfishcode]

For reference, the earlier threads are bytecodealliance/wasmtime#67, bytecodealliance/wasmtime#69, and bytecodealliance/wasmtime#74.

[programmerjake]

I would like to split the modules in such a way that there is a subset of modules that only support calls to completely deterministic operations (ignoring multithreading). This will be useful for game engines so that they can memoize the results of running a particular operation.

In particular, this means excluding random numbers and timers from the deterministic subset.

Maybe by splitting the core module into core_nd and core where all the timer and random number generation functions would be in core_nd and the rest of the functions would be in core.

The filesystem functions can be in core, but functions for network operations would be in core_nd or (better yet) a separate network module.

I am planning on using this to allow webassembly-based Rust plugins to compute the next state of a block in a minecraft-like game that uses hashlife (a tree-based memoizing cellular-automaton algorithm/data-structure) to compute/represent the state of the world.

[npmccallum]

I'm a fan of wasi:NAME:SEMVER.

[dumblob]

wasi:name:semver looks good to me, but I have one question to that - could we somehow make it shorter? E.g. by ignoring some parts of this 3-tuple depending on the context? I could imagine :name:semver (assuming we're already in the WASI namespace) or :name: (we don't care about version) or maybe wasi:: (import whole WASI - not the best idea) or even :: (import world - the worst idea 😉). Or maybe using wildcards?

[rossberg]

I'd recommend sticking to standard URI syntax/semantics instead of reinventing the wheel. For that, wasi:name:ver would need some tweaking, I believe.

[wanderer]

only support calls to completely deterministic operations

also useful for symmetric computation

[devsnek]

wasi/x/y and wasi:x/y both make sense to me, but wasi:x:y seems odd. however, wasi/x/y is an invalid url, while the other two are not. i'd say i like wasi:x/y the most.

[sbc100]

I wrote of a doc with some thoughts on this issue. The TLDR is that maybe we can and should avoid versioning completely (at least for as long as we can). Since we aren't building a packaging system, perhaps we can avoid a lot of the complexity.

This is what I was going to present on in the previous meeting, and I am happy to do so in the next meeting if necessary.

[dumblob]

@sbc100 there is a special permission needed to access the Google doc. Could you maybe make it publicly available (at least for those of us who do not want to participate in the Google ecosystem)?

[sbc100]

Sorry about that .. fixed.

[dumblob]

Well, I more or less agree with the whole document, though I'm convinced that sooner or later versioning in some form will be necessary. I really like the Go folks approach which defers all these hard decisions for later until the issue becomes urgent and just ensures, that there won't be any obstacles introduced which could constrain potential future extension.

In this case we could reserve some syntax or characters in module names for potential future use. Or somehow constrain the current WASI spec in other ways to make space for potential future versioning in any form. As the doc from @sbc100 describes, Go folks did a similar thing with their "string suffix" (which by "coincidence" has a specified form - e.g. /vN where N is a monotonically increasing positive integer number) which is only then needed when the original "version" (which is actually version-less) can't be extended and a completely new version needs to be introduced.

[sunfishcode]

Modularity has now landed in ephemeral, and simple module versioning (not semver, which I agree doesn't make sense at this level) is live now!