Repository navigation
Dart SIMD use case. #5
Description
Hello!
I just wanted to briefly share some use cases for this in the Dart world.
We are working on adding better SIMD support to the dart2wasm compiler in Dart.
One of the use cases we can't support right now is an efficient SIMD based scan.
To be able to efficiently scan through a byte stream (or a string to implement the memchr equivalent in Dart called indexOf) using SIMD we'd need to have the ability to reinterpret parts of the backing storage as appropriate types that allow us to cheaply treat a chunk of it as a SIMD type.
Here's some more Dart use cases from another perspective from @osa1 in WebAssembly/gc#395 (comment)
An 8x unrolled SIMD based scan can achieve 120GB/s on a modern mac using NEON. It would be great if use cases like these could be opened up with this proposal. Experiments show that we are at least an order of magnitude away from being able to reach those numbers with wasm gc.
Activity
brendandahl commentedon Jul 21, 2026
The proposal has moved to phase 2 and we are looking into implementing instructions in binaryen and V8. I was curious if there are any instructions that would be particularly useful for Dart that we should focus on first? List of instructions here.
modulovalue commentedon Jul 21, 2026
Thanks @brendandahl, I'm very happy to read this!
It looks to me like scalar multibyte loads/stores from
i8arrays:i64.load,i64.load16_s/u,i64.load32_s/u,f32.load,f64.load, plus the matching stores would be most relevant to Dart users.I'd also love to see
v128.loadandv128.store, but they are less important at the moment.cc @kevmoo
modulovalue commentedon Sep 13, 2026
@brendandahl I wanted to make sure the proposal covers what I need for my SIMD efforts in Dart, so I implemented intrinsics for
v128.load&v128.storein dart2wasm (and let an agent implement and expose those ops in an v8 fork) and the performance numbers are excellent! (You can find the v8 work here: https://github.com/modulovalue/v8/tree/wasm-multibyte-array-access-v128)If there's anything you need help with so we can land it earlier (e.g., an early language implementation) let me know!
Note that a potential inconsistency was found in the specification, see: #9
brendandahl commentedon Sep 16, 2026
Thanks for testing it out! Do you know of any other places in dart could use the other (non-simd) multibyte instructions that would be useful (e.g. something like a bytebuffer)? It'd be useful to show performance/code size savings with the multibyte instructions.
modulovalue commentedon Sep 16, 2026
I think the closest to your example is the following: https://github.com/dart-lang/sdk/blob/16542c389cc6f999ed5dd796a025252fc7e48d7c/sdk/lib/_internal/wasm/common/typed_data.dart#L311 it allows us to extract a uint64 from anything that offers byte access. We can't specialize that to a wasm array (so we don't reconstruct the uint64 byte-by-byte) because we are missing the reinterpretation API that your proposal offers.
It looks like the _getUint64Unchecked directly under it would turn into one instruction with this proposal.
If more concrete use cases would help you, let me know!
Does SWAR (SIMD-within-a-register) count as non-SIMD? 😃
Liedtke commentedon Sep 20, 2026
My main concern about this proposal is that it breaks aliasing rules. So far a
v128store on the JS heap cannot alias with ani32load.Our (V8's) optimizations make use of this information mainly for load-elimination.
This happens on a rather low level where we might not even know any more if something is a Wasm or a JS object, still, the engine assumes so far that this holds true on all these objects.
It is a bit unclear to me how this should be updated for the new proposal without regressing performance for use cases that don't use this new proposal. (For functions using the new instruction, the aliasing logic clearly would need to be pessimized with unclear performance costs.)
The issue above also makes these performance numbers unreliable, see this test case for the wrong behavior. For a fair comparison, we'd need to have an implementation that handles the relaxed aliasing correctly. It also may not pessimize the aliasing generically, as pointed out above, this happens on a rather low level and is shared between Wasm and JS.
Another note that could impact performance: Heap objects in V8 are currently all(?) 4-Byte-aligned by default1 (differently to the Wasm memory that always has at least full page alignment). So the v128 loads and stores would need to be marked as unaligned which doesn't seem to be the case in the current vibe-coded implementation.
Footnotes
The only exception I know about are the shared Wasm objects, (which are very much work in progress) which need 8 Byte-alignment for lock-free atomic accesses. ↩
tlively commentedon Sep 21, 2026
Despite the common binary format, Binaryen's IR treats array accesses and memory accesses as separate kinds of instructions, so the array accesses can have the same kind of aliasing as array.get/set while the memory accesses remain unchanged. Hopefully a similarly simple fix can be applied in V8?
Liedtke commentedon Sep 21, 2026
In general sure, we'd need to add some kind of "don't load-eliminate, even though it's a heap load" bit but my concern from above is:
We'd say "this new instructions are worth it to have worse load elimination (or if we talk about simple fixes no load elimination) for Wasm GC array accesses". How could we be certain that this proposal is an improvement compared to having strict aliasing rules and benefiting from it?
tlively commentedon Sep 21, 2026
If you're concerned about the interaction with memory accesses: For modules (or just functions) that don't use these new instructions, you still just have array.get/set and linear memory accesses, which you know don't interfere with each other (ignoring atomic operations) by construction. So if you model array loads and stores as separate instructions from array gets and sets, I don't see why existing code would regress.
If you're concerned about the interaction with array gets and sets: Yes, the optimization will have to be updated to know about the new ways of accessing arrays. But this doesn't seem any different from such an optimization having to know about e.g. array.init_{elem,data}, array.copy, or array.fill, right?
Liedtke commentedon Sep 21, 2026
Yes, my concern is about the
array.get/array.setcases.For linear memory we don't even attempt to optimize, you can put the exact same load twice directly after each other and we will never remove any of them. :)
It is different. The described operations have two options depending on whether we "inline" them or not:
array.initperforms aWord32store if we have[i32]or onWord64if we have[i64]. Any 128-bit load/store will never alias with any such operations.memcpy-like operation that uses SIMD. --> This happens in a builtin that can read-write memory and will have "arbitrary" side effects invalidating most loads.So right now there isn't any precedent (afaict) neither for Wasm nor for JS where loads and stores of different memory sizes can alias with each other (on the JS heap, "tagged base" in V8).
It would probably not be too difficult to model the new operations as (2): Handle them as arbitrary read / writes, so they will invalidate almost everything because we don't know that e.g. the shape load of the array can't alias with the
v128store of this proposal.Anything else would require much more complicated infrastructure, as mentioned above this happens on "low-level" loads, so a user-space wasm-gc object access looks exactly the same as an engine-generated access to e.g. the object's type (other than that they have different offsets.)
There is a risk that if we invalidate "almost everything", these instructions might have a higher implicit performance cost than they bring as benefit?
tlively commentedon Sep 21, 2026
I see, this optimization is happening on a lower-level representation of the instructions than I had in mind. I agree that this is important to consider.
simolus3 commentedon Oct 1, 2026
This might be a silly question, but
Does this only apply to
v128? Because surely for all other sizes, typed arrays already prevent that optimization in JS? (or are these not "on heap"?)Dart uses numeric wasm arrays pretty much exclusively to emulate typed arrays (and being unable to create views of different types for the same buffer is a pain even outside of SIMD use cases). I assume other high-level languages need the same thing (e.g. for
ByteBuffer.wrapin Java). Are there widely-used modules where this regression has a significant impact? I struggle to see how this would impact Dart much, but obviously I don't have the experience from other languages which might use arrays differently.Liedtke commentedon Oct 5, 2026
The
ArrayBuffer's data section is indeed not on the heap. The heap is a 4 GB virtual address region that we use for the V8 heap sandbox. AsArrayBuffers are used for large amounts of data in some cases (especially Wasm memories), they live outside the heap.I am no expert with our load-elimination and I don't know how much benefit we get from the strict-aliasing requirements. However, just because it exists there are probably enough examples where this makes a difference.
E.g. a simple for loop could already be impacted:
Right now we can already argue based on the load/store types that the
items.lengthfield cannot be overwritten by the store inside the loop as these are two incompatible representations.Otherwise the load elimination needs to prove that the
items.lengthand theitems[i]can never point to the same offset which might be easy in this case but might be much more difficult if there is a store into a struct with a field at the same offset as the length offset of an array. As mentioned above, this specific load elimination for arrays happens at a very low level (we also have a higher-level load elimination that currently ignores arrays), so it is much more difficult to guarantee that two references cannot alias because we don't know any more that they might have incompatible Wasm types.All of this is based on the "Hopefully a similarly simple fix can be applied in V8?", so I am not saying that we can't fix this, I'm just saying that there isn't a trivial easy fix that guarantees that we don't regress the performance for functions that don't make use of this new feature (whether that be JavaScript or current wasm-gc).
simolus3 commentedon Oct 15, 2026
From the "For linear memory we don't even attempt to optimize" comment, it seems like LLVM passes for modules with linear memory are considered good enough. So couldn't we expect the same thing for WasmGC modules? Flatten and LICM passes in
wasm-optcan moveitems.lengthout of the loop block too. Dart currently doesn't run those (it always optimizes for size), but maybe if one needs load-elimination they should just pass other options towasm-opt🤷Would benchmarks on say Dart with patches to adopt this proposal help? Currently, Dart has to use a "fast" (backed by
array i8)and "slow" (originally backed by another array)Uint8Listwith virtual calls to access elements, which sounds a lot worse than superfluous loads. Dart has type analysis optimizations to help with this, but this would still be a big win for simplicity, code size and (I presume) performance.(but then again that's just Dart, maybe there are other modules that actually need the current optimizations so we can't be sure)
Liedtke commentedon Oct 16, 2026
I don't think we can expect the same. The languages targeting Wasm GC and those targeting linear memory Wasm are often fundamentally different already on the source level. In C++ there are reasonably few indirect calls while in Java all calls are virtual. While static code analysis can devirtualize a lot (
wasm-optprovides huge benefits here), performance is very much dependent on the engine doing speculative inlining (see e.g. this analysis for the Google Sheets calcworker performance in V8).These numbers have probably changed in the meantime, if you are interested in seeing whether these engine-side optimizations are relevant for your use case, you could run V8 with
--no-wasm-inlining --no-turboshaft-wasm-load-elimination --no-turboshaft-load-eliminationto simulate "what if V8 Wasm relied more on AOT optimizations in these specific areas".