Add an Emscripten/wasm implementation of primitives #822
Conversation
|
@microsoft-github-policy-service agree [company="google"] |
|
@microsoft-github-policy-service agree company="google" |
|
Ah this is awesome -- apologies for the later reaction! I love emscripten -- the Koka compiler uses it as well and it would be good to have a more native implementation. |
|
I merged your PR manually into the |
|
Great, thanks for merging! And sorry I didn't know the PR should target I'll close this PR then. Btw, recently I saw a 5x speedup using mimalloc over dlmalloc and we have other positive reports coming in from other Emscripten users. Thanks again for mimalloc! |
| // we may want to improve emmalloc to support such alignment. See also | ||
| // https://github.com/emscripten-core/emscripten/issues/20645 | ||
| #define MIN_EMMALLOC_ALIGN 8 | ||
| #define MAX_EMMALLOC_ALIGN (1024*1024) |
There was a problem hiding this comment.
Coming back to this: is it possible to make the MAX_EMMALLOC_ALIGN equal to MI_SEGMENT_ALIGN (=4MiB) ? As it is now, it will never use the right alignment, and mimalloc will retry with size + 4MiB and align within that area itself. (see os.c:mi_os_prim_alloc_aligned)
Thanks!
There was a problem hiding this comment.
Actually I see we fixed this in emmalloc in emscripten-core/emscripten#20704 😄
So, yes, the MAX_EMMALLOC_ALIGN can be removed.
There was a problem hiding this comment.
I opened emscripten-core/emscripten#21905 on Emscripten to verify that all tests pass with that, and after that will open a PR here.
|
fyi, we posted about the benefits of mimalloc in Emscripten here: https://web.dev/articles/scaling-multithreaded-webassembly-applications?hl=en#mimalloc Thanks again for the great allocator! |
This PR adds a
primimplementation for the Emscripten wasm toolchain. There is already a WASI wasm prim in mimalloc, which this is separate from. WASI tends to be used on the server, and Emscripten on the browser, and there are API differences that led to a different design here. Another difference is that Emscripten has stable support for multithreading, and so that part is fully implemented here (it was the main reason for the port, actually - to get a malloc that scales well with multiple cores).The overall idea in this implementation is to build on top of
emmallocas the "system allocator". emmalloc is a minimal allocator in Emscripten, so it can be used to provide a VirtualAlloc-like API. This avoids using sbrk directly, which could lead to problems with returning memory to the OS (fragmentation and leaks).There is a PR up on the Emscripten repo with all the integration into our toolchain so that people can test it out there. Also some charts with numbers, which look great!
Thank you for the mimalloc project ❤️ it was very easy to build support for, and as the numbers there show now we have an option for a malloc implementation that looks like it scales really well, in fact just as good as native builds.