Self-hosting Emscripten

[lastmjs]

This issue will track self-hosting Emscripten, or in other words compiling/building Emscripten with Emscripten. The ultimate goal is to allow the web assembly toolchain to work in the browser, Node.js, and any other environment with a WebAssembly VM. This is a continuation of some of the issues discussed here: #6015

cc @kripken @juj

[lastmjs]

@kripken In light of this: #6350 , is continuing work on compiling fastcomp the best choice right now? Perhaps we should be working on compiling the llvm wasm backend to wasm instead. You mentioned here #6015 (comment) that a potential solution is to just go straight to the LLVM wasm backend. Maybe that is the best choice, since eventually the LLVM wasm backend will be Emscripten's default backend. How long do you think it will take for that backend to be the default? Also, are there any major disadvantages to not being able to compile to asm.js? I would personally rather work on the future-facing compiler, and not worry about getting a legacy Emscripten build of itself.

[dschuff]

I would recommend just starting with the wasm backend. I don't think it will be too long (my guess would be weeks rather than months?). The Disadvantage of not having asm.js support would primarily be targeting older environments that don't support wasm. Once you have wasm support you could also look at wasm2asm as a way to bring asm.js support if you want that (that should be quite straightforward since it's part of Binaryen already).

[lastmjs]

@dschuff That sounds good. Unless @kripken or @juj feel strongly against this, I think I'll move forward with the wasm backend.

[kripken]

Yeah, might as well use the wasm backend here. The porting work should be almost the same anyhow between the two backends, and the wasm backend is more important for the future.

The downside of not having asm.js support is significant, and I'm not sure when we'll get wasm2asm support, but I don't think that changes things enough.

[andrewrk]

This is the current problem to solve for this issue:

In file included from /home/andy/downloads/llvm-project/llvm/lib/Support/Path.cpp:1110:
/home/andy/downloads/llvm-project/llvm/lib/Support/Unix/Path.inc:410:13: error: no member named
      'f_flags' in 'statvfs'
  return !!(STATVFS_F_FLAG(Vfs) & MNT_LOCAL);
            ^~~~~~~~~~~~~~~~~~~
/home/andy/downloads/llvm-project/llvm/lib/Support/Unix/Path.inc:91:35: note: expanded from macro
      'STATVFS_F_FLAG'
#define STATVFS_F_FLAG(vfs) (vfs).f_flags
                            ~~~~~ ^
/home/andy/downloads/llvm-project/llvm/lib/Support/Unix/Path.inc:410:35: error: use of undeclared
      identifier 'MNT_LOCAL'
  return !!(STATVFS_F_FLAG(Vfs) & MNT_LOCAL);
                                  ^
2 errors generated.
shared:ERROR: compiler frontend failed to generate LLVM bitcode, halting

[sbc100]

Looks like Path.inc needs to add __EMSCRIPTEN__ to the list of platforms that call this f_flag and not f_flags.

[andrewrk]

Thanks for the tip @sbc100. I was able to get lib/Support/CMakeFiles/LLVMSupport.dir/Path.cpp.o to build with this patch:

--- a/llvm/lib/Support/Unix/Path.inc
+++ b/llvm/lib/Support/Unix/Path.inc
@@ -93,7 +93,7 @@ typedef uint_t uint;
 #define STATVFS_F_FRSIZE(vfs) static_cast<uint64_t>(vfs.f_bsize)
 #endif
 
-#if defined(__NetBSD__) || defined(__DragonFly__) || defined(__GNU__)
+#if defined(__NetBSD__) || defined(__DragonFly__) || defined(__GNU__) || defined(__EMSCRIPTEN__)
 #define STATVFS_F_FLAG(vfs) (vfs).f_flag
 #else
 #define STATVFS_F_FLAG(vfs) (vfs).f_flags
@@ -466,6 +466,8 @@ static bool is_local_impl(struct STATVFS &Vfs) {
 
   // vmount entry not found; "remote" is the conservative answer.
   return false;
+#elif defined(__EMSCRIPTEN__)
+  return true;
 #else
   return !!(STATVFS_F_FLAG(Vfs) & MNT_LOCAL);
 #endif

[andrewrk]

Here's the next issue I ran into:

[100%] Linking CXX executable ../../bin/llvm-tblgen
shared:WARNING: emcc: cannot find library "terminfo"
[100%] Built target llvm-tblgen
[ 10%] Built target LLVM-tablegen-host
Scanning dependencies of target AttributeCompatFuncTableGen
[ 10%] Building AttributesCompatFunc.inc...
/bin/sh: ../../NATIVE/bin/llvm-tblgen: Permission denied
make[2]: *** [lib/IR/CMakeFiles/AttributeCompatFuncTableGen.dir/build.make:93: lib/IR/AttributesCompatFunc.inc] Error 126
make[1]: *** [CMakeFiles/Makefile2:1445: lib/IR/CMakeFiles/AttributeCompatFuncTableGen.dir/all] Error 2
make: *** [Makefile:152: all] Error 2

It appears to have successfully built llvm-tblgen but tries to run it with bash rather than a javascript or webassembly interpreter:

[nix-shell:~/downloads/llvm-project/llvm/build-wasm]$ file bin/llvm-tblgen.js
bin/llvm-tblgen.js: ASCII text, with very long lines

[nix-shell:~/downloads/llvm-project/llvm/build-wasm]$ file bin/llvm-tblgen.wasm
bin/llvm-tblgen.wasm: WebAssembly (wasm) binary module version 0x1 (MVP)

This is on NixOS with llvm trunk (046d49a8dcbbb99596ace2592aa0386ddf2f6bce), emcc 1.38.28.

Configure & make lines:

emconfigure cmake .. -DCMAKE_INSTALL_PREFIX=$HOME/local/llvm9-wasm -DCMAKE_PREFIX_PATH=$HOME/local/llvm9-wasm -DCMAKE_BUILD_TYPE=Release -DLLVM_TARGETS_TO_BUILD="WebAssembly" -DLLVM_ENABLE_LIBXML2=OFF -DLLVM_INCLUDE_EXAMPLES=OFF -DLLVM_INCLUDE_TESTS=OFF
emmake make

[andrewrk]

One thing I'm a little confused about is this cmake output:

[ 10%] Linking CXX executable ../../bin/llvm-tblgen.js
[ 10%] Built target llvm-tblgen
Scanning dependencies of target CREATE_LLVM_NATIVE
[ 10%] Creating /home/andy/downloads/llvm-project/llvm/build-wasm/NATIVE...
[ 10%] Built target CREATE_LLVM_NATIVE
Scanning dependencies of target CONFIGURE_LLVM_NATIVE
[ 10%] Configuring NATIVE LLVM...
-- The C compiler identification is Clang 6.0.1
-- The CXX compiler identification is Clang 6.0.1
...
-- Check for working C compiler: /nix/store/b0p6qr06ljkcddf9nsfa92qjk21hfhp2-emscripten-1.38.28/share/emscripten/emcc
-- Check for working C compiler: /nix/store/b0p6qr06ljkcddf9nsfa92qjk21hfhp2-emscripten-1.38.28/share/emscripten/emcc -- works
...
-- Check for working CXX compiler: /nix/store/b0p6qr06ljkcddf9nsfa92qjk21hfhp2-emscripten-1.38.28/share/emscripten/em++
-- Check for working CXX compiler: /nix/store/b0p6qr06ljkcddf9nsfa92qjk21hfhp2-emscripten-1.38.28/share/emscripten/em++ -- works

I'm not sure where clang 6.0.1 is getting picked up. I've installed the default emscripten package from NixOS which as noted is 1.38.28. If I run clang --version I see clang version 7.1.0 (I could choose a newer one if that would help, but I wasn't sure if the emscripten package had its own dependency or if it was invoking the system compiler).

Also what is this "native" thing? I thought it was only going to build for WebAssembly? It appears to have already linked llvm-tblgen.js, and then it builds it again?

[davidar]

I figured I'd start with updating clang-in-browser to work with the latest LLVM master, and managed to get it to build, but clang.wasm fails at runtime with a "function signature mismatch" error which I haven't yet been able to debug.

In case it's useful to anyone else trying to build: The only change to the LLVM source I needed to make was to disable this rule which sets the -allow-shlib-undefined linker flag which isn't compatible with wasm-ld. Otherwise these cmake flags seem to do the trick.

Edit: difficult to debug, as compiling clang.wasm with Debug flags results in wasm-ld segfaulting at

#0  __memmove_sse2_unaligned_erms () at ../sysdeps/x86_64/multiarch/memmove-vec-unaligned-erms.S:249
#1  0x00000000006bc8f9 in lld::wasm::InputChunk::writeTo(unsigned char*) const ()
#2  0x00000000006f048d in lld::wasm::CustomSection::writeTo(unsigned char*) ()
#3  0x00000000006ebf6d in std::_Function_handler<void (), void llvm::parallel::detail::parallel_for_each<__gnu_cxx::__normal_iterator<lld::wasm::OutputSection**, std::vector<lld::wasm::OutputSection*, std::allocator<lld::wasm::OutputSection*> > >, lld::wasm::(anonymous namespace)::Writer::writeSections()::$_0>(__gnu_cxx::__normal_iterator<lld::wasm::OutputSection**, std::vector<lld::wasm::OutputSection*, std::allocator<lld::wasm::OutputSection*> > >, __gnu_cxx::__normal_iterator<lld::wasm::OutputSection**, std::vector<lld::wasm::OutputSection*, std::allocator<lld::wasm::OutputSection*> > >, lld::wasm::(anonymous namespace)::Writer::writeSections()::$_0)::{lambda()#1}>::_M_invoke(std::_Any_data const&) ()
#4  0x00007fb8911b98c5 in std::_Function_handler<void (), llvm::parallel::detail::TaskGroup::spawn(std::function<void ()>)::$_0>::_M_invoke(std::_Any_data const&) ()
   from emsdk/upstream/bin/../lib/libLLVM-10svn.so
#5  0x00007fb8911b94a7 in llvm::parallel::detail::(anonymous namespace)::ThreadPoolExecutor::work() () from emsdk/upstream/bin/../lib/libLLVM-10svn.so
#6  0x00007fb8908469e0 in ?? () from /usr/lib/x86_64-linux-gnu/libstdc++.so.6
#7  0x00007fb8942256db in start_thread (arg=0x7fb6c6906700) at pthread_create.c:463
#8  0x00007fb88ff0388f in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:95

[stale]

This issue has been automatically marked as stale because there has been no activity in the past year. It will be closed automatically if no further activity occurs in the next 30 days. Feel free to re-open at any time if this issue is still relevant.

[davidar]

Relevant: https://github.com/wapm-packages/clang

[HappyCerberus]

Could someone provide a canonical update on this issue? What is the state? Is there interest in working on this feature?

[kripken]

I'm not aware of anyone currently working on this, but I'd be excited to see someone take the time to work on it! And to help with reviews and guidance etc.

Main parts are:

• Porting clang and wasm-ld etc. take some work with the LLVM build system (which is itself self-hosted as it builds tblgen tools) and to avoid fork etc. A few past ports of clang exist, so taking bits from them might help, like https://github.com/kripken/clang-js and https://tbfleming.github.io/cib/ Ideally upstreaming the necessary changes would be best.
• Porting binaryen is trivial given the existing binaryen.js port. Just building with cmake works.
• The JS compiler runs in node, and might need some changes for the browser. Perhaps that could be built on top of WebContainers somehow, but it might be easy enough to just do directly.
• The python scripts could run on a ported Python VM. Pyodide for example has a current port of that.
• Aside from those, the rest would be to move data files around between the ports.

[kripken]

This was done by @jprendes ! 🥳

https://github.com/jprendes/emception
https://www.reddit.com/r/WebAssembly/comments/qn9a5j/i_made_emception_emscripten_running_clientside_in/

As noted in #11175 (comment)

I think we can close this issue now that the goal has been accomplished. If there are specific followups then we can open new issues, either here or in the emception repo.