WASI linker error: undefined symbol __multi3

[flavio]

I ran into this linker issue when trying to build some Go code:

$ tinygo build -x \
	-o policy.wasm \
	-target=wasi \
	-no-debug .
clang -mbulk-memory -mnontrapping-fptoint -msign-ext --sysroot=/usr/local/tinygo/lib/wasi-libc/sysroot -gdwarf-4 -Oz --target=wasm32-unknown-wasi -mcpu=generic -MD -MV -MTdeps -MF /tmp/tinygo2136342536/dep-1473137204.d -flto=thin -c -o /home/tinygo/.cache/tinygo/tmp-2233786165.bc /usr/local/tinygo/src/internal/task/task_asyncify_wasm.S -Qunused-arguments
clang -mbulk-memory -mnontrapping-fptoint -msign-ext --sysroot=/usr/local/tinygo/lib/wasi-libc/sysroot -gdwarf-4 -Oz --target=wasm32-unknown-wasi -mcpu=generic -MD -MV -MTdeps -MF /tmp/tinygo2136342536/dep-2756220689.d -flto=thin -c -o /home/tinygo/.cache/tinygo/tmp-1731379903.bc /usr/local/tinygo/src/runtime/asm_tinygowasm.S -Qunused-arguments
wasm-ld --stack-first --no-demangle -L /usr/local/tinygo -o /tmp/tinygo2136342536/main --strip-debug --compress-relocations /tmp/tinygo2136342536/main.o /home/tinygo/.cache/tinygo/obj-0bfb1c88ecd5881542b47db010b5787164fc6a554d252cff6bd66805.bc /home/tinygo/.cache/tinygo/obj-21efc40633d7331a1c71e816bcdcc1a96f69a1f1969201b2c8be7b5f.bc /usr/local/tinygo/lib/wasi-libc/sysroot/lib/wasm32-wasi/libc.a -mllvm -mcpu=generic --lto-O2 --thinlto-cache-dir=/home/tinygo/.cache/tinygo/thinlto -mllvm --rotation-max-header-size=0
tinygo:wasm-ld: error: lto.tmp: undefined symbol: __multi3
failed to run tool: wasm-ld
error: failed to link /tmp/tinygo2136342536/main: exit status 1

This happens with tinygo version 0.27.0, using the official container image. I can reproduce it also with tinygo built from the dev branch.

Root cause

The __multi3 symbol is provided by the libclang_rt.builtins-wasm32.a library, which, from what I understand, is not built by default by wasi-libc.

Quoting the official README:

Note that Clang packages typically don't include cross-compiled builds of compiler-rt, libcxx, or libcxxabi, for libclang_rt.builtins-wasm32.a, libc++.a, or libc++abi.a, respectively, so they may not be usable without extra setup. This is one of the things wasi-sdk simplifies, as it includes cross-compiled builds of compiler-rt, libc++.a, and libc++abi.a.

Each wasi-sdk release includes the pre-built library as one of its release artifacts.

Dirty workaround

Starting from the tinygo official container image, I added the libclang_rt.builtins-wasm32.a library provided by the wasi-sdk-16 release. This is the version of wasi-sdk used by tinygo 0.27.0.

Then I patched the linker section of the wasi.json target to make use of this library:

{
	# relevant section of the file
	"ldflags": [
		"--stack-first",
		"--no-demangle",
		"/usr/local/tinygo/lib/wasi-libc/sysroot/lib/wasm32-wasi/libclang_rt.builtins-wasm32.a"
	],
}

I couldn't find a way to avoid this change. I tried using different CGO environment variables, but none of them was being used.

Reproducer

I created this repository that contains:

• The code causing the linker issue
• The Dockerfile used to build the "patched" tinygo container image
• The patched wasi.json target file
• Instructions about how to use the contents of the repository

[aykevl]

We bundle compiler-rt for TinyGo. Normally it's as simple as this:

--- a/targets/wasi.json
+++ b/targets/wasi.json
@@ -6,6 +6,7 @@
        "goos":          "linux",
        "goarch":        "arm",
        "linker":        "wasm-ld",
+       "rtlib":         "compiler-rt",
        "libc":          "wasi-libc",
        "scheduler":     "asyncify",
        "default-stack-size": 16384,

...but not this time, because wasm-ld needs a symbol table in the archive file and our archive creation code doesn't support that yet. So it needs some additions to builder/ar.go.

[dgryski]

Just hit this myself while testing:

~/go/src/github.com/RoaringBitmap/roaring $ tinygo test -tags="purego" -target=wasi
tinygo:wasm-ld: error: lto.tmp: undefined symbol: __multi3
failed to run tool: wasm-ld
FAIL	github.com/RoaringBitmap/roaring	0.000s
error: failed to link /var/folders/5b/_hr1d1fd3qn4t9vtfx5p61wm0000gp/T/tinygo3620564764/main: exit status 1

[jsimnz]

Also just recently hit this bug. Not much context to be added compared to whats already here. Happened in a currently private repo.

[aykevl]

If someone would like to fix this bug, this is what needs to be done:

1. Add "rtlib": "compiler-rt" to targets/wasi.json.
2. Update builder/ar.go to also read symbol names from WebAssembly files (you can use an existing WebAssembly file parser package for that, like github.com/aykevl/go-wasm).
3. Test using tinygo clean followed by tinygo build or tinygo run.

[deadprogram]

Closing since this is part of release 0.29.0 thanks!