Skip to content

CMake shared libraries error: shared library dependency not found in library path #24195

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
donalffons opened this issue Apr 28, 2025 · 22 comments

Comments

@donalffons
Copy link

donalffons commented Apr 28, 2025

I am compiling a large CMake code base to Emscripten, where certain libraries must be compiled into dynamically linked libraries. Since Emscripten doesn't support add_library(... SHARED ...) by default, I am trying to use this workaround (another approach would be to use add_executable(...) with -sSIDE_MDOULE=1 - but that would require refactoring of a very large number of CMake files, which I want to avoid).

That approach almost works, except for this: I am building two shared libraries, A and B. A defines that B should be added as a linked library:

# A/CMakeLists.txt
set_property(GLOBAL PROPERTY TARGET_SUPPORTS_SHARED_LIBS TRUE)

add_library(A SHARED ${sources})
set_target_properties(A PROPERTIES SUFFIX ".wasm")
target_link_options(A PRIVATE "-sSIDE_MODULE=1")
target_compile_options(A PRIVATE "-sSIDE_MODULE=1")

set_property(GLOBAL PROPERTY TARGET_SUPPORTS_SHARED_LIBS FALSE)
# B/CMakeLists.txt
set_property(GLOBAL PROPERTY TARGET_SUPPORTS_SHARED_LIBS TRUE)

# ... same as A
target_link_libraries(B A) # Link library A into B

set_property(GLOBAL PROPERTY TARGET_SUPPORTS_SHARED_LIBS FALSE)

With that, the build fails with this error:

em++: error: /project/libB.wasm: shared library dependency not found in library path: `libA.wasm`. (library path: ['/project/runtime/output/wasm32-emscripten/build/vcpkg_installed/wasm32-emscripten/lib', '/project/runtime/output/wasm32-emscripten/build/vcpkg_installed/wasm32-emscripten/debug/lib', '/emsdk/upstream/emscripten/cache/sysroot/lib/wasm32-emscripten/pic']

Is there a way to modify this library path, mentioned in the error message? Has anyone managed to get this to work? Any help would be much appreciated!

@sbc100
Copy link
Collaborator

sbc100 commented Apr 28, 2025

Normally when libB.so depends on libA.so its is passed on the command line either as -L/path/to/A -lA or via its full path /path/to/libA.so. I would expect CMake to do one of these two things. If its just passing libA.wasm and expecting that work, that won't work on any platform that I know of.

Also, in this case you might as well just use .so as your dynamic library extension, like other platforms. It should work fine.

@donalffons
Copy link
Author

donalffons commented Apr 29, 2025

Thank you for your help! I created a minimal reproducer: https://github.com/donalffons/emscripten-cmake-dynamic-libraries

This builds and runs fine with a native build using gcc on linux. It has the following structure:

graph LR
    Main --> B
    B --> A
Loading

When I try to build this with Emscripten, I get this error:

100%] Linking CXX executable Main.js
em++: error: ../B/libB.so: shared library dependency not found in library path: `libA.so`. (library path: ['/home/user/.cache/emscripten/sysroot/lib/wasm32-emscripten/pic', '/usr/lib/emscripten/src/lib']
make[2]: *** [Main/CMakeFiles/Main.dir/build.make:104: Main/Main.js] Error 1
make[1]: *** [CMakeFiles/Makefile2:216: Main/CMakeFiles/Main.dir/all] Error 2
make: *** [Makefile:91: all] Error 2
emmake: error: 'make' failed (returned 2)

Interestingly, the same error appears when I link A into B using an absolute path:

target_link_libraries(B PRIVATE "/home/user/emscripten-cmake-dynamic-libraries/build/A/libA.so")

@sbc100
Copy link
Collaborator

sbc100 commented Apr 29, 2025

Thanks for the simple repro @donalffons.

(BTW, can I recommend that you switch to -G Ninja (its much better normally!) or run make VERBOSE=1 so that the actual command fails is shown. I don't understand why but the default make backend for cmake doesn't show (by default) the command that fails! So annoying for this type of thing. See how in the example about there is an em++ command that fails but you cannot see what the command is because make hid it.)

@sbc100
Copy link
Collaborator

sbc100 commented Apr 29, 2025

CC @hoodmane who is probably the biggest users of emscripten shared libraries.

@hoodmane
Copy link
Collaborator

cc @ryanking13 who usually fixes the cmake problems in Pyodide.

@donalffons
Copy link
Author

donalffons commented Apr 29, 2025

Thank you both!

Here is the output of ninja, including the failing command:

$ ninja
[6/6] Linking CXX executable Main/Main.js
FAILED: Main/Main.js 
: && /usr/lib/emscripten/em++  -sMAIN_MODULE=1 Main/CMakeFiles/Main.dir/src/main.cpp.o -o Main/Main.js  B/libB.so && :
em++: error: B/libB.so: shared library dependency not found in library path: `libA.so`. (library path: ['/home/user/.cache/emscripten/sysroot/lib/wasm32-emscripten/pic', '/usr/lib/emscripten/src/lib']
ninja: build stopped: subcommand failed.

...apparently, it's just adding B/libB.so to the em++ command. libB.so seems to define internally that it needslibA.so:

$ wasm-objdump -s B/libB.so

libB.so:        file format wasm 0x1

Contents of section Custom:
000000a: 0864 796c 696e 6b2e 3001 0400 0000 0002  .dylink.0.......
000001a: 0901 076c 6962 412e 736f                 ...libA.so               # <=== HERE!!!

Contents of section Type:
0000026: 0360 027f 7f01 7f60 0000 6003 7f7f 7f01  .`.....`..`.....
0000036: 7f                                       .

Contents of section Import:
0000039: 0503 656e 7606 6d65 6d6f 7279 0200 0003  ..env.memory....
0000049: 656e 760f 5f5f 7374 6163 6b5f 706f 696e  env.__stack_poin
0000059: 7465 7203 7f01 0365 6e76 0d5f 5f6d 656d  ter....env.__mem
0000069: 6f72 795f 6261 7365 037f 0003 656e 760c  ory_base....env.
0000079: 5f5f 7461 626c 655f 6261 7365 037f 0003  __table_base....
0000089: 656e 7611 5f5a 3131 6164 645f 6e75 6d62  env._Z11add_numb
0000099: 6572 7369 6900 00                        ersii..

Contents of section Function:
00000a2: 0201 02                                  ...

Contents of section Export:
00000a7: 0211 5f5f 7761 736d 5f63 616c 6c5f 6374  ..__wasm_call_ct
00000b7: 6f72 7300 0117 5f5a 3136 6d75 6c74 6970  ors..._Z16multip
00000c7: 6c79 5f61 6e64 5f61 6464 6969 6900 02    ly_and_addiii..

Contents of section Code:
00000d9: 0202 000b 8001 010b 7f23 8080 8080 0021  .........#.....!
00000e9: 0341 1021 0420 0320 046b 2105 2005 2480  .A.!. . .k!. .$.
00000f9: 8080 8000 2005 2000 3602 0c20 0520 0136  .... . .6.. . .6
0000109: 0208 2005 2002 3602 0420 0528 020c 2106  .. . .6.. .(..!.
0000119: 2005 2802 0821 0720 0620 076c 2108 2005   .(..!. . .l!. .
0000129: 2008 3602 0020 0528 0200 2109 2005 2802   .6.. .(..!. .(.
0000139: 0421 0a20 0920 0a10 8080 8080 0021 0b41  .!. . .......!.A
0000149: 1021 0c20 0520 0c6a 210d 200d 2480 8080  .!. . .j!. .$...
0000159: 8000 200b 0f0b                           .. ...

Contents of section Custom:
0000162: 0f74 6172 6765 745f 6665 6174 7572 6573  .target_features
0000172: 082b 0b62 756c 6b2d 6d65 6d6f 7279 2b0f  .+.bulk-memory+.
0000182: 6275 6c6b 2d6d 656d 6f72 792d 6f70 742b  bulk-memory-opt+
0000192: 1663 616c 6c2d 696e 6469 7265 6374 2d6f  .call-indirect-o
00001a2: 7665 726c 6f6e 672b 0a6d 756c 7469 7661  verlong+.multiva
00001b2: 6c75 652b 0f6d 7574 6162 6c65 2d67 6c6f  lue+.mutable-glo
00001c2: 6261 6c73 2b13 6e6f 6e74 7261 7070 696e  bals+.nontrappin
00001d2: 672d 6670 746f 696e 742b 0f72 6566 6572  g-fptoint+.refer
00001e2: 656e 6365 2d74 7970 6573 2b08 7369 676e  ence-types+.sign
00001f2: 2d65 7874                                -ext

But it doesn't contain the path to that library (even when I'm using absolute paths in target_link_libraries) and I wasn't able to find a way to customize that path.

Also, when I'm running a native build, this libA.so information doesn't seem to be included (EDIT: incorrect, it is also included and marked with a comment below):

$ objdump -s B/libB.so


B/libB.so:     file format elf64-x86-64

Contents of section .note.gnu.property:
 02a8 04000000 20000000 05000000 474e5500  .... .......GNU.
 02b8 010001c0 04000000 01000000 00000000  ................
 02c8 020001c0 04000000 01000000 00000000  ................
Contents of section .note.gnu.build-id:
 02d8 04000000 14000000 03000000 474e5500  ............GNU.
 02e8 0552624e 0ce35c27 e99a320b 914efe18  .RbN..\'..2..N..
 02f8 6c8e42f4                             l.B.            
Contents of section .gnu.hash:
 0300 02000000 06000000 01000000 06000000  ................
 0310 00000000 00010004 06000000 00000000  ................
 0320 3bca2a55                             ;.*U            
Contents of section .dynsym:
 0328 00000000 00000000 00000000 00000000  ................
 0338 00000000 00000000 46000000 22000000  ........F..."...
 0348 00000000 00000000 00000000 00000000  ................
 0358 6d000000 12000000 00000000 00000000  m...............
 0368 00000000 00000000 10000000 20000000  ............ ...
 0378 00000000 00000000 00000000 00000000  ................
 0388 01000000 20000000 00000000 00000000  .... ...........
 0398 00000000 00000000 2c000000 20000000  ........,... ...
 03a8 00000000 00000000 00000000 00000000  ................
 03b8 55000000 12000c00 09110000 00000000  U...............
 03c8 2c000000 00000000                    ,.......        
Contents of section .dynstr:
 03d0 005f5f67 6d6f6e5f 73746172 745f5f00  .__gmon_start__.
 03e0 5f49544d 5f646572 65676973 74657254  _ITM_deregisterT
 03f0 4d436c6f 6e655461 626c6500 5f49544d  MCloneTable._ITM
 0400 5f726567 69737465 72544d43 6c6f6e65  _registerTMClone
 0410 5461626c 65005f5f 6378615f 66696e61  Table.__cxa_fina
 0420 6c697a65 005f5a31 366d756c 7469706c  lize._Z16multipl
 0430 795f616e 645f6164 64696969 005f5a31  y_and_addiii._Z1
 0440 31616464 5f6e756d 62657273 6969006c  1add_numbersii.l
 0450 6962412e 736f006c 69627374 64632b2b  ibA.so.libstdc++      # <=== HERE!!!
 0460 2e736f2e 36006c69 626d2e73 6f2e3600  .so.6.libm.so.6.
 0470 6c696267 63635f73 2e736f2e 31006c69  libgcc_s.so.1.li
 0480 62632e73 6f2e3600 6c696242 2e736f00  bc.so.6.libB.so.
 0490 474c4942 435f322e 322e3500 2f686f6d  GLIBC_2.2.5./hom
 04a0 652f7365 62617374 69616e2f 446f776e  e/sebastian/Down
 04b0 6c6f6164 732f656d 73637269 7074656e  loads/emscripten
 04c0 2d636d61 6b652d64 796e616d 69632d6c  -cmake-dynamic-l
 04d0 69627261 72696573 2f627569 6c642f41  ibraries/build/A
 04e0 00                                   .               
Contents of section .gnu.version:
 04e2 00000200 01000100 01000100 0100      ..............  
Contents of section .gnu.version_r:
 04f0 01000100 ae000000 10000000 00000000  ................
 0500 751a6909 00000200 c0000000 00000000  u.i.............
Contents of section .rela.dyn:
 0510 983d0000 00000000 08000000 00000000  .=..............
 0520 00110000 00000000 a03d0000 00000000  .........=......
 0530 08000000 00000000 b0100000 00000000  ................
 0540 08400000 00000000 08000000 00000000  .@..............
 0550 08400000 00000000 c83f0000 00000000  .@.......?......
 0560 06000000 01000000 00000000 00000000  ................
 0570 d03f0000 00000000 06000000 03000000  .?..............
 0580 00000000 00000000 d83f0000 00000000  .........?......
 0590 06000000 04000000 00000000 00000000  ................
 05a0 e03f0000 00000000 06000000 05000000  .?..............
 05b0 00000000 00000000                    ........        
Contents of section .rela.plt:
 05b8 00400000 00000000 07000000 02000000  .@..............
 05c8 00000000 00000000                    ........        
Contents of section .init:
 1000 f30f1efa 4883ec08 488b05c9 2f000048  ....H...H.../..H
 1010 85c07402 ffd04883 c408c3             ..t...H....     
Contents of section .plt:
 1020 ff35ca2f 0000ff25 cc2f0000 0f1f4000  .5./...%./....@.
 1030 ff25ca2f 00006800 000000e9 e0ffffff  .%./..h.........
Contents of section .text:
 1040 488d3dc9 2f000048 8d05c22f 00004839  H.=./..H.../..H9
 1050 f8741548 8b05762f 00004885 c07409ff  .t.H..v/..H..t..
 1060 e00f1f80 00000000 c30f1f80 00000000  ................
 1070 488d3d99 2f000048 8d35922f 00004829  H.=./..H.5./..H)
 1080 fe4889f0 48c1ee3f 48c1f803 4801c648  .H..H..?H...H..H
 1090 d1fe7414 488b0545 2f000048 85c07408  ..t.H..E/..H..t.
 10a0 ffe0660f 1f440000 c30f1f80 00000000  ..f..D..........
 10b0 f30f1efa 803d552f 00000075 33554883  .....=U/...u3UH.
 10c0 3d022f00 00004889 e5740d48 8b3d362f  =./...H..t.H.=6/
 10d0 0000ff15 f02e0000 e863ffff ffc6052c  .........c.....,
 10e0 2f000001 5dc3662e 0f1f8400 00000000  /...].f.........
 10f0 c366662e 0f1f8400 00000000 0f1f4000  .ff...........@.
 1100 f30f1efa e967ffff ff554889 e54883ec  .....g...UH..H..
 1110 20897dec 8975e889 55e48b45 ec0faf45   .}..u..U..E...E
 1120 e88945fc 8b55e48b 45fc89d6 89c7e8fd  ..E..U..E.......
 1130 feffffc9 c3                          .....           
Contents of section .fini:
 1138 f30f1efa 4883ec08 4883c408 c3        ....H...H....   
Contents of section .eh_frame_hdr:
 2000 011b033b 1c000000 02000000 20f0ffff  ...;........ ...
 2010 38000000 09f1ffff 60000000           8.......`...    
Contents of section .eh_frame:
 2020 14000000 00000000 017a5200 01781001  .........zR..x..
 2030 1b0c0708 90010000 24000000 1c000000  ........$.......
 2040 e0efffff 20000000 000e1046 0e184a0f  .... ......F..J.
 2050 0b770880 003f1a3b 2a332422 00000000  .w...?.;*3$"....
 2060 1c000000 44000000 a1f0ffff 2c000000  ....D.......,...
 2070 00410e10 8602430d 06670c07 08000000  .A....C..g......
 2080 00000000                             ....            
Contents of section .init_array:
 3d98 00110000 00000000                    ........        
Contents of section .fini_array:
 3da0 b0100000 00000000                    ........        
Contents of section .dynamic:
 3da8 01000000 00000000 7f000000 00000000  ................
 3db8 01000000 00000000 87000000 00000000  ................
 3dc8 01000000 00000000 96000000 00000000  ................
 3dd8 01000000 00000000 a0000000 00000000  ................
 3de8 01000000 00000000 ae000000 00000000  ................
 3df8 0e000000 00000000 b8000000 00000000  ................
 3e08 1d000000 00000000 cc000000 00000000  ................
 3e18 0c000000 00000000 00100000 00000000  ................
 3e28 0d000000 00000000 38110000 00000000  ........8.......
 3e38 19000000 00000000 983d0000 00000000  .........=......
 3e48 1b000000 00000000 08000000 00000000  ................
 3e58 1a000000 00000000 a03d0000 00000000  .........=......
 3e68 1c000000 00000000 08000000 00000000  ................
 3e78 f5feff6f 00000000 00030000 00000000  ...o............
 3e88 05000000 00000000 d0030000 00000000  ................
 3e98 06000000 00000000 28030000 00000000  ........(.......
 3ea8 0a000000 00000000 11010000 00000000  ................
 3eb8 0b000000 00000000 18000000 00000000  ................
 3ec8 03000000 00000000 e83f0000 00000000  .........?......
 3ed8 02000000 00000000 18000000 00000000  ................
 3ee8 14000000 00000000 07000000 00000000  ................
 3ef8 17000000 00000000 b8050000 00000000  ................
 3f08 07000000 00000000 10050000 00000000  ................
 3f18 08000000 00000000 a8000000 00000000  ................
 3f28 09000000 00000000 18000000 00000000  ................
 3f38 feffff6f 00000000 f0040000 00000000  ...o............
 3f48 ffffff6f 00000000 01000000 00000000  ...o............
 3f58 f0ffff6f 00000000 e2040000 00000000  ...o............
 3f68 f9ffff6f 00000000 03000000 00000000  ...o............
 3f78 00000000 00000000 00000000 00000000  ................
 3f88 00000000 00000000 00000000 00000000  ................
 3f98 00000000 00000000 00000000 00000000  ................
 3fa8 00000000 00000000 00000000 00000000  ................
 3fb8 00000000 00000000 00000000 00000000  ................
Contents of section .got:
 3fc8 00000000 00000000 00000000 00000000  ................
 3fd8 00000000 00000000 00000000 00000000  ................
Contents of section .got.plt:
 3fe8 a83d0000 00000000 00000000 00000000  .=..............
 3ff8 00000000 00000000 36100000 00000000  ........6.......
Contents of section .data:
 4008 08400000 00000000                    .@......        
Contents of section .comment:
 0000 4743433a 2028474e 55292031 342e322e  GCC: (GNU) 14.2.
 0010 31203230 32353032 303700             1 20250207.     

@sbc100
Copy link
Collaborator

sbc100 commented Apr 29, 2025

If libB.so really depend on libA.so that its correct that libA.so should be mentiond in the DT_NEEDED section of libB.so.

Can you share the link command for libB.so? Can you figure out why the ELF build doesn't include libA.so in libB.so DT_NEEDED section. If you pass libA.so when linking libB.so the linker (both ELF and wasm) is supposed to include it as a dependency (unless you use -Wl,--as-needed and happen not to actually use anything from libA.so.

@donalffons
Copy link
Author

donalffons commented Apr 29, 2025

Here is the verbose output of ninja for an Emscripten build:

$ ninja -v
ninja -v    
[1/6] /usr/lib/emscripten/em++ -DA_EXPORTS  -std=gnu++11 -fPIC -sSIDE_MODULE=1 -MD -MT A/CMakeFiles/A.dir/src/a.cpp.o -MF A/CMakeFiles/A.dir/src/a.cpp.o.d -o A/CMakeFiles/A.dir/src/a.cpp.o -c /home/sebastian/Downloads/emscripten-cmake-dynamic-libraries/A/src/a.cpp
[2/6] /usr/lib/emscripten/em++ -DB_EXPORTS -I/home/sebastian/Downloads/emscripten-cmake-dynamic-libraries/A/src -std=gnu++11 -fPIC -sSIDE_MODULE=1 -MD -MT B/CMakeFiles/B.dir/src/b.cpp.o -MF B/CMakeFiles/B.dir/src/b.cpp.o.d -o B/CMakeFiles/B.dir/src/b.cpp.o -c /home/sebastian/Downloads/emscripten-cmake-dynamic-libraries/B/src/b.cpp
[3/6] : && /usr/lib/emscripten/em++ -fPIC   -sSIDE_MODULE=1 -shared -Wl,-soname,libA.so -o A/libA.so A/CMakeFiles/A.dir/src/a.cpp.o   && :
[4/6] : && /usr/lib/emscripten/em++ -fPIC   -sSIDE_MODULE=1 -shared -Wl,-soname,libB.so -o B/libB.so B/CMakeFiles/B.dir/src/b.cpp.o  A/libA.so && :
[5/6] /usr/lib/emscripten/em++  -I/home/sebastian/Downloads/emscripten-cmake-dynamic-libraries/B/src -std=gnu++11 -sMAIN_MODULE=1 -MD -MT Main/CMakeFiles/Main.dir/src/main.cpp.o -MF Main/CMakeFiles/Main.dir/src/main.cpp.o.d -o Main/CMakeFiles/Main.dir/src/main.cpp.o -c /home/sebastian/Downloads/emscripten-cmake-dynamic-libraries/Main/src/main.cpp
[6/6] : && /usr/lib/emscripten/em++  -sMAIN_MODULE=1 Main/CMakeFiles/Main.dir/src/main.cpp.o -o Main/Main.js  B/libB.so && :
FAILED: Main/Main.js 
: && /usr/lib/emscripten/em++  -sMAIN_MODULE=1 Main/CMakeFiles/Main.dir/src/main.cpp.o -o Main/Main.js  B/libB.so && :
em++: error: B/libB.so: shared library dependency not found in library path: `libA.so`. (library path: ['/home/sebastian/.cache/emscripten/sysroot/lib/wasm32-emscripten/pic', '/usr/lib/emscripten/src/lib']
ninja: build stopped: subcommand failed.

EDIT: And here it is for a native build:

$ ninja -v
[1/6] /usr/bin/c++ -DB_EXPORTS -I/home/sebastian/Downloads/emscripten-cmake-dynamic-libraries/A/src -std=gnu++11 -fPIC -MD -MT B/CMakeFiles/B.dir/src/b.cpp.o -MF B/CMakeFiles/B.dir/src/b.cpp.o.d -o B/CMakeFiles/B.dir/src/b.cpp.o -c /home/sebastian/Downloads/emscripten-cmake-dynamic-libraries/B/src/b.cpp
[2/6] /usr/bin/c++ -DA_EXPORTS  -std=gnu++11 -fPIC -MD -MT A/CMakeFiles/A.dir/src/a.cpp.o -MF A/CMakeFiles/A.dir/src/a.cpp.o.d -o A/CMakeFiles/A.dir/src/a.cpp.o -c /home/sebastian/Downloads/emscripten-cmake-dynamic-libraries/A/src/a.cpp
[3/6] : && /usr/bin/c++ -fPIC   -Wl,--dependency-file=A/CMakeFiles/A.dir/link.d -shared -Wl,-soname,libA.so -o A/libA.so A/CMakeFiles/A.dir/src/a.cpp.o   && :
[4/6] : && /usr/bin/c++ -fPIC   -Wl,--dependency-file=B/CMakeFiles/B.dir/link.d -shared -Wl,-soname,libB.so -o B/libB.so B/CMakeFiles/B.dir/src/b.cpp.o -L/home/sebastian/Downloads/emscripten-cmake-dynamic-libraries/build/A -Wl,-rpath,/home/sebastian/Downloads/emscripten-cmake-dynamic-libraries/build/A  -lA && :
[5/6] /usr/bin/c++  -I/home/sebastian/Downloads/emscripten-cmake-dynamic-libraries/B/src -std=gnu++11 -MD -MT Main/CMakeFiles/Main.dir/src/main.cpp.o -MF Main/CMakeFiles/Main.dir/src/main.cpp.o.d -o Main/CMakeFiles/Main.dir/src/main.cpp.o -c /home/sebastian/Downloads/emscripten-cmake-dynamic-libraries/Main/src/main.cpp
[6/6] : && /usr/bin/c++  -Wl,--dependency-file=Main/CMakeFiles/Main.dir/link.d Main/CMakeFiles/Main.dir/src/main.cpp.o -o Main/Main  -Wl,-rpath,/home/sebastian/Downloads/emscripten-cmake-dynamic-libraries/build/B  B/libB.so && :

@donalffons
Copy link
Author

donalffons commented Apr 29, 2025

...just realized that libA.so is included in the objdump output - it was just line-wrapped, which is why I missed it. I updated the comment above.

@donalffons
Copy link
Author

donalffons commented Apr 29, 2025

...but, the objdump output of the ELF build also includes the path to libA.so, just a few lines below my <=== HERE!!! comment. That's missing in the Emscripten build.

@sbc100
Copy link
Collaborator

sbc100 commented Apr 29, 2025

What does readelf -d libB.so show?

@donalffons
Copy link
Author

$ readelf -d libB.so

Dynamic section at offset 0x2da8 contains 30 entries:
  Tag        Type                         Name/Value
 0x0000000000000001 (NEEDED)             Shared library: [libA.so]
 0x0000000000000001 (NEEDED)             Shared library: [libstdc++.so.6]
 0x0000000000000001 (NEEDED)             Shared library: [libm.so.6]
 0x0000000000000001 (NEEDED)             Shared library: [libgcc_s.so.1]
 0x0000000000000001 (NEEDED)             Shared library: [libc.so.6]
 0x000000000000000e (SONAME)             Library soname: [libB.so]
 0x000000000000001d (RUNPATH)            Library runpath: [/home/sebastian/Downloads/emscripten-cmake-dynamic-libraries/build/A]
 0x000000000000000c (INIT)               0x1000
 0x000000000000000d (FINI)               0x1138
 0x0000000000000019 (INIT_ARRAY)         0x3d98
 0x000000000000001b (INIT_ARRAYSZ)       8 (bytes)
 0x000000000000001a (FINI_ARRAY)         0x3da0
 0x000000000000001c (FINI_ARRAYSZ)       8 (bytes)
 0x000000006ffffef5 (GNU_HASH)           0x300
 0x0000000000000005 (STRTAB)             0x3d0
 0x0000000000000006 (SYMTAB)             0x328
 0x000000000000000a (STRSZ)              273 (bytes)
 0x000000000000000b (SYMENT)             24 (bytes)
 0x0000000000000003 (PLTGOT)             0x3fe8
 0x0000000000000002 (PLTRELSZ)           24 (bytes)
 0x0000000000000014 (PLTREL)             RELA
 0x0000000000000017 (JMPREL)             0x5b8
 0x0000000000000007 (RELA)               0x510
 0x0000000000000008 (RELASZ)             168 (bytes)
 0x0000000000000009 (RELAENT)            24 (bytes)
 0x000000006ffffffe (VERNEED)            0x4f0
 0x000000006fffffff (VERNEEDNUM)         1
 0x000000006ffffff0 (VERSYM)             0x4e2
 0x000000006ffffff9 (RELACOUNT)          3
 0x0000000000000000 (NULL)               0x0

@sbc100
Copy link
Collaborator

sbc100 commented Apr 29, 2025

So the RUNPATH here is key. When you link libB.so do you see a -Wl,-rpath (or something similar)? Do you not see the same thing when linking libB.so with emscripten?

@sbc100
Copy link
Collaborator

sbc100 commented Apr 29, 2025

RPATH support is very new in emscripten BTW.. the runtime support was only added in #23872 which has yet to make it into a release.

However the llvm support should be in 4.0.5 and above (see #23805)

@donalffons
Copy link
Author

donalffons commented Apr 29, 2025

When you link libB.so do you see a -Wl,-rpath (or something similar)?

When building with Emscripten, the string -Wl,-rpath doesn't show up, but when building via ELF, it shows up in my /build/build.ninja

...
#############################################
# Link the executable Main/Main

build Main/Main: CXX_EXECUTABLE_LINKER__Main_ Main/CMakeFiles/Main.dir/src/main.cpp.o | B/libB.so || B/libB.so
  DEP_FILE = Main/CMakeFiles/Main.dir/link.d
  LINK_FLAGS = -Wl,--dependency-file=Main/CMakeFiles/Main.dir/link.d
  LINK_LIBRARIES = -Wl,-rpath,/home/sebastian/Downloads/emscripten-cmake-dynamic-libraries/build/B  B/libB.so
  OBJECT_DIR = Main/CMakeFiles/Main.dir
  POST_BUILD = :
  PRE_LINK = :
  TARGET_COMPILE_PDB = Main/CMakeFiles/Main.dir/
  TARGET_FILE = Main/Main
  TARGET_PDB = Main/Main.pdb
...

#############################################
# Link the shared library B/libB.so

build B/libB.so: CXX_SHARED_LIBRARY_LINKER__B_ B/CMakeFiles/B.dir/src/b.cpp.o | A/libA.so
  DEP_FILE = B/CMakeFiles/B.dir/link.d
  LINK_FLAGS = -Wl,--dependency-file=B/CMakeFiles/B.dir/link.d
  LINK_LIBRARIES = -Wl,-rpath,/home/sebastian/Downloads/emscripten-cmake-dynamic-libraries/build/A  -lA
  LINK_PATH = -L/home/sebastian/Downloads/emscripten-cmake-dynamic-libraries/build/A
  OBJECT_DIR = B/CMakeFiles/B.dir
  POST_BUILD = :
  PRE_LINK = :
  SONAME = libB.so
  SONAME_FLAG = -Wl,-soname,
  TARGET_COMPILE_PDB = B/CMakeFiles/B.dir/
  TARGET_FILE = B/libB.so
  TARGET_PDB = B/libB.pdb

I'm currently on Emscripten v4.0.4. Are you suggesting that this might have already been resolved on the current main branch? I just tried using ./emsdk install tot && ./emsdk activate tot, where em++ --version says 4.0.8-git (9b715a9dc26325aaa9c72c138c1dd56f56f15c02). When using this to build my repro, the error seems to be unchanged...

@sbc100
Copy link
Collaborator

sbc100 commented Apr 29, 2025

We need to figure out why cmake is not inserting the -rpath flag when targeting emscripten.

@ryanking13
Copy link
Contributor

Is it actually related to RPATH? It looks like the problem is happening in the linking stage not in runtime.

@ryanking13
Copy link
Contributor

ryanking13 commented Apr 30, 2025

cc @ryanking13 who usually fixes the cmake problems in Pyodide.

FWIW, what Pyodide does to tackle with shared libraries + CMake is to write a separate Cmake toolchain file which enables shared modules and adds side module flags, and passing it when calling emcmake (1, 2) (using CMAKE_TOOLCHAIN_FILE env variable).

@sbc100
Copy link
Collaborator

sbc100 commented Apr 30, 2025

I yes, it looks like its actually simply the link time path -L that is missing.

It looks like the part that should be enough to make it work here is LINK_PATH = -L/home/sebastian/Downloads/emscripten-cmake-dynamic-libraries/build/A. Do you not see that line when linking emscripten?

@donalffons
Copy link
Author

donalffons commented May 2, 2025

Here is the relevant section of the build.ninja when using emcmake cmake -G Ninja ..:

#############################################
# Link the shared library B/libB.so

build B/libB.so: CXX_SHARED_LIBRARY_LINKER__B_ B/CMakeFiles/B.dir/src/b.cpp.o | A/libA.so || A/libA.so
  LINK_FLAGS = -sSIDE_MODULE=1
  LINK_LIBRARIES = A/libA.so
  OBJECT_DIR = B/CMakeFiles/B.dir
  POST_BUILD = :
  PRE_LINK = :
  SONAME = libB.so
  SONAME_FLAG = -Wl,-soname,
  TARGET_FILE = B/libB.so
  TARGET_PDB = B.so.dbg

# ...

#############################################
# Link the executable Main/Main.js

build Main/Main.js: CXX_EXECUTABLE_LINKER__Main_ Main/CMakeFiles/Main.dir/src/main.cpp.o | B/libB.so || B/libB.so
  LINK_FLAGS = -sMAIN_MODULE=1
  LINK_LIBRARIES = B/libB.so
  OBJECT_DIR = Main/CMakeFiles/Main.dir
  POST_BUILD = :
  PRE_LINK = :
  TARGET_FILE = Main/Main.js
  TARGET_PDB = Main.js.dbg

and here when using cmake -G Ninja ..:

#############################################
# Link the shared library B/libB.so

build B/libB.so: CXX_SHARED_LIBRARY_LINKER__B_ B/CMakeFiles/B.dir/src/b.cpp.o | A/libA.so || A/libA.so
  DEP_FILE = B/CMakeFiles/B.dir/link.d
  LINK_FLAGS = -Wl,--dependency-file=B/CMakeFiles/B.dir/link.d
  LINK_LIBRARIES = -Wl,-rpath,/home/sebastian/Downloads/emscripten-cmake-dynamic-libraries/build/A  A/libA.so
  OBJECT_DIR = B/CMakeFiles/B.dir
  POST_BUILD = :
  PRE_LINK = :
  SONAME = libB.so
  SONAME_FLAG = -Wl,-soname,
  TARGET_COMPILE_PDB = B/CMakeFiles/B.dir/
  TARGET_FILE = B/libB.so
  TARGET_PDB = B/libB.pdb

# ...

#############################################
# Link the executable Main/Main

build Main/Main: CXX_EXECUTABLE_LINKER__Main_ Main/CMakeFiles/Main.dir/src/main.cpp.o | B/libB.so || B/libB.so
  DEP_FILE = Main/CMakeFiles/Main.dir/link.d
  LINK_FLAGS = -Wl,--dependency-file=Main/CMakeFiles/Main.dir/link.d
  LINK_LIBRARIES = -Wl,-rpath,/home/sebastian/Downloads/emscripten-cmake-dynamic-libraries/build/B:/home/sebastian/Downloads/emscripten-cmake-dynamic-libraries/build/A  B/libB.so  -Wl,-rpath-link,/home/sebastian/Downloads/emscripten-cmake-dynamic-libraries/build/A
  OBJECT_DIR = Main/CMakeFiles/Main.dir
  POST_BUILD = :
  PRE_LINK = :
  TARGET_COMPILE_PDB = Main/CMakeFiles/Main.dir/
  TARGET_FILE = Main/Main
  TARGET_PDB = Main/Main.pdb

I tried manually editing the build.ninja for the Emscripten case and replaced the LINK_LIBRARIES=... lines with the ones from the ELF version. That leads to this error:

$ ninja   
[1/1] Linking CXX executable Main/Main.js
FAILED: Main/Main.js 
: && /usr/lib/emscripten/em++  -sMAIN_MODULE=1 Main/CMakeFiles/Main.dir/src/main.cpp.o -o Main/Main.js  -Wl,-rpath,/home/sebastian/Downloads/emscripten-cmake-dynamic-libraries/build/B:/home/sebastian/Downloads/emscripten-cmake-dynamic-libraries/build/A  B/libB.so  -Wl,-rpath-link,/home/sebastian/Downloads/emscripten-cmake-dynamic-libraries/build/A && :
em++: warning: ignoring unsupported linker flag: `-rpath` [-Wlinkflags]
em++: warning: ignoring unsupported linker flag: `-rpath-link` [-Wlinkflags]
em++: error: B/libB.so: shared library dependency not found in library path: `libA.so`. (library path: ['/home/sebastian/.cache/emscripten/sysroot/lib/wasm32-emscripten/pic', '/usr/lib/emscripten/src/lib']
ninja: build stopped: subcommand failed.

(A/libA.so is built, B/libB.so and Main/Main.js are not)

EDIT: And there is no mention of a string like LINK_PATH=... - neither in the Emscripten nor in the ELF version, only a few mentions of a variable $LINK_PATH in rules.ninja.

@sbc100
Copy link
Collaborator

sbc100 commented May 2, 2025

If you want -rpath to work you will need the very latest version of emscripten which was released just yesterday (4.0.8)

@donalffons
Copy link
Author

$ emcc --version
emcc (Emscripten gcc/clang-like replacement + linker emulating GNU ld) 4.0.8 (70404efec4458b60b953bc8f1529f2fa112cdfd1)

After the modifications to the build.ninja mentioned above, the build fails at

$ ninja
[1/1] Linking CXX executable Main/Main.js
FAILED: Main/Main.js 
: && /home/sebastian/Downloads/emscripten-cmake-dynamic-libraries/emsdk/upstream/emscripten/em++  -sMAIN_MODULE=1 Main/CMakeFiles/Main.dir/src/main.cpp.o -o Main/Main.js  -Wl,-rpath,/home/sebastian/Downloads/emscripten-cmake-dynamic-libraries/build/B:/home/sebastian/Downloads/emscripten-cmake-dynamic-libraries/build/A  B/libB.so && :
em++: error: B/libB.so: shared library dependency not found in library path: `libA.so`. (library path: ['/home/sebastian/Downloads/emscripten-cmake-dynamic-libraries/emsdk/upstream/emscripten/cache/sysroot/lib/wasm32-emscripten/pic', '/home/sebastian/Downloads/emscripten-cmake-dynamic-libraries/emsdk/upstream/emscripten/src/lib']
ninja: build stopped: subcommand failed.

...strangely, wasm-objdump for B/libB.so now contains the full path to A/libA.so.

$ wasm-objdump -s B/libB.so

libB.so:        file format wasm 0x1

Contents of section Custom:
000000a: 0864 796c 696e 6b2e 3001 0400 0000 0002  .dylink.0.......
000001a: 0901 076c 6962 412e 736f 0546 0144 2f68  ...libA.so.F.D/h     # <=== HERE!!!
000002a: 6f6d 652f 7365 6261 7374 6961 6e2f 446f  ome/sebastian/Do
000003a: 776e 6c6f 6164 732f 656d 7363 7269 7074  wnloads/emscript
000004a: 656e 2d63 6d61 6b65 2d64 796e 616d 6963  en-cmake-dynamic
000005a: 2d6c 6962 7261 7269 6573 2f62 7569 6c64  -libraries/build
000006a: 2f41                                     /A

Contents of section Type:
000006e: 0360 027f 7f01 7f60 0000 6003 7f7f 7f01  .`.....`..`.....
000007e: 7f                                       .

Contents of section Import:
0000081: 0503 656e 7606 6d65 6d6f 7279 0200 0003  ..env.memory....
0000091: 656e 760f 5f5f 7374 6163 6b5f 706f 696e  env.__stack_poin
00000a1: 7465 7203 7f01 0365 6e76 0d5f 5f6d 656d  ter....env.__mem
00000b1: 6f72 795f 6261 7365 037f 0003 656e 760c  ory_base....env.
00000c1: 5f5f 7461 626c 655f 6261 7365 037f 0003  __table_base....
00000d1: 656e 7611 5f5a 3131 6164 645f 6e75 6d62  env._Z11add_numb
00000e1: 6572 7369 6900 00                        ersii..

Contents of section Function:
00000ea: 0201 02                                  ...

Contents of section Export:
00000ef: 0211 5f5f 7761 736d 5f63 616c 6c5f 6374  ..__wasm_call_ct
00000ff: 6f72 7300 0117 5f5a 3136 6d75 6c74 6970  ors..._Z16multip
000010f: 6c79 5f61 6e64 5f61 6464 6969 6900 02    ly_and_addiii..

Contents of section Code:
0000121: 0202 000b 8001 010b 7f23 8080 8080 0021  .........#.....!
0000131: 0341 1021 0420 0320 046b 2105 2005 2480  .A.!. . .k!. .$.
0000141: 8080 8000 2005 2000 3602 0c20 0520 0136  .... . .6.. . .6
0000151: 0208 2005 2002 3602 0420 0528 020c 2106  .. . .6.. .(..!.
0000161: 2005 2802 0821 0720 0620 076c 2108 2005   .(..!. . .l!. .
0000171: 2008 3602 0020 0528 0200 2109 2005 2802   .6.. .(..!. .(.
0000181: 0421 0a20 0920 0a10 8080 8080 0021 0b41  .!. . .......!.A
0000191: 1021 0c20 0520 0c6a 210d 200d 2480 8080  .!. . .j!. .$...
00001a1: 8000 200b 0f0b                           .. ...

Contents of section Custom:
00001aa: 0f74 6172 6765 745f 6665 6174 7572 6573  .target_features
00001ba: 082b 0b62 756c 6b2d 6d65 6d6f 7279 2b0f  .+.bulk-memory+.
00001ca: 6275 6c6b 2d6d 656d 6f72 792d 6f70 742b  bulk-memory-opt+
00001da: 1663 616c 6c2d 696e 6469 7265 6374 2d6f  .call-indirect-o
00001ea: 7665 726c 6f6e 672b 0a6d 756c 7469 7661  verlong+.multiva
00001fa: 6c75 652b 0f6d 7574 6162 6c65 2d67 6c6f  lue+.mutable-glo
000020a: 6261 6c73 2b13 6e6f 6e74 7261 7070 696e  bals+.nontrappin
000021a: 672d 6670 746f 696e 742b 0f72 6566 6572  g-fptoint+.refer
000022a: 656e 6365 2d74 7970 6573 2b08 7369 676e  ence-types+.sign
000023a: 2d65 7874                                -ext

Thank you all for your help until here - this is much appreciated. This thread gave me a lot of new clues that I'll look into, next week. In case you have other ideas, please let me know 🙂.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

4 participants