[SR-2280] swiftc -static-stdlib option fails on Ubuntu 14.04 & 15.10

[abertelrud]

Previous ID	SR-2280
Radar	None
Original Reporter	@abertelrud
Type	Bug
Status	Resolved
Resolution	Done

Environment

Ubuntu 14.04, Ubuntu 15.10

Additional Detail from JIRA

Votes	4
Component/s	Compiler
Labels	Bug
Assignee	@spevans
Priority	Medium

md5: 12b3c71761ce9762d3e3755bd28b44eb

blocks:

• SR-648 swift package manager should have the option to produce statically linked binaries

relates to:

• SR-648 swift package manager should have the option to produce statically linked binaries
• SR-730 Flag to statically link Swift standard library
• SR-1520 Create Static Libraries with Swift

Issue Description:

The `-static-stdlib` option fails on Ubuntu 14.04 and on 15.10 (on macOS it works). This is using the latest nightly toolchain (`LLVM 3a878c6fdb, Clang cb02d1caed, Swift 8436ef8`).

To reproduce:

echo 'print("hello")' > main.swift
swiftc -static-stdlib main.swift -o main

The result is this:

<unknown>:0: error: link command failed with exit code 1 (use -v to see invocation)
user@ubuntu:/var/tmp/Static$ ^C
user@ubuntu:/var/tmp/Static$ swiftc -static-stdlib main.swift
/usr/bin/ld: cannot find -lbsd
/usr/bin/ld: cannot find -licui18n
/usr/bin/ld: cannot find -licuuc
clang: error: linker command failed with exit code 1 (use -v to see invocation)
<unknown>:0: error: link command failed with exit code 1 (use -v to see invocation)

Passing `-v` yields this:

Swift version 3.0-dev (LLVM 3a878c6fdb, Clang cb02d1caed, Swift 8436ef8993)
Target: x86_64-unknown-linux-gnu
/home/user/Documents/usr/bin/swift -frontend -c -primary-file main.swift -target x86_64-unknown-linux-gnu -disable-objc-interop -color-diagnostics -module-name main -o /tmp/main-58b97c.o
/home/user/Documents/usr/bin/swift-autolink-extract /tmp/main-58b97c.o -o /tmp/main-f05ff6.autolink
/usr/bin/clang++ -fuse-ld=gold -target x86_64-unknown-linux-gnu -Xlinker -rpath -Xlinker /home/user/Documents/usr/lib/swift/linux /home/user/Documents/usr/lib/swift/linux/x86_64/swift_begin.o /tmp/main-58b97c.o -L /home/user/Documents/usr/lib/swift_static/linux -ldl -lpthread -lbsd -licui18n -licuuc -Xlinker -export-dynamic -Xlinker --exclude-libs -Xlinker ALL --target=x86_64-unknown-linux-gnu -lswiftCore @/tmp/main-f05ff6.autolink /home/user/Documents/usr/lib/swift/linux/x86_64/swift_end.o -o main
/usr/bin/ld: cannot find -lbsd
/usr/bin/ld: cannot find -licui18n
/usr/bin/ld: cannot find -licuuc
clang: error: linker command failed with exit code 1 (use -v to see invocation)
<unknown>:0: error: link command failed with exit code 1 (use -v to see invocation)

[swift-ci]

Comment by Joel Hughes (JIRA)

This issue appears to be resolved in latest preview, at least for me using:

```
swift --version

1. Swift version 3.0 (swift-3.0-PREVIEW-6)

2. Target: x86_64-unknown-linux-gnu

echo 'print("hello")' > main.swift
swiftc -static-stdlib main.swift -o main
```

[swift-ci]

Comment by Joel Hughes (JIRA)

However, when coping the resulting binary to a basic ubuntu docker container the binary still has unresolved dependencies:

```
./main
#./main: error while loading shared libraries: libicui18n.so.55: cannot open shared object file: No such file or directory
```

[spevans]

Maybe libicu is not installed in your docker container or the path is incorrect? Use ldd to see the full pathnames of the libs e.g.:

$ ldd main
    linux-vdso.so.1 (0x00007ffe74311000)
    libdl.so.2 => /lib/x86_64-linux-gnu/libdl.so.2 (0x00007fd2e2a29000)
    libpthread.so.0 => /lib/x86_64-linux-gnu/libpthread.so.0 (0x00007fd2e280c000)
    libbsd.so.0 => /lib/x86_64-linux-gnu/libbsd.so.0 (0x00007fd2e25fc000)
    libicui18n.so.52 => /usr/lib/x86_64-linux-gnu/libicui18n.so.52 (0x00007fd2e21ea000)
    libicuuc.so.52 => /usr/lib/x86_64-linux-gnu/libicuuc.so.52 (0x00007fd2e1e6c000)
    libstdc++.so.6 => /usr/lib/x86_64-linux-gnu/libstdc++.so.6 (0x00007fd2e1b61000)
    libm.so.6 => /lib/x86_64-linux-gnu/libm.so.6 (0x00007fd2e1860000)
    libgcc_s.so.1 => /lib/x86_64-linux-gnu/libgcc_s.so.1 (0x00007fd2e164a000)
    libc.so.6 => /lib/x86_64-linux-gnu/libc.so.6 (0x00007fd2e129f000)
    /lib64/ld-linux-x86-64.so.2 (0x00007fd2e2c2d000)
    libicudata.so.52 => /usr/lib/x86_64-linux-gnu/libicudata.so.52 (0x00007fd2dfa32000)

[swift-ci]

Comment by Joel Hughes (JIRA)

That's right, so I shouldn't have expected a static bin with no dependency on libicui18n.so?

[spevans]

-static-stdlib only links in libswiftCore.a it doesn't generate fully static binaries. Ive been having a look at this and I was able to statically link in the libicu library and its extra data file.

The problem is that the swift runtime actually uses dlopen()/dlsym() to examine its own binary to find the protocol and metadata information contained in other ELF segments and libdl will complain if it is linked into a static binary

I have made some random notes here: http://si.org/posts/2016/09/swift_static_binary where I get as far as linking in libicu

[spevans]

#5269 should fix this. It tries to find the .a files for ICU and creates a file of linker arguments:

$ cat ~/swift-test/usr/lib/swift_static/linux/static-stdlib-args.lnk
-ldl
-lpthread
-lswiftCore
/usr/lib/x86_64-linux-gnu/libicui18n.a
/usr/lib/x86_64-linux-gnu/libicuuc.a
/usr/lib/x86_64-linux-gnu/libicudata.a
-Xlinker
-export-dynamic
-Xlinker
--exclude-libs
-Xlinker
ALL

$ cat main.swift 
print("Hello world!")
$ ~/swift-test/usr/bin/swiftc   -emit-executable -static-stdlib -O  -o main main.swift
$ ldd main
    linux-vdso.so.1 (0x00007ffd81cbe000)
    libdl.so.2 => /lib/x86_64-linux-gnu/libdl.so.2 (0x00007f599f117000)
    libpthread.so.0 => /lib/x86_64-linux-gnu/libpthread.so.0 (0x00007f599eefa000)
    libstdc++.so.6 => /usr/lib/x86_64-linux-gnu/libstdc++.so.6 (0x00007f599ebef000)
    libm.so.6 => /lib/x86_64-linux-gnu/libm.so.6 (0x00007f599e8ee000)
    libgcc_s.so.1 => /lib/x86_64-linux-gnu/libgcc_s.so.1 (0x00007f599e6d8000)
    libc.so.6 => /lib/x86_64-linux-gnu/libc.so.6 (0x00007f599e32d000)
    /lib64/ld-linux-x86-64.so.2 (0x00007f599f31b000)
$ ./main
Hello world!
$ ls -s --block-size=K main
29605K main

The large filesize is due to the libicudata

[ddunbar]

This is resolved right?

[spevans]

Yes I would say so. On ubuntu, the ICU libraries should be statically linked in now when -static-stdlib is used which I think solves the issue. If a developer doesnt want this behaviour they can always edit the

usr/lib/swift_static/linux/static-stdlib-args.lnk

file and change the libs back to the dynamic ones.