Add github action rules for static binaries generation (Linux, Macos) #391

AltGr · 2021-05-06T15:42:12Z

Note: this PR supersedes #388.

AltGr · 2021-05-06T16:03:02Z

Aforementioned binaries can be downloaded, in the examples of this run, here: https://github.com/ocaml-sf/learn-ocaml/actions/runs/817393844

Testers welcome!

(note that this might be turned on only for master and tags, but I wanted to demonstrate the PR first :) )

erikmd · 2021-05-13T23:09:29Z

Hi @AltGr, thanks a lot for these two PRs 👍

I'll test the generated binaries ASAP − at least the Linux ones − and let you know!

scripts/setup-static-build.sh

To enable: `ln -s static-linking.sexp src/main/linking.sexp` This still links dynamically to glibc and pthreads, but should already considerably improve portability. Next to do is use musl instead of glibc. Before: ``` % ldd _build/default/src/main/learnocaml_server_main.exe linux-vdso.so.1 (0x00007fff743ee000) libssl.so.1.1 => /usr/lib/x86_64-linux-gnu/libssl.so.1.1 (0x00007f29aeb43000) libcrypto.so.1.1 => /usr/lib/x86_64-linux-gnu/libcrypto.so.1.1 (0x00007f29ae84f000) libpthread.so.0 => /lib/x86_64-linux-gnu/libpthread.so.0 (0x00007f29ae82d000) libm.so.6 => /lib/x86_64-linux-gnu/libm.so.6 (0x00007f29ae6e9000) libdl.so.2 => /lib/x86_64-linux-gnu/libdl.so.2 (0x00007f29ae6e3000) libc.so.6 => /lib/x86_64-linux-gnu/libc.so.6 (0x00007f29ae51e000) /lib64/ld-linux-x86-64.so.2 (0x00007f29af80e000) ``` After: ``` % ldd _build/default/src/main/learnocaml_server_main.exe linux-vdso.so.1 (0x00007ffcdddd5000) libpthread.so.0 => /lib/x86_64-linux-gnu/libpthread.so.0 (0x00007f90e8f0e000) libm.so.6 => /lib/x86_64-linux-gnu/libm.so.6 (0x00007f90e8dca000) libdl.so.2 => /lib/x86_64-linux-gnu/libdl.so.2 (0x00007f90e8dc4000) libc.so.6 => /lib/x86_64-linux-gnu/libc.so.6 (0x00007f90e8bff000) /lib64/ld-linux-x86-64.so.2 (0x00007f90e9e50000) ```

Dune 2 is not compatible with OCaml 4.05 which we depend on at this stage; there is the hack to still have it with compiling a secondary, more recent OCaml on the switch but I really like *not* to do that. And dune 1 seems to work fine.

`make static-binaries` will generate `learnocaml`, `learnocaml-client` and `learnocaml-server` in at the project root

Linking in the client docker builds seem to fail with the most recent dune version...

AltGr · 2021-07-16T13:28:49Z

Updated to use the env, thanks @gasche for the suggestion

@gasche

…round Thanks @gasche for the hint about this hidden dune feature :)

erikmd

Hi @AltGr, thanks a lot for this work! (and sorry for the delay… I was quite swamped with teaching this term :-/)

FYI, I've tested successfully the last version of the artifact with:

GNU/Linux Debian 10 amd64, and
macOS 11.1, thanks to the help of a colleague of mine.

Apart from minor comments I've left in the review (regarding documentation and bash details), I've identified three points that could be improved, but not necessarily within this PR:

The GHA artifacts don't seem to be public (I believe I am able to download them only because I am collaborator of the repo) and according to the docs, they are removed after 90 days, so the way to go would be to rely on the GHA https://github.com/softprops/action-gh-release (mentioned by the legacy GHA https://github.com/actions/upload-release-asset)
For the sake of CI, to detect regressions (namely, if a dependency is missing, and this missing dependency couldn't not be detected without running the binary after compilation), I believe it would be nice to add one job that fetches the artifact using actions/download-artifact@v2 and runs, say, learn-ocaml-client --version; WDYT? and do you believe running the --version command would suffice to detect missing deps?
For MacOS, my colleague got some warning Impossible d’ouvrir « … » car le développeur ne peut pas être vérifié., as the binary is not signed :-|
however it was fine after running xattr -d com.apple.quarantine learn-ocaml-client as suggested here, for example. As neither @yurug nor me have a macOS, I guess it'd be difficult to sign the binaries for the time being, so I'd propose that we document that the macOS binaries are not signed but can be enabled this way with xattr…

@AltGr, as the points 1. and 2. above are just related to GHA and can be addressed in another PR (and the point 3. has a workaround), I can propose to look after 1. and 2. myself in another PR if you want; in order to merge this one as soon as you have taken a look at my other few line-comments.

erikmd · 2021-07-19T15:55:14Z

.github/workflows/static-builds.yml

+      # Need unreleased 2.1.0~rc
+      # - name: Retrieve opam
+      #   run: |
+      #     mkdir "$HOME/bin"
+      #     wget https://github.com/ocaml/opam/releases/download/2.1.0-beta2/opam-2.1.0-beta2-x86_64-macos -O $HOME/bin/opam
+      #     chmod a+x $HOME/bin/opam
+      #     echo "$HOME/bin" >> $GITHUB_PATH
+      - name: Install latest opam
+        run: |
+          brew install opam --HEAD


Could you add a comment somewhere recalling the feature (or PR) that is needed here, and not available in opam 2.0.x?

scripts/static-build.sh

erikmd · 2021-07-19T16:06:50Z

scripts/static-build.sh

+      'tar x >&2 &&
+       sudo apk add openssl-libs-static >&2 &&
+       opam switch create . ocaml-system "dune<2" --deps-only >&2 &&
+       opam exec make LINKING_MODE=static >&2 &&


Nitpick (if you don't disagree):

Suggested change

opam exec make LINKING_MODE=static >&2 &&

opam exec -- make LINKING_MODE=static >&2 &&

src/main/linking_flags.sh

erikmd · 2021-07-19T19:04:41Z

The GHA artifacts don't seem to be public … and according to the docs, they are removed after 90 days.

Actually this is customizable, but necessarily <= 90 days:

Co-authored-by: Erik Martin-Dorel <[email protected]>

AltGr · 2021-07-22T15:50:52Z

Feel free to finalise or clarify some comments if you think it's needed before merging; for me it's good to go.

* For each OS, run "file", "ldd" or so and "_ --version" * No need for "actions/download-artifact@v2"

erikmd · 2021-07-22T23:20:55Z

@AltGr (Cc @gasche) just FTR, here is a recap of this PR status and of the few commits I've just pushed, following this Thursday's video meeting with Louis.

First, to follow-up my earlier comment #391 (review) :

The GHA artifacts don't seem to be public (I believe I am able to download them only because I am collaborator of the repo) and according to the docs, they are removed after 90 days

→ indeed, we could/should rely on https://github.com/softprops/action-gh-release
I'll take care of this before the upcoming release.

For the sake of CI, to detect regressions (namely, if a dependency is missing, and this missing dependency couldn't not be detected without running the binary after compilation), I believe it would be nice to add one job that fetches the artifact using actions/download-artifact@v2 and runs, say, learn-ocaml-client --version

Actually it seems unneeded to use actions/download-artifact@v2 because running directly learn-ocaml-client --version in the toplevel context within the GHA runner ensures we check the "portability" of the static binary: compiled in alpine (musl-based), then run in Ubuntu Linux.

Anyway I've added a few unit tests, running not only ldd learn-ocaml-client … but also file learn-ocaml-client … and learn-ocaml-client --version (likewise for the two other binaries).

Then, I took the opportunity of this PR to automate what Louis had documented in src/main/linking_flags.sh: there is a new Makefile target that prints this 🙂

$ make detect-libs  # run on Debian GNU/Linux 10 amd64
...
learnocaml_client: -lcamlstr -lcstruct_stubs -lbase_stubs -lssl_threads_stubs -lssl -lcrypto -llwt_unix_stubs -lpthread -lthreads -lunix -lpthread -lbigarray -lunix -lcamlrun -lm -ldl -lpthread
learnocaml_main: -llaolao_stubs -lbase_stubs -lssl_threads_stubs -lssl -lcrypto -llwt_unix_stubs -lpthread -lthreads -lunix -lpthread -lcamlstr -lcstruct_stubs -lbigarray -lunix -lcamlrun -lm -ldl -lpthread
learnocaml_server_main: -lcamlstr -llaolao_stubs -lbase_stubs -lssl_threads_stubs -lssl -lcrypto -llwt_unix_stubs -lpthread -lthreadsnat -lpthread -lcstruct_stubs -lbigarray -lunix -lm -ldl

I have only two remaining questions for @AltGr:

among the libs mentioned above, this one is not mentioned in src/main/linking_flags.sh nor in ./dune: -ldl. Is this OK anyway? or we should add it in COMMON_LIBS.
on macOS, the otool -L command displays (link to the logs):

_build/install/default/bin/learn-ocaml:
	/usr/lib/libSystem.B.dylib (compatibility version 1.0.0, current version 1281.100.1)
	/usr/lib/libncurses.5.4.dylib (compatibility version 5.4.0, current version 5.4.0)
_build/install/default/bin/learn-ocaml-client:
	/usr/lib/libSystem.B.dylib (compatibility version 1.0.0, current version 1281.100.1)
	/usr/lib/libncurses.5.4.dylib (compatibility version 5.4.0, current version 5.4.0)
_build/install/default/bin/learn-ocaml-server:
	/usr/lib/libSystem.B.dylib (compatibility version 1.0.0, current version 1281.100.1)

While /usr/lib/libSystem.B.dylib corresponds to the macOS standard library that looks mandatory to keep as is, do you think that /usr/lib/libncurses.5.4.dylib ought to be "bundled", somehow?

Anyway, as said earlier, we had tested these executables on macOS 11.1 and they were running fine… so maybe a good motto is "don't fix what is not broken…" :)

To sum up (and sorry for the long post):

I think the PR is ready, so if @AltGr confirms there is nothing further to do for the two points above, I could merge it on, say, this Friday.

href: https://superuser.com/q/307165

erikmd · 2021-07-23T01:42:23Z

following-up my question

on macOS

about this code

learn-ocaml/src/main/linking_flags.sh

Line 50 in 7276fab

    
           COMMON_LIBS="camlstr base_stubs ssl_threads_stubs /usr/local/opt/openssl/lib/libssl.a /usr/local/opt/openssl/lib/libcrypto.a cstruct_stubs lwt_unix_stubs bigarray unix pthread"

To sum up, thanks to these added lines:

learn-ocaml/.ci-macosx.sh

Lines 15 to 16 in 7276fab

    
           # See src/main/linking_flags.sh 
        
           make detect-libs

we get on macOS (link to the logs):

learnocaml_client: -lcamlstr -lcstruct_stubs -lbase_stubs -lssl_threads_stubs -lssl -lcrypto -llwt_unix_stubs -lpthread -lthreads -lunix -lpthread -lbigarray -lunix -lcamlrun -lcurses -lpthread
learnocaml_main: -llaolao_stubs -lbase_stubs -lssl_threads_stubs -lssl -lcrypto -llwt_unix_stubs -lpthread -lthreads -lunix -lpthread -lcamlstr -lcstruct_stubs -lbigarray -lunix -lcamlrun -lcurses -lpthread
learnocaml_server_main: -lcamlstr -llaolao_stubs -lbase_stubs -lssl_threads_stubs -lssl -lcrypto -llwt_unix_stubs -lpthread -lthreadsnat -lpthread -lcstruct_stubs -lbigarray -lunix

so I guess that we could add curses on the two lines calling ./linking_flags.sh?

learn-ocaml/src/main/dune

Lines 57 to 64 in 7276fab

    
           (rule 
        
             (targets linking_main.sexp) 
        
             (action  (with-stdout-to %{targets} 
        
               (run ./linking_flags.sh %{env:LINKING_MODE=dynamic} laolao_stubs threads camlrun)))) 
        
           (rule 
        
             (targets linking_client.sexp) 
        
             (action  (with-stdout-to %{targets} 
        
               (run ./linking_flags.sh %{env:LINKING_MODE=dynamic} threads camlrun))))

(conditionaly if (= %{ocaml-config:system} macosx), maybe)

AltGr · 2021-07-23T10:07:26Z

among the libs mentioned above, this one is not mentioned in src/main/linking_flags.sh nor in ./dune: -ldl. Is this OK anyway? or we should add it in COMMON_LIBS.

Well dl stands for dynamic loading, so... we shouldn't need it :)

Anyway, as said earlier, we had tested these executables on macOS 11.1 and they were running fine… so maybe a good motto is "don't fix what is not broken…" :)

Yes, apparently you are not really supposed to do purely static executables on macOS, and the system is much, much more homogeneous so we shouldn't need to bother either, our execs should run; the point of statically linking the libs is to avoid runtime dependencies upon optional libs/packages.

so I guess that we could add curses on the two lines calling ./linking_flags.sh?

IIRC curses is part of the base macOS install so it shouldn't be required, but we could add it (maybe there are older versions still around on some systems ??).
That leaves open the question of why the heck learn-ocaml would require ncurses but well...

erikmd · 2021-07-23T13:38:39Z

Hi @AltGr, thanks for your feedback.

That leaves open the question of why the heck learn-ocaml would require ncurses but well...

Maybe because of the cmdliner opam dependency?

Also just FYI, I've found this reference saying:

-lm or -lpthread do no harm, but are unnecessary. The -lm option links to the math library, and -lpthread links to the POSIX threads library. Since libSystem provides these functions, you don't need to use these options.
…
In Mac OS X 10.1 and earlier versions, the curses screen library (a set of functions for controlling a terminal display) was part of libSystem.dylib. In Mac OS X 10.2 (Jaguar), the ncurses library (/usr/lib/libncurses.5.dylib) took the place of curses. You may still encounter source code releases that look for curses in libSystem.dylib, which will result in linking errors. You can work around this problem by adding -lcurses to the linker arguments. This is portable to earlier versions of Mac OS X as well, …

So as a last tweak, I've removed pthread when %{ocaml-config:system} = macosx, and I've added curses as had suggested the output of make detect-libs… If the CI passes anew 😅, I'll merge the current version.

erikmd added the kind: feature New user-facing feature. label May 14, 2021

AltGr mentioned this pull request Jun 1, 2021

Static builds #388

Closed

gasche reviewed Jun 2, 2021

View reviewed changes

scripts/setup-static-build.sh Outdated Show resolved Hide resolved

AltGr added 5 commits July 16, 2021 15:05

Don't require dune 2

9aa56d3

Dune 2 is not compatible with OCaml 4.05 which we depend on at this stage; there is the hack to still have it with compiling a secondary, more recent OCaml on the switch but I really like *not* to do that. And dune 1 seems to work fine.

Add a script to build statically-linked binaries using Docker and musl

aee8d7e

`make static-binaries` will generate `learnocaml`, `learnocaml-client` and `learnocaml-server` in at the project root

Limit maximum version of dune

d20ac0b

Linking in the client docker builds seem to fail with the most recent dune version...

Github actions to generate the static binaries (Linux, Macos)

a59798b

AltGr force-pushed the gh-act-static-bin branch from 9a93dab to 0222a4a Compare July 16, 2021 13:28

Static builds: use an environment variable rather than files laying a…

1150bfa

…round Thanks @gasche for the hint about this hidden dune feature :)

AltGr force-pushed the gh-act-static-bin branch from 0222a4a to 1150bfa Compare July 16, 2021 14:59

erikmd reviewed Jul 19, 2021

View reviewed changes

More robust static-build.sh

a556f76

Co-authored-by: Erik Martin-Dorel <[email protected]>

erikmd added 3 commits July 22, 2021 18:53

test: Run more tests

d1a25f6

* For each OS, run "file", "ldd" or so and "_ --version" * No need for "actions/download-artifact@v2"

chore(Makefile): Add make detect-libs

91cef45

test(.ci-macosx.sh): Run make detect-libs

a25955c

fix: details

e41d8a1

erikmd force-pushed the gh-act-static-bin branch from 781edca to e41d8a1 Compare July 22, 2021 23:23

fix: small issue with make detect-libs on macOS

7276fab

href: https://superuser.com/q/307165

erikmd self-assigned this Jul 23, 2021

erikmd added 2 commits July 23, 2021 14:58

chore: Skip -lpthread as libpthread is builtin in libSystem.dylib

74cb300

fix(client,main): Add -lcurses if (= %{ocaml-config:system} macosx)

18e1d50

erikmd added 2 commits July 23, 2021 15:25

chore(GHA): Split cmd => (Build the binaries, Test the binaries)

a1265b2

chore(GHA): Print OS version for macosx

68a302f

erikmd merged commit ff9f091 into ocaml-sf:master Jul 23, 2021

erikmd added this to the learn-ocaml 0.13.0 milestone Sep 30, 2021

erikmd mentioned this pull request Oct 5, 2021

chore: release 0.13.0 #435

Merged

proofbot mentioned this pull request Oct 5, 2021

2 packages from ocaml-sf/learn-ocaml at 0.13.0 ocaml/opam-repository#19698

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

Add github action rules for static binaries generation (Linux, Macos) #391

Add github action rules for static binaries generation (Linux, Macos) #391

Uh oh!

AltGr commented May 6, 2021 •

edited by erikmd

Loading

Uh oh!

AltGr commented May 6, 2021

Uh oh!

erikmd commented May 13, 2021

Uh oh!

Uh oh!

AltGr commented Jul 16, 2021

Uh oh!

erikmd left a comment

Uh oh!

erikmd Jul 19, 2021

Uh oh!

Uh oh!

erikmd Jul 19, 2021

Uh oh!

Uh oh!

erikmd commented Jul 19, 2021

Uh oh!

AltGr commented Jul 22, 2021

Uh oh!

erikmd commented Jul 22, 2021

Uh oh!

erikmd commented Jul 23, 2021 •

edited

Loading

Uh oh!

AltGr commented Jul 23, 2021

Uh oh!

erikmd commented Jul 23, 2021

Uh oh!

Uh oh!

	opam exec make LINKING_MODE=static >&2 &&
	opam exec -- make LINKING_MODE=static >&2 &&

Add github action rules for static binaries generation (Linux, Macos) #391

Add github action rules for static binaries generation (Linux, Macos) #391

Uh oh!

Conversation

AltGr commented May 6, 2021 • edited by erikmd Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

AltGr commented May 6, 2021

Uh oh!

erikmd commented May 13, 2021

Uh oh!

Uh oh!

AltGr commented Jul 16, 2021

Uh oh!

erikmd left a comment

Choose a reason for hiding this comment

Uh oh!

erikmd Jul 19, 2021

Choose a reason for hiding this comment

Uh oh!

Uh oh!

erikmd Jul 19, 2021

Choose a reason for hiding this comment

Uh oh!

Uh oh!

erikmd commented Jul 19, 2021

Uh oh!

AltGr commented Jul 22, 2021

Uh oh!

erikmd commented Jul 22, 2021

To sum up (and sorry for the long post):

Uh oh!

erikmd commented Jul 23, 2021 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

AltGr commented Jul 23, 2021

Uh oh!

erikmd commented Jul 23, 2021

Uh oh!

Uh oh!

AltGr commented May 6, 2021 •

edited by erikmd

Loading

erikmd commented Jul 23, 2021 •

edited

Loading