-
Notifications
You must be signed in to change notification settings - Fork 66
Add github action rules for static binaries generation (Linux, Macos) #391
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Conversation
Aforementioned binaries can be downloaded, in the examples of this run, here: https://github.com/ocaml-sf/learn-ocaml/actions/runs/817393844 Testers welcome! (note that this might be turned on only for master and tags, but I wanted to demonstrate the PR first :) ) |
Hi @AltGr, thanks a lot for these two PRs 👍 I'll test the generated binaries ASAP − at least the Linux ones − and let you know! |
To enable: `ln -s static-linking.sexp src/main/linking.sexp` This still links dynamically to glibc and pthreads, but should already considerably improve portability. Next to do is use musl instead of glibc. Before: ``` % ldd _build/default/src/main/learnocaml_server_main.exe linux-vdso.so.1 (0x00007fff743ee000) libssl.so.1.1 => /usr/lib/x86_64-linux-gnu/libssl.so.1.1 (0x00007f29aeb43000) libcrypto.so.1.1 => /usr/lib/x86_64-linux-gnu/libcrypto.so.1.1 (0x00007f29ae84f000) libpthread.so.0 => /lib/x86_64-linux-gnu/libpthread.so.0 (0x00007f29ae82d000) libm.so.6 => /lib/x86_64-linux-gnu/libm.so.6 (0x00007f29ae6e9000) libdl.so.2 => /lib/x86_64-linux-gnu/libdl.so.2 (0x00007f29ae6e3000) libc.so.6 => /lib/x86_64-linux-gnu/libc.so.6 (0x00007f29ae51e000) /lib64/ld-linux-x86-64.so.2 (0x00007f29af80e000) ``` After: ``` % ldd _build/default/src/main/learnocaml_server_main.exe linux-vdso.so.1 (0x00007ffcdddd5000) libpthread.so.0 => /lib/x86_64-linux-gnu/libpthread.so.0 (0x00007f90e8f0e000) libm.so.6 => /lib/x86_64-linux-gnu/libm.so.6 (0x00007f90e8dca000) libdl.so.2 => /lib/x86_64-linux-gnu/libdl.so.2 (0x00007f90e8dc4000) libc.so.6 => /lib/x86_64-linux-gnu/libc.so.6 (0x00007f90e8bff000) /lib64/ld-linux-x86-64.so.2 (0x00007f90e9e50000) ```
Dune 2 is not compatible with OCaml 4.05 which we depend on at this stage; there is the hack to still have it with compiling a secondary, more recent OCaml on the switch but I really like *not* to do that. And dune 1 seems to work fine.
`make static-binaries` will generate `learnocaml`, `learnocaml-client` and `learnocaml-server` in at the project root
Linking in the client docker builds seem to fail with the most recent dune version...
Updated to use the env, thanks @gasche for the suggestion |
…round Thanks @gasche for the hint about this hidden dune feature :)
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Hi @AltGr, thanks a lot for this work! (and sorry for the delay… I was quite swamped with teaching this term :-/)
FYI, I've tested successfully the last version of the artifact with:
- GNU/Linux Debian 10 amd64, and
- macOS 11.1, thanks to the help of a colleague of mine.
Apart from minor comments I've left in the review (regarding documentation and bash details), I've identified three points that could be improved, but not necessarily within this PR:
-
The GHA artifacts don't seem to be public (I believe I am able to download them only because I am collaborator of the repo) and according to the docs, they are removed after 90 days, so the way to go would be to rely on the GHA https://github.com/softprops/action-gh-release (mentioned by the legacy GHA https://github.com/actions/upload-release-asset)
-
For the sake of CI, to detect regressions (namely, if a dependency is missing, and this missing dependency couldn't not be detected without running the binary after compilation), I believe it would be nice to add one job that fetches the artifact using
actions/download-artifact@v2
and runs, say,learn-ocaml-client --version
; WDYT? and do you believe running the--version
command would suffice to detect missing deps? -
For MacOS, my colleague got some warning
Impossible d’ouvrir « … » car le développeur ne peut pas être vérifié.
, as the binary is not signed :-|
however it was fine after runningxattr -d com.apple.quarantine learn-ocaml-client
as suggested here, for example. As neither @yurug nor me have a macOS, I guess it'd be difficult to sign the binaries for the time being, so I'd propose that we document that the macOS binaries are not signed but can be enabled this way withxattr
…
@AltGr, as the points 1. and 2. above are just related to GHA and can be addressed in another PR (and the point 3. has a workaround), I can propose to look after 1. and 2. myself in another PR if you want; in order to merge this one as soon as you have taken a look at my other few line-comments.
# Need unreleased 2.1.0~rc | ||
# - name: Retrieve opam | ||
# run: | | ||
# mkdir "$HOME/bin" | ||
# wget https://github.com/ocaml/opam/releases/download/2.1.0-beta2/opam-2.1.0-beta2-x86_64-macos -O $HOME/bin/opam | ||
# chmod a+x $HOME/bin/opam | ||
# echo "$HOME/bin" >> $GITHUB_PATH | ||
- name: Install latest opam | ||
run: | | ||
brew install opam --HEAD |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Could you add a comment somewhere recalling the feature (or PR) that is needed here, and not available in opam 2.0.x?
'tar x >&2 && | ||
sudo apk add openssl-libs-static >&2 && | ||
opam switch create . ocaml-system "dune<2" --deps-only >&2 && | ||
opam exec make LINKING_MODE=static >&2 && |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Nitpick (if you don't disagree):
opam exec make LINKING_MODE=static >&2 && | |
opam exec -- make LINKING_MODE=static >&2 && |
Co-authored-by: Erik Martin-Dorel <[email protected]>
Feel free to finalise or clarify some comments if you think it's needed before merging; for me it's good to go. |
* For each OS, run "file", "ldd" or so and "_ --version" * No need for "actions/download-artifact@v2"
@AltGr (Cc @gasche) just FTR, here is a recap of this PR status and of the few commits I've just pushed, following this Thursday's video meeting with Louis. First, to follow-up my earlier comment #391 (review) :
→ indeed, we could/should rely on https://github.com/softprops/action-gh-release
Actually it seems unneeded to use Anyway I've added a few unit tests, running not only Then, I took the opportunity of this PR to automate what Louis had documented in
I have only two remaining questions for @AltGr:
While Anyway, as said earlier, we had tested these executables on macOS 11.1 and they were running fine… so maybe a good motto is "don't fix what is not broken…" :) To sum up (and sorry for the long post):I think the PR is ready, so if @AltGr confirms there is nothing further to do for the two points above, I could merge it on, say, this Friday. |
781edca
to
e41d8a1
Compare
following-up my question
about this code learn-ocaml/src/main/linking_flags.sh Line 50 in 7276fab
To sum up, thanks to these added lines: Lines 15 to 16 in 7276fab
we get on macOS (link to the logs):
so I guess that we could add Lines 57 to 64 in 7276fab
(conditionaly if (= %{ocaml-config:system} macosx) , maybe)
|
Well
Yes, apparently you are not really supposed to do purely static executables on macOS, and the system is much, much more homogeneous so we shouldn't need to bother either, our execs should run; the point of statically linking the libs is to avoid runtime dependencies upon optional libs/packages.
IIRC curses is part of the base macOS install so it shouldn't be required, but we could add it (maybe there are older versions still around on some systems ??). |
Hi @AltGr, thanks for your feedback.
Maybe because of the Also just FYI, I've found this reference saying:
So as a last tweak, I've removed |
Note: this PR supersedes #388.