Skip to content

Ninja fails to compile on alpine linux #3666

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
briangorman opened this issue Jul 8, 2019 · 19 comments
Closed

Ninja fails to compile on alpine linux #3666

briangorman opened this issue Jul 8, 2019 · 19 comments

Comments

@briangorman
Copy link

briangorman commented Jul 8, 2019

Alpine 3.8

Seems to be an upstream issue with ninja

ninja-build/ninja#1510

One possibility to "fix" this from bucklescript's side is to use the operating system supplied ninja if possible. This would also speed up builds.

https://pkgs.alpinelinux.org/packages?name=ninja&branch=v3.8

#!/bin/bash -eo pipefail
yarn install --non-interactive
yarn install v1.12.3
info No lockfile found.
[1/4] Resolving packages...
warning react-dates > react-addons-shallow-compare > fbjs > [email protected]: core-js@<2.6.8 is no longer maintained. Please, upgrade to core-js@3 or at least to actual version of core-js@2.
warning Resolution field "[email protected]" is incompatible with requested version "react-dates@^16.5.0"
warning @glennsl/bs-jest > jest > jest-cli > prompts > [email protected]: Please upgrade to kleur@3 or migrate to 'ansi-colors' if you prefer the old syntax. Visit <https://github.com/lukeed/kleur/releases/tag/v3.0.0\> for migration path(s).
warning @glennsl/bs-jest > jest > jest-cli > jest-environment-jsdom > jsdom > [email protected]: use String.prototype.padStart()
warning bs-dom-testing-library > [email protected]: 🚨  dom-testing-library has moved to @testing-library/dom. Please uninstall dom-testing-library and install @testing-library/dom instead, or use an older version of dom-testing-library. Learn more about this change here: https://github.com/testing-library/dom-testing-library/issues/260 Thanks! :)
warning bs-react-testing-library > [email protected]: 🚨  react-testing-library has moved to @testing-library/react. Please uninstall react-testing-library and install @testing-library/react instead, or use an older version of react-testing-library. Learn more about this change here: https://github.com/testing-library/dom-testing-library/issues/260 Thanks! :)
warning bs-react-testing-library > react-testing-library > [email protected]: 🚨  dom-testing-library has moved to @testing-library/dom. Please uninstall dom-testing-library and install @testing-library/dom instead, or use an older version of dom-testing-library. Learn more about this change here: https://github.com/testing-library/dom-testing-library/issues/260 Thanks! :)
[2/4] Fetching packages...
info [email protected]: The platform "linux" is incompatible with this module.
info "[email protected]" is an optional dependency and failed compatibility check. Excluding it from installation.
[3/4] Linking dependencies...
warning " > [email protected]" has unmet peer dependency "moment@^2.18.1".
warning "react-dates > [email protected]" has unmet peer dependency "moment@>=1.6.0".
warning " > @ahrefs/[email protected]" has incorrect peer dependency "bs-platform@^4.0.5".
warning " > @glennsl/[email protected]" has incorrect peer dependency "bs-platform@^4.0.2".
warning " > @fuse/[email protected]" has incorrect peer dependency "[email protected]".
warning " > @fuse/[email protected]" has incorrect peer dependency "[email protected]".
warning "@fuse/s3-browser > [email protected]" has unmet peer dependency "webpack@^1 || ^2 || ^3 || ^4".
warning "@fuse/s3-browser > [email protected]" has unmet peer dependency "@babel/runtime@>=7.0.0".
warning "@fuse/s3-browser > [email protected]" has unmet peer dependency "@emotion/styled@^10.0.10".
warning " > [email protected]" has incorrect peer dependency "bs-platform@^4.0.6".
warning " > [email protected]" has incorrect peer dependency "bs-platform@^5.0.0".
warning " > [email protected]" has incorrect peer dependency "bs-platform@^5.0.0".
[4/4] Building fresh packages...
error /home/opam/project/node_modules/bs-platform: Command failed.
Exit code: 1
Command: node scripts/install.js
Arguments: 
Directory: /home/opam/project/node_modules/bs-platform
Output:
ninja not compatible? Error: Command failed: "/home/opam/project/node_modules/bs-platform/lib/ninja.exe" --version
building ninja
.clang-format
.gitignore
.travis.yml
COPYING
HACKING.md
README
RELEASING
appveyor.yml
bootstrap.py
configure.py
doc/
doc/README.md
doc/dblatex.xsl
doc/docbook.xsl
doc/doxygen.config
doc/manual.asciidoc
doc/style.css
misc/
misc/afl-fuzz-tokens/
misc/afl-fuzz-tokens/kw_build
misc/afl-fuzz-tokens/kw_default
misc/afl-fuzz-tokens/kw_include
misc/afl-fuzz-tokens/kw_pool
misc/afl-fuzz-tokens/kw_rule
misc/afl-fuzz-tokens/kw_subninja
misc/afl-fuzz-tokens/misc_a
misc/afl-fuzz-tokens/misc_b
misc/afl-fuzz-tokens/misc_colon
misc/afl-fuzz-tokens/misc_cont
misc/afl-fuzz-tokens/misc_dollar
misc/afl-fuzz-tokens/misc_eq
misc/afl-fuzz-tokens/misc_indent
misc/afl-fuzz-tokens/misc_pipe
misc/afl-fuzz-tokens/misc_pipepipe
misc/afl-fuzz-tokens/misc_space
misc/afl-fuzz/
misc/afl-fuzz/build.ninja
misc/bash-completion
misc/ci.py
misc/inherited-fds.ninja
misc/long-slow-build.ninja
misc/measure.py
misc/ninja-mode.el
misc/ninja.vim
misc/ninja_syntax.py
misc/ninja_syntax_test.py
misc/output_test.py
misc/packaging/
misc/packaging/ninja.spec
misc/packaging/rpmbuild.sh
misc/write_fake_manifests.py
misc/zsh-completion
snapshot.js
src/
src/browse.cc
src/browse.h
src/browse.py
src/build.cc
src/build.h
src/build_log.cc
src/build_log.h
src/build_log_perftest.cc
src/build_log_test.cc
src/build_test.cc
src/canon_perftest.cc
src/clean.cc
src/clean.h
src/clean_test.cc
src/clparser.cc
src/clparser.h
src/clparser_perftest.cc
src/clparser_test.cc
src/debug_flags.cc
src/debug_flags.h
src/depfile_parser.cc
src/depfile_parser.h
src/depfile_parser.in.cc
src/depfile_parser_perftest.cc
src/depfile_parser_test.cc
src/deps_log.cc
src/deps_log.h
src/deps_log_test.cc
src/disk_interface.cc
src/disk_interface.h
src/disk_interface_test.cc
src/dyndep.cc
src/dyndep.h
src/dyndep_parser.cc
src/dyndep_parser.h
src/dyndep_parser_test.cc
src/edit_distance.cc
src/edit_distance.h
src/edit_distance_test.cc
src/eval_env.cc
src/eval_env.h
src/exit_status.h
src/gen_doxygen_mainpage.sh
src/getopt.c
src/getopt.h
src/graph.cc
src/graph.h
src/graph_test.cc
src/graphviz.cc
src/graphviz.h
src/hash_collision_bench.cc
src/hash_map.h
src/includes_normalize-win32.cc
src/includes_normalize.h
src/includes_normalize_test.cc
src/inline.sh
src/lexer.cc
src/lexer.h
src/lexer.in.cc
src/lexer_test.cc
src/line_printer.cc
src/line_printer.h
src/manifest_parser.cc
src/manifest_parser.h
src/manifest_parser_perftest.cc
src/manifest_parser_test.cc
src/metrics.cc
src/metrics.h
src/minidump-win32.cc
src/msvc_helper-win32.cc
src/msvc_helper.h
src/msvc_helper_main-win32.cc
src/msvc_helper_test.cc
src/ninja.cc
src/ninja_test.cc
src/parser.cc
src/parser.h
src/state.cc
src/state.h
src/state_test.cc
src/string_piece.h
src/string_piece_util.cc
src/string_piece_util.h
src/string_piece_util_test.cc
src/subprocess-posix.cc
src/subprocess-win32.cc
src/subprocess.h
src/subprocess_test.cc
src/test.cc
src/test.h
src/timestamp.h
src/util.cc
src/util.h
src/util_test.cc
src/version.cc
src/version.h
src/win32port.h
No prebuilt Ninja, building Ninja now
bootstrapping ninja...
./src/disk_interface.cc: In member function 'virtual TimeStamp RealDiskInterface::Stat(const string&, std::__cxx11::string*) const':
./src/disk_interface.cc:219:51: error: 'struct stat' has no member named 'st_mtimensec'; did you mean 'st_mtim'?
   return (int64_t)st.st_mtime * 1000000000LL + st.st_mtimensec;
                                                   ^~~~~~~~~~~~
warning: A compatible version of re2c (>= 0.11.3) was not found; changes to src/*.in.cc will not affect your build.
when running:  g++ -MMD -MT build/disk_interface.o -MF build/disk_interface.o.d -g -Wall -Wextra -Wno-deprecated -Wno-missing-field-initializers -Wno-unused-parameter -fno-rtti -fno-exceptions -fvisibility=hidden -pipe '-DNINJA_PYTHON="python"' -O2 -DNDEBUG -fdiagnostics-color -DUSE_PPOLL -DNINJA_HAVE_BROWSE -I. -c ./src/disk_interface.cc -o build/disk_interface.o
Traceback (most recent call last):
  File "./configure.py", line 514, in <module>
    objs += cxx(name, variables=cxxvariables)
  File "./configure.py", line 282, in cxx
    return n.build(built(name + objext), 'cxx', src(name + '.cc'), **kwargs)
  File "./configure.py", line 164, in build
    self._run_command(self._expand(cmd, local_vars))
  File "./configure.py", line 189, in _run_command
    subprocess.check_call(cmdline, shell=True)
  File "/usr/lib/python2.7/subprocess.py", line 190, in check_call
    raise CalledProcessError(retcode, cmd)
subprocess.CalledProcessError: Command 'g++ -MMD -MT build/disk_interface.o -MF build/disk_interface.o.d -g -Wall -Wextra -Wno-deprecated -Wno-missing-field-initializers -Wno-unused-parameter -fno-rtti -fno-exceptions -fvisibility=hidden -pipe '-DNINJA_PYTHON="python"' -O2 -DNDEBUG -fdiagnostics-color -DUSE_PPOLL -DNINJA_HAVE_BROWSE -I. -c ./src/disk_interface.cc -o build/disk_interface.o' returned non-zero exit status 1
child_process.js:656
    throw err;
    ^

Error: Command failed: ./configure.py --bootstrap
    at checkExecSyncError (child_process.js:616:11)
    at Object.execSync (child_process.js:653:13)
    at build_ninja (/home/opam/project/node_modules/bs-platform/scripts/install.js:60:8)
    at provideNinja (/home/opam/project/node_modules/bs-platform/scripts/install.js:112:7)
    at Object.<anonymous> (/home/opam/project/node_modules/bs-platform/scripts/install.js:373:1)
    at Module._compile (internal/modules/cjs/loader.js:689:30)
    at Object.Module._extensions..js (internal/modules/cjs/loader.js:700:10)
    at Module.load (internal/modules/cjs/loader.js:599:32)
    at tryModuleLoad (internal/modules/cjs/loader.js:538:12)
    at Function.Module._load (internal/modules/cjs/loader.js:530:3)
info Visit https://yarnpkg.com/en/docs/cli/install for documentation about this command.
Exited with code 1```
@bobzhang
Copy link
Member

bobzhang commented Jul 9, 2019

we will apply the patch linked, unfortunately we can not use system provided ninja since we have important patches. will ping @briangorman you later for testing (I don't have alpine linux)

@bobzhang
Copy link
Member

bobzhang commented Jul 9, 2019

@briangorman can you confirm that it would be fixed in #3668?

@ashutheone
Copy link

ashutheone commented Jul 9, 2019

Facing the same issue while building with Docker using node:alpine
Log lines include following message:
error /node_modules/bs-platform: Command failed. Exit code: 1 Command: node scripts/install.js Arguments: Directory: /node_modules/bs-platform Output: ninja not compatible? Error: Command failed: "/node_modules/bs-platform/lib/ninja.exe" --version

@briangorman
Copy link
Author

@bobzhang I can confirm that the fix_3666 branch results in an executable ninja.exe in the lib folder, while master's install.js script fails to build ninja under alpine 3.10.

Thanks for the quick fix!

@tejesh014
Copy link

What is the latest version that contains this fix ?

@bobzhang
Copy link
Member

bobzhang commented Jul 10, 2019 via email

@ashutheone
Copy link

I tried the latest version (5.0.6) but that too did not work. Is it supposed to fix this issue? I was previously on v5.0.4.

@bobzhang
Copy link
Member

bobzhang commented Jul 10, 2019 via email

@GTDev87
Copy link

GTDev87 commented Jul 19, 2019

I'm having a hard time with deployment because of this. Has anyone found any simple workarounds for this till this is released? Maybe downloading an earlier version of Ninja?

@bobzhang
Copy link
Member

bobzhang commented Jul 20, 2019 via email

@bobzhang
Copy link
Member

Hi @GTDev87 , I published [email protected] in #3694 with a beta tag, hopefully it would unblock your work. Do make a test before in case some bugs leaked in the latest release.

@GTDev87
Copy link

GTDev87 commented Jul 21, 2019

Hey @bobzhang, I put it in and it seems to build now Thanks a lot!! I'll tell you if I find any problems.

@pm-mck
Copy link

pm-mck commented Aug 2, 2019

@bobzhang would you mind porting the ninja fix to a beta release of the 6.x tree?

@bobzhang
Copy link
Member

bobzhang commented Aug 4, 2019

@pm-mck certainly

@tlonist-sang
Copy link

Is this issue resolved? I'm using node:alpine with bs-platform version 8.2.0, still facing the same issue.

Error: Command failed: /app/node_modules/bs-platform/linux/ninja.exe -f release.ninja -v
at checkExecSyncError (child_process.js:630:11)
at Object.execFileSync (child_process.js:648:15)
at provideCompiler (/app/node_modules/bs-platform/scripts/install.js:206:8)
at Object. (/app/node_modules/bs-platform/scripts/install.js:217:1)
at Module._compile (internal/modules/cjs/loader.js:1137:30)
at Object.Module._extensions..js (internal/modules/cjs/loader.js:1157:10)
at Module.load (internal/modules/cjs/loader.js:985:32)
at Function.Module._load (internal/modules/cjs/loader.js:878:14)
at Function.executeUserEntryPoint [as runMain] (internal/modules/run_main.js:71:12)
at internal/main/run_main_module.js:17:47 {

@bobzhang
Copy link
Member

@tlonist-sang do you have a detailed log? it should build its own ninja for apline

@DZakh
Copy link
Member

DZakh commented Sep 18, 2021

After installing everything needed to build ninja it started working. The guys above should add more details, because for me it works fine.

Here is my Dockerfile for a nextjs application:

FROM node:14.17.0-alpine

ENV NINJA_DEPS \
    python3 \
    g++ \
    make

RUN apk add --update --no-cache --virtual .build-deps ${NINJA_DEPS} && \
    ln -sf python3 /usr/bin/python

RUN mkdir -p /workspace/promo && chown 1000:1000 /workspace/promo
WORKDIR /workspace/promo
USER 1000

ENV NEXT_TELEMETRY_DISABLED=1

COPY package*.json ./
RUN npm ci

USER root
RUN apk del .build-deps \
    && rm -rf /var/cache/apk/* /tmp/*

USER 1000
COPY . .

ENV NODE_ENV=production
RUN npm run build

CMD ["npm", "run", "start"]

The only negative thing is that it takes 7 minutes to build the ninja, so it becomes very said when docker's cache doesn't work.

@DZakh
Copy link
Member

DZakh commented Sep 20, 2021

In some reason the image built by the config I've posted above has the 1.84GB size. I didn't really wanted to debug it, so I switched to the node:14.17.0-slim base image. The image size became 648MB, the config became much simpler, and no waiting for 10+ minutes while the ninja is building.

The updated version of Dockerfile:

FROM node:14.17.0-slim

RUN mkdir -p /workspace/promo && chown 1000:1000 /workspace/promo
WORKDIR /workspace/promo
USER 1000

ENV NEXT_TELEMETRY_DISABLED=1

COPY package*.json ./
RUN npm ci

COPY . .

ENV NODE_ENV=production
RUN npm run build

CMD ["npm", "run", "start"]

@bobzhang
Copy link
Member

bobzhang commented Sep 26, 2021

It seems ninja does compile on alpine Linux, but it requires more deps, e.g, make, python.
I don't have time to work on it, but we are open to PRs to help remove thoese deps

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

8 participants