Build broken on Apple Silicon computers (and/or Graviton2 builds) #2

brainstorm · 2021-10-01T01:47:23Z

Hello @nmoutschen, branching out our discussions over here from https://twitter.com/braincode/status/1443549985803431947...

Also, btw, this is an issue I originally reported upstream on briansmith/ring#1332

$ git clone https://github.com/aws-samples/serverless-rust-demo.git
Cloning into 'serverless-rust-demo'...
remote: Enumerating objects: 43, done.
remote: Counting objects: 100% (43/43), done.
remote: Compressing objects: 100% (32/32), done.
remote: Total 43 (delta 13), reused 34 (delta 10), pack-reused 0
Receiving objects: 100% (43/43), 14.43 KiB | 3.61 MiB/s, done.
Resolving deltas: 100% (13/13), done.
$ cd serverless-rust-demo
$ make build
cargo build --release --target aarch64-unknown-linux-gnu
    Updating crates.io index
    Updating git repository `https://github.com/awslabs/aws-sdk-rust`
   Compiling libc v0.2.103
   Compiling proc-macro2 v1.0.29
   Compiling unicode-xid v0.2.2
   Compiling syn v1.0.77
   Compiling autocfg v1.0.1
   Compiling cfg-if v1.0.0
   Compiling log v0.4.14
   Compiling pin-project-lite v0.2.7
   Compiling once_cell v1.8.0
   Compiling lazy_static v1.4.0
   Compiling memchr v2.4.1
   Compiling bytes v1.1.0
   Compiling smallvec v1.7.0
   Compiling futures-core v0.3.17
   Compiling itoa v0.4.8
   Compiling parking_lot_core v0.8.5
   Compiling scopeguard v1.1.0
   Compiling proc-macro-hack v0.5.19
   Compiling futures-channel v0.3.17
   Compiling proc-macro-nested v0.1.7
   Compiling futures-task v0.3.17
   Compiling futures-sink v0.3.17
   Compiling pin-utils v0.1.0
   Compiling slab v0.4.4
   Compiling futures-io v0.3.17
   Compiling ryu v1.0.5
   Compiling fnv v1.0.7
   Compiling hashbrown v0.11.2
   Compiling httparse v1.5.1
   Compiling tower-service v0.3.1
   Compiling try-lock v0.2.3
   Compiling percent-encoding v2.1.0
   Compiling httpdate v1.0.1
   Compiling cc v1.0.70
   Compiling spin v0.5.2
   Compiling untrusted v0.7.1
   Compiling semver v1.0.4
   Compiling either v1.6.1
   Compiling regex-syntax v0.6.25
   Compiling ppv-lite86 v0.2.10
   Compiling base64 v0.13.0
   Compiling anyhow v1.0.44
   Compiling serde_derive v1.0.130
   Compiling serde v1.0.130
   Compiling tower-layer v0.3.1
   Compiling zeroize v1.4.2
   Compiling matches v0.1.9
   Compiling serde_json v1.0.68
   Compiling openssl-probe v0.1.4
   Compiling hex v0.4.3
   Compiling async-trait v0.1.51
   Compiling fixedbitset v0.2.0
   Compiling remove_dir_all v0.5.3
   Compiling crossbeam-utils v0.8.5
   Compiling unicode-segmentation v1.8.0
   Compiling ansi_term v0.12.1
   Compiling multimap v0.8.3
   Compiling fastrand v1.5.0
   Compiling urlencoding v1.3.3
   Compiling xmlparser v0.13.3
   Compiling instant v0.1.11
   Compiling tracing-core v0.1.20
   Compiling sharded-slab v0.1.3
   Compiling thread_local v1.1.3
   Compiling indexmap v1.7.0
   Compiling tokio v1.12.0
   Compiling futures-macro v0.3.17
   Compiling futures-util v0.3.17
   Compiling num-traits v0.2.14
   Compiling num-integer v0.1.44
   Compiling lock_api v0.4.5
   Compiling http v0.2.5
   Compiling itertools v0.10.1
   Compiling bytes-utils v0.1.1
   Compiling form_urlencoded v1.0.1
   Compiling ring v0.16.20
   Compiling heck v0.3.3
   Compiling regex v1.5.4
   Compiling regex-automata v0.1.10
   Compiling want v0.3.0
   Compiling tracing-log v0.1.2
error: failed to run custom build command for `ring v0.16.20`

Caused by:
  process didn't exit successfully: `/Users/rvalls/dev/umccr/serverless-rust-demo/target/release/build/ring-5100f9369a354546/build-script-build` (exit status: 101)
  --- stdout
  OPT_LEVEL = Some("3")
  TARGET = Some("aarch64-unknown-linux-gnu")
  HOST = Some("aarch64-apple-darwin")
  CC_aarch64-unknown-linux-gnu = None
  CC_aarch64_unknown_linux_gnu = Some("aarch64-linux-gnu-gcc")
  CFLAGS_aarch64-unknown-linux-gnu = None
  CFLAGS_aarch64_unknown_linux_gnu = None
  TARGET_CFLAGS = None
  CFLAGS = None
  CRATE_CC_NO_DEFAULTS = None
  DEBUG = Some("false")
  CARGO_CFG_TARGET_FEATURE = None

  --- stderr
  running "aarch64-linux-gnu-gcc" "-O3" "-ffunction-sections" "-fdata-sections" "-fPIC" "-I" "include" "-Wall" "-Wextra" "-pedantic" "-pedantic-errors" "-Wall" "-Wextra" "-Wcast-align" "-Wcast-qual" "-Wconversion" "-Wenum-compare" "-Wfloat-equal" "-Wformat=2" "-Winline" "-Winvalid-pch" "-Wmissing-field-initializers" "-Wmissing-include-dirs" "-Wredundant-decls" "-Wshadow" "-Wsign-compare" "-Wsign-conversion" "-Wundef" "-Wuninitialized" "-Wwrite-strings" "-fno-strict-aliasing" "-fvisibility=hidden" "-fstack-protector" "-g3" "-DNDEBUG" "-c" "-o/Users/rvalls/dev/umccr/serverless-rust-demo/target/aarch64-unknown-linux-gnu/release/build/ring-5243877d4a8e9964/out/aesv8-armx-linux64.o" "/Users/rvalls/.cargo/registry/src/github.1485827954.workers.dev-1ecc6299db9ec823/ring-0.16.20/pregenerated/aesv8-armx-linux64.S"
  thread 'main' panicked at 'failed to execute ["aarch64-linux-gnu-gcc" "-O3" "-ffunction-sections" "-fdata-sections" "-fPIC" "-I" "include" "-Wall" "-Wextra" "-pedantic" "-pedantic-errors" "-Wall" "-Wextra" "-Wcast-align" "-Wcast-qual" "-Wconversion" "-Wenum-compare" "-Wfloat-equal" "-Wformat=2" "-Winline" "-Winvalid-pch" "-Wmissing-field-initializers" "-Wmissing-include-dirs" "-Wredundant-decls" "-Wshadow" "-Wsign-compare" "-Wsign-conversion" "-Wundef" "-Wuninitialized" "-Wwrite-strings" "-fno-strict-aliasing" "-fvisibility=hidden" "-fstack-protector" "-g3" "-DNDEBUG" "-c" "-o/Users/rvalls/dev/umccr/serverless-rust-demo/target/aarch64-unknown-linux-gnu/release/build/ring-5243877d4a8e9964/out/aesv8-armx-linux64.o" "/Users/rvalls/.cargo/registry/src/github.1485827954.workers.dev-1ecc6299db9ec823/ring-0.16.20/pregenerated/aesv8-armx-linux64.S"]: No such file or directory (os error 2)', /Users/rvalls/.cargo/registry/src/github.1485827954.workers.dev-1ecc6299db9ec823/ring-0.16.20/build.rs:653:9
  note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace
warning: build failed, waiting for other jobs to finish...
error: build failed
make: *** [build] Error 101

The text was updated successfully, but these errors were encountered:

nmoutschen · 2021-10-22T08:57:19Z

Hey @brainstorm !

After some experimentation on M1 macs (thanks @msailes for the help):

cross build doesn't seem to run into a container on M1 macs
using CC_aarch64_unknown_linux_gnu=clang cross build --target aarch64-unknown-linux-gnu fixes the No such file or directory (os error 2) issue.

However, there is another issue popping up ('assert.h' file not found) when building ring.

nmoutschen · 2021-10-22T09:23:15Z

For Graviton2 instances, I just tried with cross and it works fine.

brainstorm · 2021-10-25T06:22:51Z

From what I just learned in briansmith/ring#1332 (comment) and spack/spack#26317 (comment), this might require GCC12 to be released in order to be fixed properly for M1 machines :-S

michaelbrewer · 2021-11-08T17:36:21Z

For sam i have had to make a custom Makefile

AWSTemplateFormatVersion : '2010-09-09'
Transform: AWS::Serverless-2016-10-31
Description: Hello World Rust Lambda function 

Resources:
  HelloRustFunction:
    Type: AWS::Serverless::Function
    Properties:
      FunctionName: HelloRust
      Handler: bootstrap.is.real.handler
      Runtime: provided.al2
      MemorySize: 256
      CodeUri: .
      Architectures:
        - arm64
    Metadata:
      BuildMethod: makefile

Which currently just references a simple shell

build-HelloRustFunction:
	sh ./build.sh
	cp ./target/aarch64-unknown-linux-gnu/release/bootstrap $(ARTIFACTS_DIR)

The actual docker build

LAMBDA_ARCH="linux/arm64"
RUST_TARGET="aarch64-unknown-linux-gnu"
RUST_VERSION="latest"

docker run \
 --platform ${LAMBDA_ARCH} \
 --rm --user "$(id -u)":"$(id -g)" \
 -v "${PWD}":/usr/src/myapp -w /usr/src/myapp rust:${RUST_VERSION} \
 cargo build --release --target ${RUST_TARGET}

brainstorm · 2021-11-08T21:09:15Z

Thanks @michaelbrewer, that's pretty much what I use in:

https://github.com/umccr/s3-rust-noodles-bam

Not ideal though:

https://github.com/umccr/s3-rust-noodles-bam/blob/master/FLUKES.md

Local builds should take less that 6 minutes and, ideally, they should not involve docker at all (just regular cargo build, which takes a few seconds in a M1)... it should just work and compile aws-rust-sdk dependencies like ring, IMHO.

nmoutschen · 2021-11-15T18:41:01Z

I finally managed to get my hands on an M1 mac to test around. The latest release of cross (0.2.1, released in June 2020) doesn't actually run cargo in a container when run on an M1 mac.

It does work when installing it from the Git repo directly (cargo install --git https://github.com/rust-embedded/cross.git). However, it's very, very slow, so. I haven't completed a build yet.

brainstorm · 2021-12-01T00:30:13Z

Did you manage to reproduce my steps on briansmith/ring#1332 (comment) ? I'm curious to see if you are able to crack this ring dependency nut, IMO it's all what's needed to solve in order to have good M1-based builds sans docker/cross.

nmoutschen · 2021-12-01T09:27:12Z

I've got the same error message (No such file or directory (os error 2)) even on non-M1 Macs. I see that you're using aarch64-linux-gnu-gcc as compiler, but depending on what toolchain you've installed, it might have a slightly different name (e.g. aarch64-unknown-linux-gnu-gcc on my x86 mac).

That's actually one of the reasons why I prefer to use cross here: while it doesn't work well for M1 macs yet, it abstracts those small differences away so that most people can build this project no matter their setups.

nmoutschen · 2022-02-18T10:24:47Z

This now uses zigbuild, which should solve the issue with M1 macs compared to cross. I'll close this issue for now as I got confirmation it worked, but let me know if it didn't work for you.

brainstorm · 2022-04-04T02:03:43Z

Works a treat! Thanks Nicolas! ;)

This was referenced Oct 1, 2021

Add basic CI #3

Merged

My ideal DX for Rust on AWS Lambda #4

Closed

brainstorm changed the title ~~Build broken on Apple Silicon computers~~ Build broken on Apple Silicon computers (and/or Graviton2 builds) Oct 5, 2021

nmoutschen mentioned this issue Oct 11, 2021

fix: switch to cross for cross-compilation #5

Merged

brainstorm mentioned this issue Oct 12, 2021

Apple Silicon and basic cross compiling support aws/aws-sam-build-images#26

Closed

This was referenced Oct 23, 2021

"include/GFp/check.h:27:11: fatal error: 'assert.h' file not found" on M1 apple silicon computers briansmith/ring#1332

Closed

Cannot use RUST for ARM Graviton2 Lambda awslabs/aws-lambda-rust-runtime#354

Closed

nmoutschen closed this as completed Feb 18, 2022

nmoutschen mentioned this issue Feb 18, 2022

docs: Refer to zigbuild/AL2 awslabs/aws-lambda-rust-runtime#422

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Build broken on Apple Silicon computers (and/or Graviton2 builds) #2

Build broken on Apple Silicon computers (and/or Graviton2 builds) #2

brainstorm commented Oct 1, 2021 •

edited

Loading

nmoutschen commented Oct 22, 2021

nmoutschen commented Oct 22, 2021

brainstorm commented Oct 25, 2021 •

edited

Loading

michaelbrewer commented Nov 8, 2021

brainstorm commented Nov 8, 2021 •

edited

Loading

nmoutschen commented Nov 15, 2021

brainstorm commented Dec 1, 2021

nmoutschen commented Dec 1, 2021

nmoutschen commented Feb 18, 2022

brainstorm commented Apr 4, 2022

Build broken on Apple Silicon computers (and/or Graviton2 builds) #2

Build broken on Apple Silicon computers (and/or Graviton2 builds) #2

Comments

brainstorm commented Oct 1, 2021 • edited Loading

nmoutschen commented Oct 22, 2021

nmoutschen commented Oct 22, 2021

brainstorm commented Oct 25, 2021 • edited Loading

michaelbrewer commented Nov 8, 2021

brainstorm commented Nov 8, 2021 • edited Loading

nmoutschen commented Nov 15, 2021

brainstorm commented Dec 1, 2021

nmoutschen commented Dec 1, 2021

nmoutschen commented Feb 18, 2022

brainstorm commented Apr 4, 2022

brainstorm commented Oct 1, 2021 •

edited

Loading

brainstorm commented Oct 25, 2021 •

edited

Loading

brainstorm commented Nov 8, 2021 •

edited

Loading