feat: Rust cargo lambda workflow

[calavera]

This is a continuation of #174. It updates the Rust workflow to use cargo-lambda, a tool that we've been building to simplify how to compile Rust functions for AWS Lambda. It supports cross-compiling natively, so you don't have to worry about it. The workflow is much more simple now, since it just calls cargo-lambda and copies the resulting artifacts.

I've also updated tests and example projects to work with this new version.
#174 can be closed, since this PR picks up where it left.

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.

[calavera]

@jfuss can you help me figure out what's going on with Appveyor? I think at this point none of the errors I'm seeing are due to the changes in this PR.

[hawflau]

On the Linux Appveyor jobs, it seems all tests passed but the format check failed due to click-8.1.2 breaking black. We can pin click to 8.0.4 as described in this issue

I'm still investigating the Windows Appveyor jobs. Will update later.

Successfully installed appdirs-1.4.4 astroid-2.4.2 attrs-21.4.0 aws-lambda-builders-1.14.0 black-20.8b1 click-8.1.2 coverage-5.3 flake8-3.8.4 iniconfig-1.1.1 isort-5.10.1 lazy-object-proxy-1.4.3 mccabe-0.6.1 mock-4.0.2 mypy-extensions-0.4.3 packaging-21.3 parameterized-0.7.4 pathspec-0.9.0 pluggy-1.0.0 py-1.11.0 pycodestyle-2.6.0 pyelftools-0.28 pyflakes-2.2.0 pylint-2.6.2 pyparsing-3.0.7 pytest-7.1.1 pytest-cov-2.10.1 regex-2022.3.15 six-1.16.0 toml-0.10.2 tomli-2.0.1 typed-ast-1.5.2 typing-extensions-4.1.1 wrapt-1.14.0

if [[ $PYTHON_VERSION = "3"* ]]; then black --check setup.py tests aws_lambda_builders; fi
Traceback (most recent call last):
  File "/home/appveyor/venv3.9.10/bin/black", line 8, in <module>
    sys.exit(patched_main())
  File "/home/appveyor/venv3.9.10/lib/python3.9/site-packages/black/__init__.py", line 6606, in patched_main
    patch_click()
  File "/home/appveyor/venv3.9.10/lib/python3.9/site-packages/black/__init__.py", line 6595, in patch_click
    from click import _unicodefun  # type: ignore
ImportError: cannot import name '_unicodefun' from 'click' (/home/appveyor/venv3.9.10/lib/python3.9/site-packages/click/__init__.py)
Command exited with code 1
Build failed

[mndeveci]

Windows checks have been failing with following error:

"C:\\Users\\appveyor\\.rustup\\toolchains\\stable-x86_64-pc-windows-msvc\\lib\\rustlib\\x86_64-unknown-linux-gnu\\lib\\libgimli-1e3e01ed4f561dc1.rlib" "C:\\Users\\appveyor\\.rustup\\toolchains\\stable-x86_64-pc-windows-msvc\\lib\\rustlib\\x86_64-unknown-linux-gnu\\lib\\librustc_demangle-dd8dddcba7c13fee.rlib" "C:\\Users\\appveyor\\.rustup\\toolchains\\stable-x86_64-pc-windows-msvc\\lib\\rustlib\\x86_64-unknown-linux-gnu\\lib\\libstd_detect-00b13d31c3420656.rlib" "C:\\Users\\appveyor\\.rustup\\toolchains\\stable-x86_64-pc-windows-msvc\\lib\\rustlib\\x86_64-unknown-linux-gnu\\lib\\libhashbrown-e4c8460b81557bd5.rlib" "C:\\Users\\appveyor\\.rustup\\toolchains\\stable-x86_64-pc-windows-msvc\\lib\\rustlib\\x86_64-unknown-linux-gnu\\lib\\librustc_std_workspace_alloc-600d46e5f0005455.rlib" "C:\\Users\\appveyor\\.rustup\\toolchains\\stable-x86_64-pc-windows-msvc\\lib\\rustlib\\x86_64-unknown-linux-gnu\\lib\\libunwind-e4a08a2585b62d37.rlib" "C:\\Users\\appveyor\\.rustup\\toolchains\\stable-x86_64-pc-windows-msvc\\lib\\rustlib\\x86_64-unknown-linux-gnu\\lib\\libcfg_if-fbc21ec567cb9dc7.rlib" "C:\\Users\\appveyor\\.rustup\\toolchains\\stable-x86_64-pc-windows-msvc\\lib\\rustlib\\x86_64-unknown-linux-gnu\\lib\\liblibc-10b3fa8e49bd978f.rlib" "C:\\Users\\appveyor\\.rustup\\toolchains\\stable-x86_64-pc-windows-msvc\\lib\\rustlib\\x86_64-unknown-linux-gnu\\lib\\liballoc-64625b73694ffce7.rlib" "C:\\Users\\appveyor\\.rustup\\toolchains\\stable-x86_64-pc-windows-msvc\\lib\\rustlib\\x86_64-unknown-linux-gnu\\lib\\librustc_std_workspace_core-fb6b78ac543a58ee.rlib" "C:\\Users\\appveyor\\.rustup\\toolchains\\stable-x86_64-pc-windows-msvc\\lib\\rustlib\\x86_64-unknown-linux-gnu\\lib\\libcore-1d1f2d1bec6f51b8.rlib" "-Wl,--end-group" "C:\\Users\\appveyor\\.rustup\\toolchains\\stable-x86_64-pc-windows-msvc\\lib\\rustlib\\x86_64-unknown-linux-gnu\\lib\\libcompiler_builtins-acf5ff6e9595d982.rlib" "-Wl,-Bdynamic" "-lgcc_s" "-lutil" "-lrt" "-lpthread" "-lm" "-ldl" "-lc" "-Wl,--eh-frame-hdr" "-Wl,-znoexecstack" "-L" "C:\\Users\\appveyor\\.rustup\\toolchains\\stable-x86_64-pc-windows-msvc\\lib\\rustlib\\x86_64-unknown-linux-gnu\\lib" "-o" "C:\\projects\\aws-lambda-builders\\tests\\integration\\workflows\\rust_cargo\\testdata\\workspaces\\target\\x86_64-unknown-linux-gnu\\debug\\deps\\foo-e440e43715e67040" "-Wl,--gc-sections" "-pie" "-Wl,-zrelro,-znow" "-nodefaultlibs"
E                 = note: warning: unsupported linker arg: -znoexecstack
E                         warning: unsupported linker arg: -zrelro
E                         warning: unsupported linker arg: -znow
E                         error(link): unable to spawn C:\Users\appveyor\zig-windows-x86_64-0.9.1\zig.exe: Unexpected
E                         error: UnableToSpawnSelf
E                         
E                         C:\projects\aws-lambda-builders\tests\integration\workflows\rust_cargo\testdata\workspaces>C:\Users\appveyor\.cargo\bin\cargo-lambda.exe zig cc -- -target x86_64-linux-gnu -g -Wl,--version-script=C:\Users\appveyor\AppData\Local\cargo-zigbuild\0.8.7\fcntl.map -include C:\Users\appveyor\AppData\Local\cargo-zigbuild\0.8.7\fcntl.h @C:\Users\appveyor\AppData\Local\Temp\1\rustc0vR5ZG\linker-arguments 
E                         
E               
E               error: could not compile `foo` due to previous error
aws_lambda_builders\workflow.py:309: WorkflowFailedError
=========================== short test summary info ===========================
FAILED tests/integration/workflows/rust_cargo/test_rust_cargo.py::TestRustCargo::test_builds_hello_project
FAILED tests/integration/workflows/rust_cargo/test_rust_cargo.py::TestRustCargo::test_builds_hello_project_for_arm64
FAILED tests/integration/workflows/rust_cargo/test_rust_cargo.py::TestRustCargo::test_builds_workspaces_project
============ 3 failed, 80 passed, 6 skipped in 1006.58s (0:16:46) =============

[calavera]

Yup, I'm aware of the error, I just don't know how to reproduce it unfortunately because the tests pass in my windows laptop. I have more information here if you're curious: cargo-lambda/cargo-lambda#77

[mndeveci]

Can you check if you disable (or extend) this maximum path option for windows?
https://www.howtogeek.com/266621/how-to-make-windows-10-accept-file-paths-over-260-characters/

[calavera]

That option is disabled in my windows machine, I've never changed it:

[hawflau]

Converting the PR to draft as it seems it's still work in progress

[mndeveci]

@calavera do you have any updates regarding to this windows failures? I wonder if we can push this change under a feature flag, and noting that Windows might have issues that we are still investigating.

Rust is getting traction from the community, and providing a native builder for SAM CLI would be good for our customers.

[calavera]

@mndeveci I have not been able to reproduce those errors in any other environment unfortunately. I'm all for pushing this. I can make the changes if you tell me how to add a feature flag.

[calavera]

@mndeveci for what is worth, I have integration tests on Windows that run on GitHub Actions in cargo-lambda. Builds work there without this issue:

https://github.com/cargo-lambda/cargo-lambda/runs/7226829676

[mndeveci]

@mndeveci for what is worth, I have integration tests on Windows that run on GitHub Actions in cargo-lambda. Builds work there without this issue:

https://github.com/cargo-lambda/cargo-lambda/runs/7226829676

I also saw that Appveyor has a newer windows image, can you give it a try with that? You need to update .appevyor.yml file with Visual Studio 2022 instead of Visual Studio 2019 there.

[mndeveci]

I saw windows ones succeeded in last try but they also took like couple of seconds. There is also a condition below in same file, you need to update there as well;

for:
  - matrix:
      only:
        - image: Visual Studio 2019

[calavera]

I saw windows ones succeeded in last try

I think that was a missconfiguration, the build took 6 seconds only 🙈

[calavera]

I was finally able to reproduce this on my personal laptop. So at least I can investigate why it's happening.

[mndeveci]

I was finally able to reproduce this on my personal laptop. So at least I can investigate why it's happening.

That is great news! I will be preparing feature flag in SAM CLI so that we can use it with this builder.

[calavera]

@hawflau @mndeveci I messed up the history with a rebase, sorry 🤦 I'm guessing all these commits will be squashed before merging, so it should not affect the develop branch.

[hawflau]

I did some integration tests and found that this doesn't copy the permission bits of the executable. We should use copy2 instead.

[calavera]

Thanks for catching that @hawflau ! I've updated the code to use copy2.

[calavera]

oh my! Thanks for helping get this through @mndeveci @hawflau !

[milesgranger]

Awesome work @calavera (and to the reviewers!) 🎉