-
Notifications
You must be signed in to change notification settings - Fork 156
feat(build): publish lib as a Lambda Layer #884
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Merged
Merged
Changes from all commits
Commits
Show all changes
35 commits
Select commit
Hold shift + click to select a range
dcf724d
Add a cdk app capable of building and publishing Lambda Layer
18ed8c9
add github action for e2e
6c8939e
add local layer construct npm package
76fa1d8
disable fail fast
e063843
fix RUNTIME naming in matrix
5a884ee
fix tests
bc0a318
fix outputs export name
b057127
Add capability to pass package version to layer builder/publisher
583a12f
Add initial doc
e12dc75
Add github workflows
022c517
add gitignore
e24df14
fix install deps
0f50311
pass layer name
01b91a8
make layer public and store details in ssm
e286e8d
fix e2e tests
cdf4e1f
fix context
b03413e
remvove groups for tests
e7a254d
publish layer on release
44f0728
fix account in doc
8401da8
fix test deps
fb99afc
deploy to all region
297907e
fix account number for layer
2ebe192
fix unit tests
5d8ad97
add install for layer deps in pr workflow
e70e5c1
run unit test of layer publisher for supported node versions only
f509c6a
fix node version in doc
a2d1be5
add node 16 to layer e2e
1e58383
fix comments
f8fee27
fix doc
810e0dc
rename layer to TypeScript
d07ccf9
take care of comments
c8465e2
remove layer from doc for now
c2b0f1b
remove pasted commenbt
4a1c436
chore(build): broke up pr workflow & measure package size (#1031)
dreamorosi fa2f59b
Merge branch 'feat/layerPublisher' of github.com:awslabs/aws-lambda-p…
dreamorosi File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,40 @@ | ||
name: Measure packages size | ||
|
||
on: | ||
workflow_dispatch: | ||
inputs: | ||
prNumber: | ||
description: "PR Number" | ||
required: true | ||
|
||
jobs: | ||
measure-utils-sizes: | ||
runs-on: ubuntu-latest | ||
env: | ||
NODE_ENV: dev | ||
PR_NUMBER: ${{ inputs.prNumber }} | ||
steps: | ||
# Since we are manually triggering the workflow the previous checkout has the main branch. In order to checkout the branch/code of the PR | ||
# we need first to use the PR number to retrieve the PR SHA number. This means we need three steps to: checkout the repo, | ||
# run a custom script to get the SHA, and then finally checkout the PR branch | ||
- name: Checkout Repo | ||
uses: actions/checkout@v3 | ||
- name: Extract PR details | ||
id: extract_PR_details | ||
uses: actions/github-script@v6 | ||
with: | ||
script: | | ||
const script = require('.github/scripts/get_pr_info.js'); | ||
await script({github, context, core}); | ||
- name: Checkout PR code | ||
uses: actions/checkout@v3 | ||
with: | ||
ref: ${{ steps.extract_PR_details.outputs.headSHA }} | ||
- name: Packages size report | ||
uses: flochaz/[email protected] | ||
with: | ||
build-command: mkdir dist && npm run package -w packages/logger -w packages/tracer -w packages/metrics -w packages/commons && npm run package-bundle -w packages/logger -w packages/tracer -w packages/metrics -w packages/commons && bash -c "mv ./packages/*/dist/* dist/" && ls dist | ||
dist-directory: /dist | ||
pr-number: ${{ inputs.prNumber }} | ||
env: | ||
GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }} |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -1,9 +1,10 @@ | ||
name: pr-lint-and-test | ||
name: On PR code update | ||
|
||
on: | ||
pull_request: | ||
types: [opened, synchronize] | ||
jobs: | ||
on_push: | ||
run-unit-tests-on-utils: | ||
runs-on: ubuntu-latest | ||
env: | ||
NODE_ENV: dev | ||
|
@@ -12,63 +13,94 @@ jobs: | |
version: [12, 14, 16] | ||
fail-fast: false | ||
steps: | ||
- uses: actions/checkout@v3 | ||
- name: "Use NodeJS" | ||
- name: Checkout code | ||
uses: actions/checkout@v3 | ||
- name: Setup NodeJS | ||
uses: actions/setup-node@v3 | ||
with: | ||
node-version: ${{ matrix.version }} | ||
- name: Install [email protected] | ||
cache: "npm" | ||
- name: Setup npm | ||
run: npm i -g npm@next-8 | ||
- name: "Setup npm" | ||
run: | | ||
npm set "//registry.npmjs.org/:_authToken=${{ secrets.NPM_TOKEN }}" | ||
- name: Install monorepo packages | ||
# This installs all the dependencies of ./packages/* | ||
- name: Cache node modules | ||
id: cache-node-modules | ||
uses: actions/cache@v3 | ||
with: | ||
path: "./node_modules" | ||
# Use the combo between node version, name, and SHA-256 hash of the lock file as cache key so that | ||
# if one of them changes the cache is invalidated/discarded | ||
key: ${{ matrix.version }}-cache-utils-node-modules-${{ hashFiles('./package-lock.json') }} | ||
- name: Install dependencies | ||
# We can skip the install if there was a cache hit | ||
if: steps.cache-node-modules.outputs.cache-hit != 'true' | ||
# See https://github.com/npm/cli/issues/4475 to see why --foreground-scripts | ||
run: npm ci --foreground-scripts | ||
- name: Install CDK example packages | ||
# Since we are not managing the CDK examples with npm workspaces we install | ||
# the dependencies in a separate step | ||
working-directory: ./examples/cdk | ||
run: npm ci | ||
- name: "Setup SAM" | ||
# We use an ad-hoc action so we can specify the SAM CLI version | ||
uses: aws-actions/setup-sam@v2 | ||
- name: Build packages | ||
# If there's a cache hit we still need to manually build the packages | ||
# this would otherwise have been done automatically as a part of the | ||
# postinstall npm hook | ||
if: steps.cache-node-modules.outputs.cache-hit == 'true' | ||
run: | | ||
npm run build -w packages/commons | ||
npm run build -w packages/logger & npm run build -w packages/tracer & npm run build -w packages/metrics | ||
- name: Lint | ||
run: npm run lint -w packages/commons -w packages/logger -w packages/tracer -w packages/metrics | ||
- name: Run unit tests | ||
run: npm t -w packages/commons -w packages/logger -w packages/tracer -w packages/metrics | ||
check-examples: | ||
runs-on: ubuntu-latest | ||
env: | ||
NODE_ENV: dev | ||
strategy: | ||
matrix: | ||
example: ["sam", "cdk"] | ||
fail-fast: false | ||
defaults: | ||
run: | ||
working-directory: examples/${{ matrix.example }} | ||
steps: | ||
- name: Checkout code | ||
uses: actions/checkout@v3 | ||
- name: Setup NodeJS | ||
uses: actions/setup-node@v3 | ||
with: | ||
node-version: 16 | ||
cache: "npm" | ||
- name: Cache node modules | ||
id: cache-node-modules | ||
uses: actions/cache@v3 | ||
with: | ||
version: 1.49.0 | ||
- name: Install SAM example packages | ||
# Since we are not managing the SAM examples with npm workspaces we install | ||
# the dependencies in a separate step | ||
working-directory: ./examples/sam | ||
path: "./examples/${{ matrix.example }}/node_modules" | ||
# Use the combo between example, name, and SHA-256 hash of all example lock files as cache key. | ||
# It's not possible to use the ${{ matrix.example }} key in the hashFiles fn so | ||
# if any of the lock files (wich should be fairly similar anyway) changes the cache is | ||
# invalidated/discarded for all. | ||
key: ${{ matrix.example }}-cache-examples-node-modules-${{ hashFiles('./examples/*/package-lock.json') }} | ||
- name: Install dependencies | ||
run: npm ci | ||
- name: Run lint | ||
run: npm run lerna-lint | ||
- name: Run tests | ||
run: npm run lerna-test | ||
- name: Collate Coverage Reports | ||
if: ${{ github.actor != 'dependabot[bot]' }} | ||
run: | | ||
for d in ./packages/*/ ; do | ||
mkdir -p coverage | ||
if [[ ! -f coverage/lcov.info ]] | ||
then | ||
continue | ||
fi | ||
filename="$d""coverage/lcov.info" | ||
targetSource="SF:""$d""src" | ||
sed "s|SF:src|$targetSource|g" $filename >> coverage/lcov.info | ||
done | ||
- name: Report Coverage | ||
#Dependabot user will only have read-only perms, so don't try to report coverage | ||
if: ${{ github.actor != 'dependabot[bot]' }} | ||
uses: romeovs/[email protected] | ||
run: npm t | ||
check-layer-publisher: | ||
runs-on: ubuntu-latest | ||
env: | ||
NODE_ENV: dev | ||
defaults: | ||
run: | ||
working-directory: layer-publisher | ||
steps: | ||
- name: Checkout code | ||
uses: actions/checkout@v3 | ||
- name: Setup NodeJS | ||
uses: actions/setup-node@v3 | ||
with: | ||
github-token: ${{ secrets.GITHUB_TOKEN }} | ||
lcov-file: ./coverage/lcov.info | ||
- name: Packages size report | ||
uses: flochaz/[email protected] | ||
node-version: 16 | ||
cache: "npm" | ||
- name: Cache node modules | ||
id: cache-node-modules | ||
uses: actions/cache@v3 | ||
with: | ||
build-command: mkdir dist && npm run lerna-package && npm run lerna-package-bundle && bash -c "mv ./packages/*/dist/* dist/" && ls dist | ||
dist-directory: /dist | ||
env: | ||
GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }} | ||
path: "./layer-publisher/node_modules" | ||
# Use the combo between example, name, and SHA-256 hash of the layer-publisher lock files as cache key. | ||
key: cache-layer-publisher-node-modules-${{ hashFiles('./layer-publisher/*/package-lock.json') }} | ||
- name: Install Layer publisher app | ||
run: npm ci |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,81 @@ | ||
name: Deploy layer to all regions | ||
|
||
permissions: | ||
id-token: write | ||
contents: read | ||
|
||
on: | ||
# Manual trigger | ||
workflow_dispatch: | ||
inputs: | ||
latest_published_version: | ||
description: "Latest npm published version to rebuild corresponding layer for, e.g. v1.0.2" | ||
default: "v1.0.2" | ||
required: true | ||
# Automatic trigger after release | ||
workflow_run: | ||
workflows: ["release"] | ||
types: | ||
- completed | ||
|
||
jobs: | ||
# Build layer by running cdk synth in layer-publisher directory and uploading cdk.out for deployment | ||
build-layer: | ||
runs-on: ubuntu-latest | ||
if: ${{ (github.event.workflow_run.conclusion == 'success') || (github.event_name == 'workflow_dispatch') }} | ||
defaults: | ||
run: | ||
working-directory: ./layer-publisher | ||
steps: | ||
- name: checkout | ||
uses: actions/checkout@v3 | ||
with: | ||
fetch-depth: 0 | ||
- name: Setup Node.js | ||
uses: actions/setup-node@v3 | ||
with: | ||
node-version: "16.12" | ||
- name: Set release notes tag | ||
run: | | ||
RELEASE_INPUT=${{ inputs.latest_published_version }} | ||
LATEST_TAG=$(git describe --tag --abbrev=0) | ||
RELEASE_TAG_VERSION=${RELEASE_INPUT:-$LATEST_TAG} | ||
echo "RELEASE_TAG_VERSION=${RELEASE_TAG_VERSION:1}" >> $GITHUB_ENV | ||
ijemmy marked this conversation as resolved.
Show resolved
Hide resolved
|
||
- name: install cdk and deps | ||
run: | | ||
npm install -g [email protected] | ||
cdk --version | ||
- name: install deps | ||
run: | | ||
npm ci | ||
- name: CDK build | ||
run: cdk synth --context PowerToolsPackageVersion=$RELEASE_TAG_VERSION -o cdk.out | ||
- name: zip output | ||
run: zip -r cdk.out.zip cdk.out | ||
- name: Archive CDK artifacts | ||
uses: actions/upload-artifact@v3 | ||
with: | ||
name: cdk-layer-artefact | ||
path: layer-publisher/cdk.out.zip | ||
|
||
# Deploy layer to all regions in beta account | ||
deploy-beta: | ||
needs: | ||
- build-layer | ||
uses: ./.github/workflows/reusable_deploy_layer_stack.yml | ||
with: | ||
stage: "BETA" | ||
artefact-name: "cdk-layer-artefact" | ||
secrets: | ||
target-account-role: ${{ secrets.AWS_LAYERS_BETA_ROLE_ARN }} | ||
|
||
# Deploy layer to all regions in prod account | ||
deploy-prod: | ||
needs: | ||
- deploy-beta | ||
uses: ./.github/workflows/reusable_deploy_layer_stack.yml | ||
with: | ||
stage: "PROD" | ||
artefact-name: "cdk-layer-artefact" | ||
secrets: | ||
target-account-role: ${{ secrets.AWS_LAYERS_PROD_ROLE_ARN }} |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,81 @@ | ||
name: Deploy cdk stack | ||
|
||
permissions: | ||
id-token: write | ||
contents: read | ||
|
||
on: | ||
workflow_call: | ||
inputs: | ||
stage: | ||
required: true | ||
type: string | ||
artefact-name: | ||
required: true | ||
type: string | ||
secrets: | ||
target-account-role: | ||
required: true | ||
|
||
jobs: | ||
deploy-cdk-stack: | ||
runs-on: ubuntu-latest | ||
defaults: | ||
run: | ||
working-directory: ./layer-publisher | ||
strategy: | ||
fail-fast: false | ||
matrix: | ||
region: | ||
[ | ||
"af-south-1", | ||
"eu-central-1", | ||
"us-east-1", | ||
"us-east-2", | ||
"us-west-1", | ||
"us-west-2", | ||
"ap-east-1", | ||
"ap-south-1", | ||
"ap-northeast-1", | ||
"ap-northeast-2", | ||
"ap-southeast-1", | ||
"ap-southeast-2", | ||
"ca-central-1", | ||
"eu-west-1", | ||
"eu-west-2", | ||
"eu-west-3", | ||
"eu-south-1", | ||
"eu-north-1", | ||
"sa-east-1", | ||
"ap-southeast-3", | ||
"ap-northeast-3", | ||
"me-south-1", | ||
] | ||
steps: | ||
- name: checkout | ||
uses: actions/checkout@v3 | ||
- name: aws credentials | ||
uses: aws-actions/configure-aws-credentials@v1 | ||
with: | ||
aws-region: ${{ matrix.region }} | ||
role-to-assume: ${{ secrets.target-account-role }} | ||
- name: Setup Node.js | ||
uses: actions/setup-node@v3 | ||
with: | ||
node-version: "16.12" | ||
- name: install cdk and deps | ||
run: | | ||
npm install -g [email protected] | ||
cdk --version | ||
- name: install deps | ||
run: | | ||
npm ci | ||
- name: Download artifact | ||
uses: actions/download-artifact@v3 | ||
with: | ||
name: ${{ inputs.artefact-name }} | ||
path: layer-publisher | ||
- name: unzip artefact | ||
run: unzip cdk.out.zip | ||
- name: CDK Deploy Layer | ||
run: cdk deploy --app cdk.out --context region=${{ matrix.region }} 'LayerPublisherStack' --require-approval never --verbose | ||
ijemmy marked this conversation as resolved.
Show resolved
Hide resolved
|
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Uh oh!
There was an error while loading. Please reload this page.