Skip to content

Commit 84a6c1f

Browse files
badsyntaxbadsyntax
authored
Merge pull request #24 from badsyntax/sync-strategy
Add configurable sync strategy
2 parents 38ed4ee + 23f88e0 commit 84a6c1f

File tree

17 files changed

+481
-191
lines changed

17 files changed

+481
-191
lines changed

‎.github/workflows/sync-s3.yml renamed to ‎.github/workflows/build-test-deploy.yml

Lines changed: 40 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -1,4 +1,4 @@
1-
name: 'Sync S3'
1+
name: 'Build, Test & Deploy'
22

33
concurrency:
44
group: prod_deploy
@@ -14,12 +14,48 @@ on:
1414
- master
1515

1616
jobs:
17+
test:
18+
name: 'Build & Test'
19+
runs-on: ubuntu-20.04
20+
steps:
21+
- uses: actions/checkout@v2
22+
23+
- name: Set Node.js 16.x
24+
uses: actions/[email protected]
25+
with:
26+
node-version: 16.x
27+
28+
- name: Install dependencies
29+
run: npm ci
30+
31+
- name: Lint
32+
run: npm run lint && npm run format-check
33+
34+
- name: Test
35+
run: npm test
36+
37+
- name: Build
38+
run: npm run build
39+
40+
- name: Package
41+
run: npm run package
42+
43+
- name: Compare the expected and actual dist/ directories
44+
run: |
45+
if [ "$(git diff --ignore-space-at-eol dist/ | wc -l)" -gt "0" ]; then
46+
echo "Detected uncommitted changes after build. See status below:"
47+
git diff
48+
exit 1
49+
fi
50+
1751
deploy:
18-
name: 'Sync'
52+
name: 'Sync to S3'
1953
runs-on: ubuntu-20.04
54+
needs: test
2055
if: github.actor != 'dependabot[bot]' && (github.event_name != 'pull_request' || github.event.pull_request.head.repo.full_name == github.repository)
2156
steps:
2257
- uses: actions/checkout@v2
58+
2359
- name: Configure AWS Credentials
2460
uses: aws-actions/configure-aws-credentials@v1
2561
with:
@@ -43,7 +79,7 @@ jobs:
4379
with:
4480
bucket: ${{ steps.update-stack.outputs.S3BucketName }}
4581
action: 'sync'
46-
src-dir: './out'
82+
src-dir: './test-fixtures'
4783
files-glob: '**/*.html'
4884
aws-region: 'us-east-1'
4985
prefix: 'preview'
@@ -56,7 +92,7 @@ jobs:
5692
with:
5793
bucket: ${{ steps.update-stack.outputs.S3BucketName }}
5894
action: 'sync'
59-
src-dir: './out'
95+
src-dir: './test-fixtures'
6096
files-glob: 'css/**'
6197
aws-region: 'us-east-1'
6298
prefix: 'preview'

‎.github/workflows/codeql-analysis.yml

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -1,7 +1,7 @@
11
# For most projects, this workflow file will not need changing; you simply need
22
# to commit it to your repository.
33
#
4-
# You may wish to alter this file to override the set of languages analyzed,
4+
# You may wish to alter this file to override the set of languages analysed,
55
# or to provide custom queries or build logic.
66
#
77
# ******** NOTE ********

‎.github/workflows/test.yml

Lines changed: 0 additions & 48 deletions
This file was deleted.

‎.vscode/settings.json

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -27,8 +27,10 @@
2727
"files.trimTrailingWhitespace": true,
2828
"cSpell.language": "en-GB",
2929
"cSpell.words": [
30+
"Autobuild",
3031
"badsyntax",
3132
"Behavior",
33+
"codeql",
3234
"color",
3335
"etag",
3436
"globber",

‎README.md

Lines changed: 29 additions & 27 deletions
Original file line numberDiff line numberDiff line change
@@ -1,14 +1,13 @@
11
# AWS S3 GitHub Action
22

3-
[![Build & Test](https://github.com/badsyntax/github-action-aws-s3/actions/workflows/test.yml/badge.svg?branch=master)](https://github.com/badsyntax/github-action-aws-s3/actions/workflows/test.yml)
4-
[![Sync S3](https://github.com/badsyntax/github-action-aws-s3/actions/workflows/sync-s3.yml/badge.svg?branch=master)](https://github.com/badsyntax/github-action-aws-s3/actions/workflows/sync-s3.yml)
3+
[![Build, Test & Deploy](https://github.com/badsyntax/github-action-aws-s3/actions/workflows/build-test-deploy.yml/badge.svg?branch=master)](https://github.com/badsyntax/github-action-aws-s3/actions/workflows/build-test-deploy.yml)
54
[![CodeQL](https://github.com/badsyntax/github-action-aws-s3/actions/workflows/codeql-analysis.yml/badge.svg?branch=master)](https://github.com/badsyntax/github-action-aws-s3/actions/workflows/codeql-analysis.yml)
65

76
A GitHub Action to sync files to S3.
87

98
## Features
109

11-
- Sync based on contents hash (includes accurate ETAG comparisons, even for multipart uploads)
10+
- Configurable sync strategy (with accurate ETAG comparisons, even for multipart uploads)
1211
- Parallel uploads with configurable concurrency & multipart chunk sizes
1312
- Bucket prefixes
1413
- Clean an object path (remove a "directory")
@@ -55,11 +54,15 @@ jobs:
5554
id: sync-html-s3
5655
with:
5756
bucket: ${{ steps.update-stack.outputs.S3BucketName }}
57+
aws-region: 'us-east-1'
5858
action: 'sync' # sync|clean
5959
src-dir: './out' # required only if action is sync
6060
files-glob: '**/*.html' # required only if action is sync
61-
aws-region: 'us-east-1'
6261
prefix: 'preview'
62+
sync-strategy: |
63+
ETag
64+
Content-Type
65+
Cache-Control
6366
strip-extension-glob: '**/**.html'
6467
cache-control: 'public,max-age=0,s-maxage=31536000,must-revalidate'
6568

@@ -73,27 +76,34 @@ jobs:
7376
7477
## Action Inputs
7578
76-
| Name | Description | Example |
77-
| ----------------------------------- | --------------------------------------------------------------------------------------------------------------------------------------------------------- | ----------------------------------- |
78-
| `bucket` | The name of the S3 bucket | `example-bucket-us-east-1` |
79-
| `action` | The action to perform. Accepted values: `sync` or `clean` | `sync` |
80-
| `src-dir` | Source directory of local files to sync (if using the sync action) | `./src` |
81-
| `files-glob` | Glob pattern for source files to sync to S3 (if using the sync action) | `**/*.html` |
82-
| `aws-region` | The AWS region | `us-east-1` |
83-
| `cache-control` | Cache-control headers | `public,max-age=31536000,immutable` |
84-
| `prefix` (optional) | The prefix for the uploaded object | `custom/folder` |
85-
| `strip-extension-glob` (optional) | Glob pattern to strip extension (if using the sync action) | `**/**.html` |
86-
| `acl` (optional) | Access control list (options: `authenticated-read, aws-exec-read, bucket-owner-full-control, bucket-owner-read, private, public-read, public-read-write`) | `private` |
87-
| `multipart-file-size-mb` (optional) | The minimum file size, in megabytes, for which to upload files using multipart. The default is `100` | `100` |
88-
| `multipart-chunk-bytes` (optional) | The chunk size, in bytes, to upload multipart file parts in. The default is `10485760` (10MB) | `10485760` |
89-
| `concurrency` (optional) | How many processes to perform at once. The default is `6` | `6` |
79+
| Name | Description | Example |
80+
| ----------------------------------- | -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | --------------------------------------------- |
81+
| `bucket` | The name of the S3 bucket | `example-bucket-us-east-1` |
82+
| `aws-region` | The AWS region | `us-east-1` |
83+
| `action` | The action to perform. Accepted values: `sync` or `clean` | `sync` |
84+
| `src-dir` | Source directory of local files to sync (if using the sync action) | `./src` |
85+
| `files-glob` | Glob pattern for source files to sync to S3 (if using the sync action) | `**/*.html` |
86+
| `prefix` (optional) | The prefix for the uploaded object | `custom/folder` |
87+
| `cache-control` | Cache-control header | `public,max-age=31536000,immutable` |
88+
| `sync-strategy` (optional) | A newline-separated list of criteria to define the sync strategy. Criteria values: `ETag`, `ContentType`, `CacheControl`, `LastModified`, `ContentLength`.<br/>**PLEASE NOTE** `ETag` cannot be used if your bucket is encrypted | `ETag`<br/>`Content-Type`<br/>`Cache-Control` |
89+
| `strip-extension-glob` (optional) | Glob pattern to strip extension (if using the sync action) | `**/**.html` |
90+
| `acl` (optional) | Access control list (options: `authenticated-read, aws-exec-read, bucket-owner-full-control, bucket-owner-read, private, public-read, public-read-write`) | `private` |
91+
| `multipart-file-size-mb` (optional) | The minimum file size, in megabytes, for which to upload files using multipart. The default is `100` | `100` |
92+
| `multipart-chunk-bytes` (optional) | The chunk size, in bytes, to upload multipart file parts in. The default is `10485760` (10MB) | `10485760` |
93+
| `concurrency` (optional) | How many processes to perform at once. The default is `6` | `6` |
9094

9195
## Action Outputs
9296

9397
| Name | Description | Example |
9498
| --------------- | ------------------------------------------------------------------------- | ------------------------- |
9599
| `modified-keys` | A comma separated list of modified object keys (either synced or removed) | `file1,folder1/file2.ext` |
96100

101+
## Debugging
102+
103+
Check the Action output for logs.
104+
105+
If you need to see more verbose logs you can set `ACTIONS_STEP_DEBUG` to `true` as an Action Secret.
106+
97107
## Related Projects
98108

99109
- [badsyntax/github-action-aws-cloudfront](https://github.com/badsyntax/github-action-aws-cloudfront)
@@ -102,15 +112,7 @@ jobs:
102112

103113
## Motivation
104114

105-
The `aws cli` syncs files based on file modified times or file size. This approach is not ideal when syncing in CI or when build hashes might change but file size is unchanged.
106-
107-
This Action compares the md5 hash against the uploaded file, and if there's a match it will not sync the file.
108-
109-
## Debugging
110-
111-
Check the Action output for logs.
112-
113-
If you need to see more verbose logs you can set `ACTIONS_STEP_DEBUG` to `true` as an Action Secret.
115+
The [`aws s3 sync`](https://awscli.amazonaws.com/v2/documentation/api/latest/reference/s3/sync.html) cli command syncs files based on modified times or file size, but this approach is not appropriate in situations where build hashes might change but file size is unchanged. This action provides a flexible and configuration sync strategy, as well as additional features like stripping file extensions and cleaning a bucket path.
114116

115117
## License
116118

‎action.yml

Lines changed: 8 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -26,14 +26,21 @@ inputs:
2626
require: false
2727
default: ''
2828
description: "The prefix for the uploaded object. For example: 'custom/folder'"
29+
sync-strategy:
30+
required: false
31+
default: |
32+
ETag
33+
ContentType
34+
CacheControl
35+
description: 'A newline-separated list of criteria to define the sync strategy. Criteria include: ETag, ContentType, CacheControl, LastModified, ContentLength'
2936
strip-extension-glob:
3037
require: false
3138
default: ''
3239
description: "Glob pattern to strip extension (f using the sync action). For example: '**/**.html'"
3340
cache-control:
3441
require: false
3542
default: ''
36-
description: 'Cache-control headers. For example: public,max-age=31536000,immutable'
43+
description: 'Cache-control header. For example: public,max-age=31536000,immutable'
3744
acl:
3845
required: false
3946
default: ''

0 commit comments

Comments
 (0)