Skip to content

Commit b115e05

Browse files
badsyntaxbadsyntax
committed
Add configurable sync strategy
1 parent 38ed4ee commit b115e05

File tree

15 files changed

+434
-100
lines changed

15 files changed

+434
-100
lines changed

‎.github/workflows/sync-s3.yml

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -43,7 +43,7 @@ jobs:
4343
with:
4444
bucket: ${{ steps.update-stack.outputs.S3BucketName }}
4545
action: 'sync'
46-
src-dir: './out'
46+
src-dir: './test-fixtures'
4747
files-glob: '**/*.html'
4848
aws-region: 'us-east-1'
4949
prefix: 'preview'
@@ -56,7 +56,7 @@ jobs:
5656
with:
5757
bucket: ${{ steps.update-stack.outputs.S3BucketName }}
5858
action: 'sync'
59-
src-dir: './out'
59+
src-dir: './test-fixtures'
6060
files-glob: 'css/**'
6161
aws-region: 'us-east-1'
6262
prefix: 'preview'

‎.github/workflows/test.yml

Lines changed: 8 additions & 7 deletions
Original file line numberDiff line numberDiff line change
@@ -39,10 +39,11 @@ jobs:
3939
node-version: 16.x
4040
- name: Install dependencies
4141
run: npm ci
42-
- name: Run all
43-
run: |
44-
npm run lint
45-
npm run format-check
46-
npm test
47-
npm run build
48-
npm run package
42+
- name: Lint
43+
run: npm run lint && npm run format-check
44+
- name: Test
45+
run: npm test
46+
- name: Build
47+
run: npm run build
48+
- name: Package
49+
run: npm run package

‎README.md

Lines changed: 28 additions & 25 deletions
Original file line numberDiff line numberDiff line change
@@ -8,7 +8,7 @@ A GitHub Action to sync files to S3.
88

99
## Features
1010

11-
- Sync based on contents hash (includes accurate ETAG comparisons, even for multipart uploads)
11+
- Configurable sync strategy (with accurate ETAG comparisons, even for multipart uploads)
1212
- Parallel uploads with configurable concurrency & multipart chunk sizes
1313
- Bucket prefixes
1414
- Clean an object path (remove a "directory")
@@ -55,11 +55,15 @@ jobs:
5555
id: sync-html-s3
5656
with:
5757
bucket: ${{ steps.update-stack.outputs.S3BucketName }}
58+
aws-region: 'us-east-1'
5859
action: 'sync' # sync|clean
5960
src-dir: './out' # required only if action is sync
6061
files-glob: '**/*.html' # required only if action is sync
61-
aws-region: 'us-east-1'
6262
prefix: 'preview'
63+
sync-strategy: |
64+
ETag
65+
Content-Type
66+
Cache-Control
6367
strip-extension-glob: '**/**.html'
6468
cache-control: 'public,max-age=0,s-maxage=31536000,must-revalidate'
6569

@@ -73,27 +77,34 @@ jobs:
7377
7478
## Action Inputs
7579
76-
| Name | Description | Example |
77-
| ----------------------------------- | --------------------------------------------------------------------------------------------------------------------------------------------------------- | ----------------------------------- |
78-
| `bucket` | The name of the S3 bucket | `example-bucket-us-east-1` |
79-
| `action` | The action to perform. Accepted values: `sync` or `clean` | `sync` |
80-
| `src-dir` | Source directory of local files to sync (if using the sync action) | `./src` |
81-
| `files-glob` | Glob pattern for source files to sync to S3 (if using the sync action) | `**/*.html` |
82-
| `aws-region` | The AWS region | `us-east-1` |
83-
| `cache-control` | Cache-control headers | `public,max-age=31536000,immutable` |
84-
| `prefix` (optional) | The prefix for the uploaded object | `custom/folder` |
85-
| `strip-extension-glob` (optional) | Glob pattern to strip extension (if using the sync action) | `**/**.html` |
86-
| `acl` (optional) | Access control list (options: `authenticated-read, aws-exec-read, bucket-owner-full-control, bucket-owner-read, private, public-read, public-read-write`) | `private` |
87-
| `multipart-file-size-mb` (optional) | The minimum file size, in megabytes, for which to upload files using multipart. The default is `100` | `100` |
88-
| `multipart-chunk-bytes` (optional) | The chunk size, in bytes, to upload multipart file parts in. The default is `10485760` (10MB) | `10485760` |
89-
| `concurrency` (optional) | How many processes to perform at once. The default is `6` | `6` |
80+
| Name | Description | Example |
81+
| ----------------------------------- | -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | --------------------------------------------- |
82+
| `bucket` | The name of the S3 bucket | `example-bucket-us-east-1` |
83+
| `aws-region` | The AWS region | `us-east-1` |
84+
| `action` | The action to perform. Accepted values: `sync` or `clean` | `sync` |
85+
| `src-dir` | Source directory of local files to sync (if using the sync action) | `./src` |
86+
| `files-glob` | Glob pattern for source files to sync to S3 (if using the sync action) | `**/*.html` |
87+
| `prefix` (optional) | The prefix for the uploaded object | `custom/folder` |
88+
| `cache-control` | Cache-control header | `public,max-age=31536000,immutable` |
89+
| `sync-strategy` (optional) | A newline-separated list of criteria to define the sync strategy. Criteria values: `ETag`, `ContentType`, `CacheControl`, `LastModified`, `ContentLength`.<br/>**PLEASE NOTE** `ETag` cannot be used if your bucket is encrypted | `ETag`<br/>`Content-Type`<br/>`Cache-Control` |
90+
| `strip-extension-glob` (optional) | Glob pattern to strip extension (if using the sync action) | `**/**.html` |
91+
| `acl` (optional) | Access control list (options: `authenticated-read, aws-exec-read, bucket-owner-full-control, bucket-owner-read, private, public-read, public-read-write`) | `private` |
92+
| `multipart-file-size-mb` (optional) | The minimum file size, in megabytes, for which to upload files using multipart. The default is `100` | `100` |
93+
| `multipart-chunk-bytes` (optional) | The chunk size, in bytes, to upload multipart file parts in. The default is `10485760` (10MB) | `10485760` |
94+
| `concurrency` (optional) | How many processes to perform at once. The default is `6` | `6` |
9095

9196
## Action Outputs
9297

9398
| Name | Description | Example |
9499
| --------------- | ------------------------------------------------------------------------- | ------------------------- |
95100
| `modified-keys` | A comma separated list of modified object keys (either synced or removed) | `file1,folder1/file2.ext` |
96101

102+
## Debugging
103+
104+
Check the Action output for logs.
105+
106+
If you need to see more verbose logs you can set `ACTIONS_STEP_DEBUG` to `true` as an Action Secret.
107+
97108
## Related Projects
98109

99110
- [badsyntax/github-action-aws-cloudfront](https://github.com/badsyntax/github-action-aws-cloudfront)
@@ -102,15 +113,7 @@ jobs:
102113

103114
## Motivation
104115

105-
The `aws cli` syncs files based on file modified times or file size. This approach is not ideal when syncing in CI or when build hashes might change but file size is unchanged.
106-
107-
This Action compares the md5 hash against the uploaded file, and if there's a match it will not sync the file.
108-
109-
## Debugging
110-
111-
Check the Action output for logs.
112-
113-
If you need to see more verbose logs you can set `ACTIONS_STEP_DEBUG` to `true` as an Action Secret.
116+
The [`aws s3 sync`](https://awscli.amazonaws.com/v2/documentation/api/latest/reference/s3/sync.html) cli command syncs files based on modified times or file size, but this approach is not appropriate in situations where build hashes might change but file size is unchanged. This action provides a flexible and configuration sync strategy, as well as additional features like stripping file extensions and cleaning a bucket path.
114117

115118
## License
116119

‎action.yml

Lines changed: 10 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -26,14 +26,23 @@ inputs:
2626
require: false
2727
default: ''
2828
description: "The prefix for the uploaded object. For example: 'custom/folder'"
29+
sync-strategy:
30+
required: false
31+
default: |
32+
ETag
33+
ContentType
34+
CacheControl
35+
LastModified
36+
ContentLength
37+
description: 'A newline-separated list of criteria to define the sync strategy. Criteria include: ETag, ContentType, CacheControl, LastModified, ContentLength'
2938
strip-extension-glob:
3039
require: false
3140
default: ''
3241
description: "Glob pattern to strip extension (f using the sync action). For example: '**/**.html'"
3342
cache-control:
3443
require: false
3544
default: ''
36-
description: 'Cache-control headers. For example: public,max-age=31536000,immutable'
45+
description: 'Cache-control header. For example: public,max-age=31536000,immutable'
3746
acl:
3847
required: false
3948
default: ''

0 commit comments

Comments
 (0)