Skip to content

PBM-1484 Updated GCS configuration example and options #261

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
wants to merge 2 commits into
base: main
Choose a base branch
from
Open

PBM-1484 Updated GCS configuration example and options #261

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
2 commits
Select commit Hold shift + click to select a range
218fdc7
PBM-1484 Updated GCS configuration example and options
nastena1606 Mar 4, 2025
555e006
Updated GCS credentials and config samples in setup section
nastena1606 Mar 5, 2025
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
11 changes: 5 additions & 6 deletions docs/details/storage-config-example.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -29,15 +29,14 @@ storage:

```yaml
storage:
type: s3
s3:
region: northamerica-northeast1
type: gcs
gcs:
bucket: pbm-testing
chunkSize: 16777216
prefix: pbm/test
endpointUrl: https://storage.googleapis.com
credentials:
access-key-id: <your-access-key-id-here>
secret-access-key: <your-secret-key-here>
clientEmail: <your-google-cloud-project-id-here>
privateKey: <your-private-key-here>
```

### MinIO
Expand Down
12 changes: 6 additions & 6 deletions docs/install/backup-storage.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -30,15 +30,15 @@ Percona Backup for MongoDB needs its own dedicated S3 bucket exclusively for bac

```yaml
storage:
type: s3
s3:
region: northamerica-northeast1
type: gcs
gcs:
bucket: pbm-testing
chunkSize: 16777216
prefix: pbm/test
endpointUrl: https://storage.googleapis.com
credentials:
access-key-id: <your-access-key-id-here>
secret-access-key: <your-secret-key-here>
clientEmail: <your-google-cloud-project-id-here>
privateKey: <your-private-key-here>

```

=== ":material-microsoft-azure: Microsoft Azure Blob Storage"
Expand Down
55 changes: 50 additions & 5 deletions docs/reference/configuration-options.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -50,20 +50,20 @@

The storage provider’s name.

Supported values: `aws`, `gcs`
Supported values: `aws`
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What's the purpose of that configuration if only 1 value is allowed? Should we deprecate it?


### storage.s3.bucket


*Type*: string <br>
*Required*: YES

The name of the storage bucket. See the [AWS Bucket naming rules](https://docs.aws.amazon.com/AmazonS3/latest/dev/BucketRestrictions.html#bucketnamingrules) and [GCS bucket naming guidelines](https://cloud.google.com/storage/docs/naming-buckets#requirements) for bucket name requirements.
The name of the storage bucket. See the [AWS Bucket naming rules](https://docs.aws.amazon.com/AmazonS3/latest/dev/BucketRestrictions.html#bucketnamingrules) for bucket name requirements.

### storage.s3.region

*Type*: string <br>
*Required*: YES (for AWS and GCS)
*Required*: YES (for AWS)

Check warning on line 66 in docs/reference/configuration-options.md

View workflow job for this annotation

GitHub Actions / vale

[vale] docs/reference/configuration-options.md#L66 

[Google.Colons] ': Y' should be in lowercase.
Raw output 
{"message": "[Google.Colons] ': Y' should be in lowercase.", "location": {"path": "docs/reference/configuration-options.md", "range": {"start": {"line": 66, "column": 11}}}, "severity": "WARNING"}

Check notice on line 66 in docs/reference/configuration-options.md

View workflow job for this annotation

GitHub Actions / vale

[vale] docs/reference/configuration-options.md#L66 

[Google.Acronyms] Spell out 'YES', if it's unfamiliar to the audience.
Raw output 
{"message": "[Google.Acronyms] Spell out 'YES', if it's unfamiliar to the audience.", "location": {"path": "docs/reference/configuration-options.md", "range": {"start": {"line": 66, "column": 13}}}, "severity": "INFO"}
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If it's required for AWS, and AWS is the only option - it's just "YES"


The location of the storage bucket.
Use the [AWS region list](https://docs.aws.amazon.com/general/latest/gr/rande.html#s3_region) and [GCS region list](https://cloud.google.com/storage/docs/locations) to define the bucket region
Expand All @@ -78,9 +78,9 @@
### storage.s3.endpointUrl

*Type*: string <br>
*Required*: YES (for MinIO and GCS)
*Required*: YES (for MinIO)

Check warning on line 81 in docs/reference/configuration-options.md

View workflow job for this annotation

GitHub Actions / vale

[vale] docs/reference/configuration-options.md#L81 

[Google.Colons] ': Y' should be in lowercase.
Raw output 
{"message": "[Google.Colons] ': Y' should be in lowercase.", "location": {"path": "docs/reference/configuration-options.md", "range": {"start": {"line": 81, "column": 11}}}, "severity": "WARNING"}

Check notice on line 81 in docs/reference/configuration-options.md

View workflow job for this annotation

GitHub Actions / vale

[vale] docs/reference/configuration-options.md#L81 

[Google.Acronyms] Spell out 'YES', if it's unfamiliar to the audience.
Raw output 
{"message": "[Google.Acronyms] Spell out 'YES', if it's unfamiliar to the audience.", "location": {"path": "docs/reference/configuration-options.md", "range": {"start": {"line": 81, "column": 13}}}, "severity": "INFO"}

The URL to access the bucket. The default value for GCS is `https://storage.googleapis.com`
The URL to access the bucket.

### storage.s3.endpointUrlMap

Expand Down Expand Up @@ -176,6 +176,51 @@

Use this option with caution as it might leave a hole for man-in-the-middle attacks.

## GCS type storage options

Check warning on line 179 in docs/reference/configuration-options.md

View workflow job for this annotation

GitHub Actions / vale

[vale] docs/reference/configuration-options.md#L179 

[Google.Headings] 'GCS type storage options' should use sentence-style capitalization.
Raw output 
{"message": "[Google.Headings] 'GCS type storage options' should use sentence-style capitalization.", "location": {"path": "docs/reference/configuration-options.md", "range": {"start": {"line": 179, "column": 4}}}, "severity": "WARNING"}

Check notice on line 179 in docs/reference/configuration-options.md

View workflow job for this annotation

GitHub Actions / vale

[vale] docs/reference/configuration-options.md#L179 

[Google.Acronyms] Spell out 'GCS', if it's unfamiliar to the audience.
Raw output 
{"message": "[Google.Acronyms] Spell out 'GCS', if it's unfamiliar to the audience.", "location": {"path": "docs/reference/configuration-options.md", "range": {"start": {"line": 179, "column": 4}}}, "severity": "INFO"}
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

A configuration example is appreciated.


### gcs.bucket

Check warning on line 181 in docs/reference/configuration-options.md

View workflow job for this annotation

GitHub Actions / vale

[vale] docs/reference/configuration-options.md#L181 

[Google.Headings] 'gcs.bucket' should use sentence-style capitalization.
Raw output 
{"message": "[Google.Headings] 'gcs.bucket' should use sentence-style capitalization.", "location": {"path": "docs/reference/configuration-options.md", "range": {"start": {"line": 181, "column": 5}}}, "severity": "WARNING"}

*Type*: string <br>
*Required*: YES

Check warning on line 184 in docs/reference/configuration-options.md

View workflow job for this annotation

GitHub Actions / vale

[vale] docs/reference/configuration-options.md#L184 

[Google.Colons] ': Y' should be in lowercase.
Raw output 
{"message": "[Google.Colons] ': Y' should be in lowercase.", "location": {"path": "docs/reference/configuration-options.md", "range": {"start": {"line": 184, "column": 11}}}, "severity": "WARNING"}

Check notice on line 184 in docs/reference/configuration-options.md

View workflow job for this annotation

GitHub Actions / vale

[vale] docs/reference/configuration-options.md#L184 

[Google.Acronyms] Spell out 'YES', if it's unfamiliar to the audience.
Raw output 
{"message": "[Google.Acronyms] Spell out 'YES', if it's unfamiliar to the audience.", "location": {"path": "docs/reference/configuration-options.md", "range": {"start": {"line": 184, "column": 13}}}, "severity": "INFO"}

The name of the storage bucket. See the [GCS bucket naming guidelines](https://cloud.google.com/storage/docs/naming-buckets#requirements) for bucket name requirements.

Check notice on line 186 in docs/reference/configuration-options.md

View workflow job for this annotation

GitHub Actions / vale

[vale] docs/reference/configuration-options.md#L186 

[Google.Acronyms] Spell out 'GCS', if it's unfamiliar to the audience.
Raw output 
{"message": "[Google.Acronyms] Spell out 'GCS', if it's unfamiliar to the audience.", "location": {"path": "docs/reference/configuration-options.md", "range": {"start": {"line": 186, "column": 42}}}, "severity": "INFO"}

### gcs.chunkSize

Check warning on line 188 in docs/reference/configuration-options.md

View workflow job for this annotation

GitHub Actions / vale

[vale] docs/reference/configuration-options.md#L188 

[Google.Headings] 'gcs.chunkSize' should use sentence-style capitalization.
Raw output 
{"message": "[Google.Headings] 'gcs.chunkSize' should use sentence-style capitalization.", "location": {"path": "docs/reference/configuration-options.md", "range": {"start": {"line": 188, "column": 5}}}, "severity": "WARNING"}

*Type*: string <br>
*Required*: NO

Check warning on line 191 in docs/reference/configuration-options.md

View workflow job for this annotation

GitHub Actions / vale

[vale] docs/reference/configuration-options.md#L191 

[Google.Colons] ': N' should be in lowercase.
Raw output 
{"message": "[Google.Colons] ': N' should be in lowercase.", "location": {"path": "docs/reference/configuration-options.md", "range": {"start": {"line": 191, "column": 11}}}, "severity": "WARNING"}

The size of data chunks in bytes to be uploaded to the storage bucket in a single request. Larger data chunks will be split over multiple requests. Default data chunk size is 16MB.

Check notice on line 193 in docs/reference/configuration-options.md

View workflow job for this annotation

GitHub Actions / vale

[vale] docs/reference/configuration-options.md#L193 

[Google.Passive] In general, use active voice instead of passive voice ('be uploaded').
Raw output 
{"message": "[Google.Passive] In general, use active voice instead of passive voice ('be uploaded').", "location": {"path": "docs/reference/configuration-options.md", "range": {"start": {"line": 193, "column": 37}}}, "severity": "INFO"}

Check warning on line 193 in docs/reference/configuration-options.md

View workflow job for this annotation

GitHub Actions / vale

[vale] docs/reference/configuration-options.md#L193 

[Google.Will] Avoid using 'will'.
Raw output 
{"message": "[Google.Will] Avoid using 'will'.", "location": {"path": "docs/reference/configuration-options.md", "range": {"start": {"line": 193, "column": 111}}}, "severity": "WARNING"}

Check notice on line 193 in docs/reference/configuration-options.md

View workflow job for this annotation

GitHub Actions / vale

[vale] docs/reference/configuration-options.md#L193 

[Google.Passive] In general, use active voice instead of passive voice ('be split').
Raw output 
{"message": "[Google.Passive] In general, use active voice instead of passive voice ('be split').", "location": {"path": "docs/reference/configuration-options.md", "range": {"start": {"line": 193, "column": 116}}}, "severity": "INFO"}

Check failure on line 193 in docs/reference/configuration-options.md

View workflow job for this annotation

GitHub Actions / vale

[vale] docs/reference/configuration-options.md#L193 

[Google.Units] Put a nonbreaking space between the number and the unit in '16MB'.
Raw output 
{"message": "[Google.Units] Put a nonbreaking space between the number and the unit in '16MB'.", "location": {"path": "docs/reference/configuration-options.md", "range": {"start": {"line": 193, "column": 176}}}, "severity": "ERROR"}

### gcs.prefix

Check warning on line 195 in docs/reference/configuration-options.md

View workflow job for this annotation

GitHub Actions / vale

[vale] docs/reference/configuration-options.md#L195 

[Google.Headings] 'gcs.prefix' should use sentence-style capitalization.
Raw output 
{"message": "[Google.Headings] 'gcs.prefix' should use sentence-style capitalization.", "location": {"path": "docs/reference/configuration-options.md", "range": {"start": {"line": 195, "column": 5}}}, "severity": "WARNING"}

*Type*: string <br>
*Required*: NO

Check warning on line 198 in docs/reference/configuration-options.md

View workflow job for this annotation

GitHub Actions / vale

[vale] docs/reference/configuration-options.md#L198 

[Google.Colons] ': N' should be in lowercase.
Raw output 
{"message": "[Google.Colons] ': N' should be in lowercase.", "location": {"path": "docs/reference/configuration-options.md", "range": {"start": {"line": 198, "column": 11}}}, "severity": "WARNING"}

The path to the data directory in the bucket. If undefined, backups are stored in the bucket's root directory.

Check notice on line 200 in docs/reference/configuration-options.md

View workflow job for this annotation

GitHub Actions / vale

[vale] docs/reference/configuration-options.md#L200 

[Google.Passive] In general, use active voice instead of passive voice ('are stored').
Raw output 
{"message": "[Google.Passive] In general, use active voice instead of passive voice ('are stored').", "location": {"path": "docs/reference/configuration-options.md", "range": {"start": {"line": 200, "column": 69}}}, "severity": "INFO"}

### gcs.endpointUrl

Check warning on line 202 in docs/reference/configuration-options.md

View workflow job for this annotation

GitHub Actions / vale

[vale] docs/reference/configuration-options.md#L202 

[Google.Headings] 'gcs.endpointUrl' should use sentence-style capitalization.
Raw output 
{"message": "[Google.Headings] 'gcs.endpointUrl' should use sentence-style capitalization.", "location": {"path": "docs/reference/configuration-options.md", "range": {"start": {"line": 202, "column": 5}}}, "severity": "WARNING"}

*Type*: string <br>
*Required*: NO

Check warning on line 205 in docs/reference/configuration-options.md

View workflow job for this annotation

GitHub Actions / vale

[vale] docs/reference/configuration-options.md#L205 

[Google.Colons] ': N' should be in lowercase.
Raw output 
{"message": "[Google.Colons] ': N' should be in lowercase.", "location": {"path": "docs/reference/configuration-options.md", "range": {"start": {"line": 205, "column": 11}}}, "severity": "WARNING"}

The URL to access the bucket. The default value is `https://storage.googleapis.com`.

### gcs.credentials.clientEmail

Check warning on line 209 in docs/reference/configuration-options.md

View workflow job for this annotation

GitHub Actions / vale

[vale] docs/reference/configuration-options.md#L209 

[Google.Headings] 'gcs.credentials.clientEmail' should use sentence-style capitalization.
Raw output 
{"message": "[Google.Headings] 'gcs.credentials.clientEmail' should use sentence-style capitalization.", "location": {"path": "docs/reference/configuration-options.md", "range": {"start": {"line": 209, "column": 5}}}, "severity": "WARNING"}

*Type*: string <br>
*Required*: YES

Check warning on line 212 in docs/reference/configuration-options.md

View workflow job for this annotation

GitHub Actions / vale

[vale] docs/reference/configuration-options.md#L212 

[Google.Colons] ': Y' should be in lowercase.
Raw output 
{"message": "[Google.Colons] ': Y' should be in lowercase.", "location": {"path": "docs/reference/configuration-options.md", "range": {"start": {"line": 212, "column": 11}}}, "severity": "WARNING"}

Check notice on line 212 in docs/reference/configuration-options.md

View workflow job for this annotation

GitHub Actions / vale

[vale] docs/reference/configuration-options.md#L212 

[Google.Acronyms] Spell out 'YES', if it's unfamiliar to the audience.
Raw output 
{"message": "[Google.Acronyms] Spell out 'YES', if it's unfamiliar to the audience.", "location": {"path": "docs/reference/configuration-options.md", "range": {"start": {"line": 212, "column": 13}}}, "severity": "INFO"}

The clientEmail represents the email address associated with the service account. It is used to identify the service account when making API requests to Google Cloud services.

Check failure on line 214 in docs/reference/configuration-options.md

View workflow job for this annotation

GitHub Actions / vale

[vale] docs/reference/configuration-options.md#L214 

[Vale.Spelling] Did you really mean 'clientEmail'?
Raw output 
{"message": "[Vale.Spelling] Did you really mean 'clientEmail'?", "location": {"path": "docs/reference/configuration-options.md", "range": {"start": {"line": 214, "column": 5}}}, "severity": "ERROR"}

Check notice on line 214 in docs/reference/configuration-options.md

View workflow job for this annotation

GitHub Actions / vale

[vale] docs/reference/configuration-options.md#L214 

[Google.Contractions] Use 'it's' instead of 'It is'.
Raw output 
{"message": "[Google.Contractions] Use 'it's' instead of 'It is'.", "location": {"path": "docs/reference/configuration-options.md", "range": {"start": {"line": 214, "column": 83}}}, "severity": "INFO"}

Check notice on line 214 in docs/reference/configuration-options.md

View workflow job for this annotation

GitHub Actions / vale

[vale] docs/reference/configuration-options.md#L214 

[Google.Passive] In general, use active voice instead of passive voice ('is used').
Raw output 
{"message": "[Google.Passive] In general, use active voice instead of passive voice ('is used').", "location": {"path": "docs/reference/configuration-options.md", "range": {"start": {"line": 214, "column": 86}}}, "severity": "INFO"}

Check warning on line 214 in docs/reference/configuration-options.md

View workflow job for this annotation

GitHub Actions / vale

[vale] docs/reference/configuration-options.md#L214 

[Google.WordList] Use 'Google Cloud Platform' or 'GCP' instead of 'Cloud'.
Raw output 
{"message": "[Google.WordList] Use 'Google Cloud Platform' or 'GCP' instead of 'Cloud'.", "location": {"path": "docs/reference/configuration-options.md", "range": {"start": {"line": 214, "column": 161}}}, "severity": "WARNING"}

### gcs.credentials.privateKey

Check warning on line 216 in docs/reference/configuration-options.md

View workflow job for this annotation

GitHub Actions / vale

[vale] docs/reference/configuration-options.md#L216 

[Google.Headings] 'gcs.credentials.privateKey' should use sentence-style capitalization.
Raw output 
{"message": "[Google.Headings] 'gcs.credentials.privateKey' should use sentence-style capitalization.", "location": {"path": "docs/reference/configuration-options.md", "range": {"start": {"line": 216, "column": 5}}}, "severity": "WARNING"}

*Type*: string <br>
*Required*: YES

Check warning on line 219 in docs/reference/configuration-options.md

View workflow job for this annotation

GitHub Actions / vale

[vale] docs/reference/configuration-options.md#L219 

[Google.Colons] ': Y' should be in lowercase.
Raw output 
{"message": "[Google.Colons] ': Y' should be in lowercase.", "location": {"path": "docs/reference/configuration-options.md", "range": {"start": {"line": 219, "column": 11}}}, "severity": "WARNING"}

Check notice on line 219 in docs/reference/configuration-options.md

View workflow job for this annotation

GitHub Actions / vale

[vale] docs/reference/configuration-options.md#L219 

[Google.Acronyms] Spell out 'YES', if it's unfamiliar to the audience.
Raw output 
{"message": "[Google.Acronyms] Spell out 'YES', if it's unfamiliar to the audience.", "location": {"path": "docs/reference/configuration-options.md", "range": {"start": {"line": 219, "column": 13}}}, "severity": "INFO"}

The private key of the service account used to authenticate the request.


## Server-side encryption options

### serverSideEncryption.sseAlgorithm
Expand Down