e2e: set defaultVolumesToRestic for restic backup, enable restic restore data verification, remove two-phase restore. #790
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
kaovilai
force-pushed
the
e2eDefaultVolumesToRestic
branch
from
91699c6 to
6ef8888
Compare
|
/cherry-pick oadp-1.1 |
|
@kaovilai: once the present PR merges, I will cherry-pick it on top of oadp-1.1 in a new PR and assign it to you. In response to this:
Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository. |
sseago
approved these changes
Aug 16, 2022
kaovilai
changed the title
|
Seeing restic permission issue on aws time="2022-08-16T15:39:31Z" level=error msg="Error backing up item" backup=openshift-adp/mongo-restic-e2e-27b080db-1d79-11ed-ba2a-0a580a812025 error="pod volume backup failed: running Restic backup, stderr=Fatal: unable to open config file: Stat: Access Denied.\nIs there a repository at the following location?\ns3:s3-us-east-1.amazonaws.com/velero-6109f5e9711c8c58131acdd2f490f451/velero-e2e-640f08b6-a42d-4088-87ab-2569b8e15c4c/restic/mongo-persistent\n: exit status 1" error.file="/go/src/github.com/vmware-tanzu/velero/pkg/restic/backupper.go:199" error.function="github.com/vmware-tanzu/velero/pkg/restic.(*backupper).BackupPodVolumes" logSource="pkg/backup/backup.go:417" name=mongo-64fdc89c69-b4t6x
|
|
We need to run the script located at https://github.com/openshift/oadp-operator/blob/master/docs/scripts/dc-restic-post-restore.sh per instruction at #684 |
Codecov Report
@@ Coverage Diff @@
## master #790 +/- ##
==========================================
- Coverage 31.98% 31.68% -0.30%
==========================================
Files 16 16
Lines 3064 3080 +16
==========================================
- Hits 980 976 -4
- Misses 1993 2012 +19
- Partials 91 92 +1
Help us with your feedback. Take ten seconds to tell us how you rate us. Have a feature suggestion? Share it here. |
kaovilai
force-pushed
the
e2eDefaultVolumesToRestic
branch
8 times, most recently
from
f4b3f3e to
ff408e6
Compare
fix restic prebackup verify delete backupFile if it exists Print command output, fix err handling. ignore log `num errors=0` merge fix fix cutset
eliminate custom "credentialsFile", prefering bsl.Credentials trim restore name, run post restore script keep 43 chars for restore name with uuid restore name no longer needs trimming Fix creating restore name detect sha256sum is working else use shasum update checksum_cmd to uppercase and actually use var
kaovilai
force-pushed
the
e2eDefaultVolumesToRestic
branch
from
ff408e6 to
7caf097
Compare
kaovilai
force-pushed
the
e2eDefaultVolumesToRestic
branch
from
8f8b07c to
2419c30
Compare
kaovilai
force-pushed
the
e2eDefaultVolumesToRestic
branch
from
2419c30 to
369af21
Compare
kaovilai
changed the title
|
gcp BSL is different than expected. |
print diff Make nill velero config map equal to empty map.
kaovilai
force-pushed
the
e2eDefaultVolumesToRestic
branch
from
29c399a to
4268d24
Compare
|
Fixed BSL diff check. GCP was erroring due to config being nil compared to empty config. |
kaovilai
changed the title
|
/test 4.7-operator-e2e-azure |
sseago
approved these changes
Aug 17, 2022
|
@kaovilai: all tests passed! Full PR test history. Your PR dashboard. Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository. I understand the commands that are listed here. |
|
@kaovilai: new pull request created: #797 In response to this:
Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository. |
kaovilai
added a commit
to kaovilai/oadp-operator
that referenced
this pull request
Aug 24, 2022
set defaultVolumesToRestic in e2e for restic backup fix restic prebackup verify delete backupFile if it exists Print command output, fix err handling. ignore log `num errors=0` merge fix fix cutset variable renames, DpaCustomResource struct cleanup eliminate custom "credentialsFile", prefering bsl.Credentials trim restore name, run post restore script keep 43 chars for restore name with uuid restore name no longer needs trimming Fix creating restore name detect sha256sum is working else use shasum update checksum_cmd to uppercase and actually use var remove two-phase restore. Use single restore with post restore script. refactor DoesBSLExist, DoesVSLExist print diff Make nill velero config map equal to empty map.
kaovilai
added a commit
to kaovilai/oadp-operator
that referenced
this pull request
Aug 24, 2022
set defaultVolumesToRestic in e2e for restic backup fix restic prebackup verify delete backupFile if it exists Print command output, fix err handling. ignore log `num errors=0` merge fix fix cutset variable renames, DpaCustomResource struct cleanup eliminate custom "credentialsFile", prefering bsl.Credentials trim restore name, run post restore script keep 43 chars for restore name with uuid restore name no longer needs trimming Fix creating restore name detect sha256sum is working else use shasum update checksum_cmd to uppercase and actually use var remove two-phase restore. Use single restore with post restore script. refactor DoesBSLExist, DoesVSLExist print diff Make nill velero config map equal to empty map.
kaovilai
added a commit
to kaovilai/oadp-operator
that referenced
this pull request
Aug 26, 2022
set defaultVolumesToRestic in e2e for restic backup fix restic prebackup verify delete backupFile if it exists Print command output, fix err handling. ignore log `num errors=0` merge fix fix cutset variable renames, DpaCustomResource struct cleanup eliminate custom "credentialsFile", prefering bsl.Credentials trim restore name, run post restore script keep 43 chars for restore name with uuid restore name no longer needs trimming Fix creating restore name detect sha256sum is working else use shasum update checksum_cmd to uppercase and actually use var remove two-phase restore. Use single restore with post restore script. refactor DoesBSLExist, DoesVSLExist print diff Make nill velero config map equal to empty map. cherry-pick of openshift#801 e2e: Increase app build timeouts to 5 minutes Create PVC for CSI before installing app. Add mongo pvc for ibm cloud InstallApplication Update if already exists. Resolves issue if existing is outdated Revert "Create PVC for CSI before installing app." This reverts commit efdb994. e2e: resolve pvc create failure if previous test fail to uninstall app namespace Change backup timeout to 12 minutes Example of test failing because it didn't wait long enough https://prow.ci.openshift.org/view/gs/origin-ci-test/pr-logs/pull/openshift_oadp-operator/801/pull-ci-openshift-oadp-operator-master-4.11-operator-e2e-aws/1561223345897213952#1:build-log.txt%3A7247 Default velero csi snapshot timeout is 10 minutes. Resolve azure 4.8, 4.9 fallback namespace deletion failure due to empty resource name. Revert "cherry-pick of openshift#801" This reverts commit b9bc43e.
weshayutin
pushed a commit
that referenced
this pull request
Aug 26, 2022
…tion, tuning timeouts, reliability (#806) * cherry-pick #790 set defaultVolumesToRestic in e2e for restic backup fix restic prebackup verify delete backupFile if it exists Print command output, fix err handling. ignore log `num errors=0` merge fix fix cutset variable renames, DpaCustomResource struct cleanup eliminate custom "credentialsFile", prefering bsl.Credentials trim restore name, run post restore script keep 43 chars for restore name with uuid restore name no longer needs trimming Fix creating restore name detect sha256sum is working else use shasum update checksum_cmd to uppercase and actually use var remove two-phase restore. Use single restore with post restore script. refactor DoesBSLExist, DoesVSLExist print diff Make nill velero config map equal to empty map. cherry-pick of #801 e2e: Increase app build timeouts to 5 minutes Create PVC for CSI before installing app. Add mongo pvc for ibm cloud InstallApplication Update if already exists. Resolves issue if existing is outdated Revert "Create PVC for CSI before installing app." This reverts commit efdb994. e2e: resolve pvc create failure if previous test fail to uninstall app namespace Change backup timeout to 12 minutes Example of test failing because it didn't wait long enough https://prow.ci.openshift.org/view/gs/origin-ci-test/pr-logs/pull/openshift_oadp-operator/801/pull-ci-openshift-oadp-operator-master-4.11-operator-e2e-aws/1561223345897213952#1:build-log.txt%3A7247 Default velero csi snapshot timeout is 10 minutes. Resolve azure 4.8, 4.9 fallback namespace deletion failure due to empty resource name. Revert "cherry-pick of #801" This reverts commit b9bc43e. * e2e: print container logs on failure only, increase app build timeouts (#801) * e2e: do not print container logs on server version skip * e2e: Increase app build timeouts to 5 minutes * Create PVC for CSI before installing app. * Add mongo pvc for ibm cloud * InstallApplication Update if already exists. Resolves issue if existing is outdated * Revert "Create PVC for CSI before installing app." This reverts commit efdb994. * e2e: resolve pvc create failure if previous test fail to uninstall app namespace * Change backup timeout to 12 minutes Example of test failing because it didn't wait long enough https://prow.ci.openshift.org/view/gs/origin-ci-test/pr-logs/pull/openshift_oadp-operator/801/pull-ci-openshift-oadp-operator-master-4.11-operator-e2e-aws/1561223345897213952#1:build-log.txt%3A7247 Default velero csi snapshot timeout is 10 minutes. * Resolve azure 4.8, 4.9 fallback namespace deletion failure due to empty resource name. * DeepEqual check resource fields before making update call Signed-off-by: Tiger Kaovilai <[email protected]> Signed-off-by: Tiger Kaovilai <[email protected]> Signed-off-by: Tiger Kaovilai <[email protected]>
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Currently restic backup cases are not setting this value and thus we have not actually verified restic backup of pod volumes are working.
This PR enables restic backup and enable restic restore data verification.
Stopped setting
bsl.spec.config["credentialsFile"]to avoid vmware-tanzu/velero#5228