USHIFT-3153: Speed up container image mirroring

[ggiguash]

All tests run on a metal AWS host.
Performance gain ~3m.
Note: the gain may be over 5m on dev virtual machines

Number of images to sync

$ cat _output/test-images/container-images-list | wc -l
64
$ sort -u _output/test-images/container-images-list | wc -l
42

Times before the fix

$ time ./test/bin/mirror_registry.sh
real    3m35.432s
user    1m43.600s
sys     0m27.821s

Times after the switch to mirror

$ time ./test/bin/mirror_registry.sh
real    2m15.528s
user    0m23.288s
sys     0m10.137s

Times after the copy parallelization

real    0m26.411s
user    0m30.755s
sys     0m13.053s

[openshift-ci-robot]

@ggiguash: This pull request references USHIFT-3153 which is a valid jira issue.

Warning: The referenced jira issue has an invalid target version for the target branch this PR targets: expected the story to target the "4.16.0" version, but no target version was set.

In response to this:

All tests run on a metal AWS host.
Performance gain ~3m.
Note: the gain may be over 5m on dev virtual machines

Number of images to sync

$ cat _output/test-images/container-images-list | wc -l
64
$ sort -u _output/test-images/container-images-list | wc -l
42

Times before the fix

$ time ./test/bin/mirror_registry.sh
real    3m35.432s
user    1m43.600s
sys     0m27.821s

Times after the switch to mirror

$ time ./test/bin/mirror_registry.sh
real    2m15.528s
user    0m23.288s
sys     0m10.137s

Times after the copy parallelization

real    0m26.411s
user    0m30.755s
sys     0m13.053s

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the openshift-eng/jira-lifecycle-plugin repository.

[ggiguash]

/assign @pacevedom
/test metal-periodic-test-arm

[pacevedom]

/lgtm

[ggiguash]

Putting on hold to give priority to #3310
/hold

[openshift-ci]

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: ggiguash, pacevedom

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files:

• OWNERS [ggiguash,pacevedom]

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

[ggiguash]

/unhold

[openshift-ci-robot]

/retest-required

Remaining retests: 0 against base HEAD 72ba40c and 2 for PR HEAD f8f5408 in total

[openshift-ci-robot]

/retest-required

Remaining retests: 0 against base HEAD eb65c2c and 1 for PR HEAD f8f5408 in total

[openshift-ci-robot]

/retest-required

Remaining retests: 0 against base HEAD 05bb1ed and 0 for PR HEAD f8f5408 in total

[ggiguash]

The test failures are not related to the changes. Saving on CI cycles
/override ci/prow/metal-periodic-test-arm

[openshift-ci]

@ggiguash: Overrode contexts on behalf of ggiguash: ci/prow/metal-periodic-test-arm

In response to this:

The test failures are not related to the changes. Saving on CI cycles
/override ci/prow/metal-periodic-test-arm

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository.

[openshift-ci-robot]

/hold

Revision f8f5408 was retested 3 times: holding

[ggiguash]

/unhold
/test all

[openshift-ci-robot]

/retest-required

Remaining retests: 0 against base HEAD 05bb1ed and 2 for PR HEAD f8f5408 in total

[openshift-ci-robot]

/retest-required

Remaining retests: 0 against base HEAD 4a3ef2e and 1 for PR HEAD f8f5408 in total

[openshift-ci]

@ggiguash: all tests passed!

Full PR test history. Your PR dashboard.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository. I understand the commands that are listed here.