Skip to content

USHIFT-5202: Optimize embedded image pull when building containers #4324

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Closed
wants to merge 6 commits into from
Closed

USHIFT-5202: Optimize embedded image pull when building containers #4324

wants to merge 6 commits into from

Conversation

ggiguash
Copy link
Contributor

@ggiguash ggiguash commented Dec 16, 2024 •
edited
Loading

Depends on #4318

The following optimizations were implemented:

  • Share host /etc/containers/registries.conf.d during the container build to benefit from local mirror registry configuration
  • Pull images in a loop (single layer) rather than in separate RUN commands (a layer per image)

Note: Also added code to measure shell command elapsed time in Python.

The overall build step times are around 30m now on both x86 and arm platforms. This includes RPM install commands, which are around 10m now.

@openshift-ci-robot openshift-ci-robot added the jira/valid-reference Indicates that this PR references a valid Jira ticket of any type. label Dec 16, 2024
@openshift-ci-robot
Copy link

openshift-ci-robot commented Dec 16, 2024 •
edited by openshift-ci bot
Loading

@ggiguash: This pull request references USHIFT-5202 which is a valid jira issue.

In response to this:

Depends on #4318

The following optimizations were implemented:

  • Share host /etc/containers/registries.conf.d during the container build to benefit from local mirror registry configuration
  • Pull images in a loop (single layer) rather than in separate RUN commands (a layer per image)

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the openshift-eng/jira-lifecycle-plugin repository.

@openshift-ci openshift-ci bot added the do-not-merge/work-in-progress Indicates that a PR should not merge because it is a work in progress. label Dec 16, 2024
@openshift-ci OpenShift CI
Copy link
Contributor

openshift-ci bot commented Dec 16, 2024

Skipping CI for Draft Pull Request.
If you want CI signal for your change, please convert it to an actual PR.
You can still manually trigger a test run with /test all

@ggiguash
Copy link
Contributor Author

/test ?

@openshift-ci OpenShift CI
Copy link
Contributor

openshift-ci bot commented Dec 16, 2024

@ggiguash: The following commands are available to trigger required jobs:

/test e2e-aws-footprint-and-performance

/test e2e-aws-tests

/test e2e-aws-tests-arm

/test e2e-aws-tests-bootc

/test e2e-aws-tests-bootc-arm

/test e2e-aws-tests-bootc-periodic

/test e2e-aws-tests-bootc-periodic-arm

/test e2e-aws-tests-cache

/test e2e-aws-tests-cache-arm

/test e2e-aws-tests-periodic

/test e2e-aws-tests-periodic-arm

/test images

/test ocp-full-conformance-rhel-eus

/test ocp-full-conformance-serial-rhel-eus

/test test-rpm

/test test-unit

/test verify

The following commands are available to trigger optional jobs:

/test security

/test test-rebase

Use /test all to run the following jobs that were automatically triggered:

pull-ci-openshift-microshift-main-e2e-aws-tests

pull-ci-openshift-microshift-main-e2e-aws-tests-arm

pull-ci-openshift-microshift-main-e2e-aws-tests-bootc

pull-ci-openshift-microshift-main-e2e-aws-tests-bootc-arm

pull-ci-openshift-microshift-main-e2e-aws-tests-bootc-periodic

pull-ci-openshift-microshift-main-e2e-aws-tests-bootc-periodic-arm

pull-ci-openshift-microshift-main-e2e-aws-tests-periodic

pull-ci-openshift-microshift-main-e2e-aws-tests-periodic-arm

pull-ci-openshift-microshift-main-images

pull-ci-openshift-microshift-main-security

pull-ci-openshift-microshift-main-test-unit

pull-ci-openshift-microshift-main-verify

In response to this:

/test ?

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository.

@openshift-ci OpenShift CI
Copy link
Contributor

openshift-ci bot commented Dec 16, 2024

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: ggiguash

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files:

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

@openshift-ci openshift-ci bot added the approved Indicates a PR has been approved by an approver from all required OWNERS files. label Dec 16, 2024
@ggiguash
Copy link
Contributor Author

/test e2e-aws-tests-bootc
/test e2e-aws-tests-bootc-arm
/test e2e-aws-tests-bootc-periodic
/test e2e-aws-tests-bootc-periodic-arm

@ggiguash ggiguash force-pushed the optimize_isolated_source_builds branch 2 times, most recently from e33c596 to 7dd4596 Compare December 16, 2024 06:54
@ggiguash ggiguash force-pushed the optimize_isolated_source_builds branch from 7dd4596 to 60e0ad0 Compare December 16, 2024 06:55
@openshift-ci-robot
Copy link

openshift-ci-robot commented Dec 16, 2024 •
edited by openshift-ci bot
Loading

@ggiguash: This pull request references USHIFT-5202 which is a valid jira issue.

In response to this:

Depends on #4318

The following optimizations were implemented:

  • Share host /etc/containers/registries.conf.d during the container build to benefit from local mirror registry configuration
  • Pull images in a loop (single layer) rather than in separate RUN commands (a layer per image)

Note: Also added code to measure shell command elapsed time in Python.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the openshift-eng/jira-lifecycle-plugin repository.

@ggiguash
Copy link
Contributor Author

/test e2e-aws-tests-bootc
/test e2e-aws-tests-bootc-arm
/test e2e-aws-tests-bootc-periodic
/test e2e-aws-tests-bootc-periodic-arm

@openshift-ci-robot
Copy link

openshift-ci-robot commented Dec 16, 2024 •
edited by openshift-ci bot
Loading

@ggiguash: This pull request references USHIFT-5202 which is a valid jira issue.

In response to this:

Depends on #4318

The following optimizations were implemented:

  • Share host /etc/containers/registries.conf.d during the container build to benefit from local mirror registry configuration
  • Pull images in a loop (single layer) rather than in separate RUN commands (a layer per image)

Note: Also added code to measure shell command elapsed time in Python.

The overall build step times are around 30m now on both x86 and arm platforms. This includes RPM install commands, which are around 10m now.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the openshift-eng/jira-lifecycle-plugin repository.

@openshift-ci-robot
Copy link

openshift-ci-robot commented Dec 16, 2024 •
edited by openshift-ci bot
Loading

@ggiguash: This pull request references USHIFT-5202 which is a valid jira issue.

In response to this:

Depends on #4318

The following optimizations were implemented:

  • Share host /etc/containers/registries.conf.d during the container build to benefit from local mirror registry configuration
  • Pull images in a loop (single layer) rather than in separate RUN commands (a layer per image)

Note: Also added code to measure shell command elapsed time in Python.

The overall build step times are around 30m now on both x86 and arm platforms. This includes RPM install commands, which are around 10m now.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the openshift-eng/jira-lifecycle-plugin repository.

@ggiguash
Copy link
Contributor Author

/test e2e-aws-tests-bootc
/test e2e-aws-tests-bootc-arm
/test e2e-aws-tests-bootc-periodic
/test e2e-aws-tests-bootc-periodic-arm

@openshift-ci OpenShift CI
Copy link
Contributor

openshift-ci bot commented Dec 17, 2024

@ggiguash: The following tests failed, say /retest to rerun all failed tests or /retest-required to rerun all mandatory failed tests:

Test name Commit Details Required Rerun command
ci/prow/e2e-aws-tests-bootc-arm 60e0ad0 link true /test e2e-aws-tests-bootc-arm
ci/prow/e2e-aws-tests-bootc-periodic 60e0ad0 link true /test e2e-aws-tests-bootc-periodic
ci/prow/e2e-aws-tests-bootc 60e0ad0 link true /test e2e-aws-tests-bootc
ci/prow/e2e-aws-tests-bootc-periodic-arm 60e0ad0 link true /test e2e-aws-tests-bootc-periodic-arm

Full PR test history. Your PR dashboard.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository. I understand the commands that are listed here.

@ggiguash
Copy link
Contributor Author

/close

Closing this PR because we're getting the following errors when pulling images from the local mirror registry.

rhel95-bootc-source-isolated: 2d29cf7883677e8af733de722fa3c2e41d9d62b56b4e00ab8fab79e783ceae16
rhel95-bootc-source-isolated: Trying to pull registry.redhat.io/lvms4/lvms-rhel9-operator@sha256:bd6dc4d6e90fdbcdb844759e203c9c591abc5ac29a956257a90bda101a37b76e...
rhel95-bootc-source-isolated: Error: copying system image from manifest list: Source image rejected: A signature was required, but no signature exists
rhel95-bootc-source-isolated: Attempt 1 failed. Retrying in 5 seconds...
rhel95-bootc-source-isolated: Trying to pull registry.redhat.io/lvms4/lvms-rhel9-operator@sha256:bd6dc4d6e90fdbcdb844759e203c9c591abc5ac29a956257a90bda101a37b76e...
rhel95-bootc-source-isolated: Error: copying system image from manifest list: Source image rejected: A signature was required, but no signature exists
rhel95-bootc-source-isolated: Attempt 2 failed. Retrying in 5 seconds...
rhel95-bootc-source-isolated: Trying to pull registry.redhat.io/lvms4/lvms-rhel9-operator@sha256:bd6dc4d6e90fdbcdb844759e203c9c591abc5ac29a956257a90bda101a37b76e...
rhel95-bootc-source-isolated: Error: copying system image from manifest list: Source image rejected: A signature was required, but no signature exists
rhel95-bootc-source-isolated: Attempt 3 failed. Retrying in 5 seconds...
rhel95-bootc-source-isolated: Trying to pull quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:d49a8e92e26b4e27adbb1d37cf15a2ea2ae0501054ff9db1a355c80802fdc9ba...

@openshift-ci openshift-ci bot closed this Dec 17, 2024
@openshift-ci OpenShift CI
Copy link
Contributor

openshift-ci bot commented Dec 17, 2024

@ggiguash: Closed this PR.

In response to this:

/close

Closing this PR because we're getting the following errors when pulling images from the local mirror registry.

rhel95-bootc-source-isolated: 2d29cf7883677e8af733de722fa3c2e41d9d62b56b4e00ab8fab79e783ceae16
rhel95-bootc-source-isolated: Trying to pull registry.redhat.io/lvms4/lvms-rhel9-operator@sha256:bd6dc4d6e90fdbcdb844759e203c9c591abc5ac29a956257a90bda101a37b76e...
rhel95-bootc-source-isolated: Error: copying system image from manifest list: Source image rejected: A signature was required, but no signature exists
rhel95-bootc-source-isolated: Attempt 1 failed. Retrying in 5 seconds...
rhel95-bootc-source-isolated: Trying to pull registry.redhat.io/lvms4/lvms-rhel9-operator@sha256:bd6dc4d6e90fdbcdb844759e203c9c591abc5ac29a956257a90bda101a37b76e...
rhel95-bootc-source-isolated: Error: copying system image from manifest list: Source image rejected: A signature was required, but no signature exists
rhel95-bootc-source-isolated: Attempt 2 failed. Retrying in 5 seconds...
rhel95-bootc-source-isolated: Trying to pull registry.redhat.io/lvms4/lvms-rhel9-operator@sha256:bd6dc4d6e90fdbcdb844759e203c9c591abc5ac29a956257a90bda101a37b76e...
rhel95-bootc-source-isolated: Error: copying system image from manifest list: Source image rejected: A signature was required, but no signature exists
rhel95-bootc-source-isolated: Attempt 3 failed. Retrying in 5 seconds...
rhel95-bootc-source-isolated: Trying to pull quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:d49a8e92e26b4e27adbb1d37cf15a2ea2ae0501054ff9db1a355c80802fdc9ba...

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository.

@ggiguash ggiguash deleted the optimize_isolated_source_builds branch December 19, 2024 15:42
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers
No reviews
Assignees
No one assigned
Labels
approved Indicates a PR has been approved by an approver from all required OWNERS files. do-not-merge/work-in-progress Indicates that a PR should not merge because it is a work in progress. jira/valid-reference Indicates that this PR references a valid Jira ticket of any type.
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
2 participants
@ggiguash @openshift-ci-robot