Skip to content

image-builder fails with 'OCI runtime create failed' error #1999

Closed
gitpod-io/retired-gatsby-website
#811

Description

@jgallucci32

Describe the bug

The image-builder pod fails to start with the following error message when using Docker CE on RHEL7

Error response from daemon: OCI runtime create failed: container_linux.go:348: starting container process caused "process_linux.go:301: running exec setns process for init caused \"exit status 41\"": unknown

Steps to reproduce

  1. Install Gitpod self-hosted on a Red Hat Enterprise Linux 7 host with Docker CE
  2. Deploy self-hosted install from helm chart

Expected behavior

The pod image-builder should be active.

Additional information

Docker version 19.03.13, build 4484c46d9d
Red Hat Enterprise Linux 7.8 (3.10.0-1127.el7.x86_64)

Example repository

Gitpod Helm Chart v0.4

Activity

  1. jgallucci32 commented on Oct 14, 2020

    @jgallucci32
    ContributorAuthor

    As a workaround, I was able to get the pod to start by removing --userns-remap=default from the entrypoint of the image-builder pod.

  2. csweichel commented on Oct 15, 2020

    @csweichel
    Contributor

    Thanks for reporting this.
    We haven't tested Gitpod Self-Hosted on any Red Hat platform, but it would seem that out of the box they're incompatible with Docker's user namespace remapping feature.

    I'm happy to see you've already found the workaround. Beware: this gives all users of your installation potentially privileged access to your host (albeit wrapped in a couple of linux namespaces). For trying out Gitpod, even for small teams where you trust everyone using it, that's totally fine though.

  3. jgallucci32 commented on Oct 15, 2020

    @jgallucci32
    ContributorAuthor

    @csweichel Thanks for confirming. I am going to attempt to do the following from the RHEL guide which says it is supported since RHEL 7.4 according to this document.

    Add option to kernel

    grubby --args="namespace.unpriv_enable=1" \
       --update-kernel=/boot/vmlinuz-$(uname -r)
    

    Modify sysctl.conf

    echo "user.max_user_namespaces=15076" >> /etc/sysctl.conf
    

    Assign users and groups to be mapped by user namespaces

    echo dockremap:808080:1000 >> /etc/subuid
    echo dockremap:808080:1000 >> /etc/subgid
    

    I will report back if this is successful or not.

  4. jgallucci32 commented on Oct 15, 2020

    @jgallucci32
    ContributorAuthor

    Well that was easy, running the above commands seems to have resolved the issue. I was able to restart the image-builder pod with the --userns-remap=default flag back in and it worked first try. I will keep monitoring for other issues but it appears to be working just the same as it was before.

    Suggest this gets added to the documentation as a prerequisite for using RHEL/CentOS.

  5. akosyakov commented on Oct 16, 2020

    @akosyakov
    Member

    Self hosted docs are open sourced here: https://github.com/gitpod-io/website/tree/master/src/docs/self-hosted If someone would like to make a PR.

  6. jgallucci32 commented on Oct 18, 2020

    @jgallucci32
    ContributorAuthor

    PR created for documentation https://github.com/gitpod-io/website/pull/811

    NOTE: I created a separate page for other settings such as configuring for Rancher Kubernetes. This way the page can expand in the future for other vendor-specific implementations.

  7. stale commented on Mar 17, 2021

    @stale

    This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.

  8. added
    meta: staleThis issue/PR is stale and will be closed soon
    on Mar 17, 2021
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    meta: staleThis issue/PR is stale and will be closed soonself-hostedtype: bugSomething isn't working

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions