Can't failback to O_TMPFILE when memfd_create failed

[lifubang]

Description

In some old kernels, we will use O_TMPFILE or mkostemp, but at this time, the runc state dir has not been created yet. So we will got an error like this:
FATA[0000] nsexec[37893]: could not ensure we are a cloned binary: Permission denied WARN[0000] error="waiting for init preliminary setup: EOF" ERRO[0000] runc run failed: unable to start container process: can't copy bootstrap data to pipe: write init-p: broken pipe

Steps to reproduce the issue

1. Remove memfd code: https://github.com/opencontainers/runc/blob/main/libcontainer/nsenter/cloned_binary.c#L358-L378
2. make
3. use runc run an example container

Describe the results you received and expected

The container should be started normally.

What version of runc are you using?

runc version 1.1.0+dev
commit: v1.1.0-680-g0d890ad6-dirty
spec: 1.1.0
go: go1.18.4
libseccomp: 2.5.1

Host OS information

NAME="Ubuntu"
VERSION="20.04.5 LTS (Focal Fossa)"
ID=ubuntu
ID_LIKE=debian
PRETTY_NAME="Ubuntu 20.04.5 LTS"
VERSION_ID="20.04"
HOME_URL="https://www.ubuntu.com/"
SUPPORT_URL="https://help.ubuntu.com/"
BUG_REPORT_URL="https://bugs.launchpad.net/ubuntu/"
PRIVACY_POLICY_URL="https://www.ubuntu.com/legal/terms-and-policies/privacy-policy"
VERSION_CODENAME=focal
UBUNTU_CODENAME=focal

Host kernel information

Linux acmcoder 5.15.0-78-generic #85~20.04.1-Ubuntu SMP Mon Jul 17 09:42:39 UTC 2023 x86_64 x86_64 x86_64 GNU/Linux

[cyphar]

Based on my tests, this is something weird going on on those old kernels (maybe an LSM issue?). If the issue was the lack of O_TMPFILE support, you should get -EISDIR (which we handle -- O_TMPFILE implies O_DIRECTORY internally as an implementation detail). If the issue was the runc state directory not existing (it does exist AFAICS), you would get -ENOENT. runc also should have access to the state directory, because we write to it in runc.

Very strange. I will fire up an Ubuntu VM and figure out what is going on...

[lifubang]

If the issue was the lack of O_TMPFILE support, you should get -EISDIR

Yes, I also feel very strange.
But I add some logs and find out there is no state dir exactly at that time. If I set prefix with the value /tmp, it works normally.

[lifubang]

but at this time, the runc state dir has not been created yet.

Sorry for this description, but it's another silly problem. Because in my vm, the /run dir is mouted with noexec option.
But I think that maybe we still need to fix this situation, because it's some os's default mount option.

[cyphar]

Ah, I see. The problem is that /tmp can also be noexec. Maybe we need to have bindfd as the final fallback... I don't like bindfd because (aside from the performance issues) it doesn't feel like as good of a protection as memfds or O_TMPFILE -- and this was the case with Dirty COW. One mount-based alternative would be to use overlayfs (the benefit being that there's no way to get the lowerdir inode of an overlayfs handle directly) but there are loads of issues with that idea.