pkg/machine: add custom policy.json logic #21765

Luap99 · 2024-02-20T15:00:45Z

see commits

Does this PR introduce a user-facing change?

podman machine init requires a policy.json file to pull images from the container registry. A default policy.json file is located in our repo at pkg/machine/ocipull/policy.json which should be installed for the user. When building podman via Makefile `MACHINE_POLICY_JSON_DIR` should be used to set the directory where the installed policy.json file will be located. This can be relative directory from the podman binary.

openshift-ci · 2024-02-20T15:00:52Z

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: Luap99

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files:

~~OWNERS~~ [Luap99]

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

Luap99 · 2024-02-20T16:01:31Z

So the macos installer change is completely untested and have no idea about the changes there really and just dud what I think seems logical so please someone with a mac test that!

Luap99 · 2024-02-20T16:07:18Z

@n1hility I was unable to figure out how to add this file to the windows installer, can you help me here? We can build podman with MACHINE_POLICY_JSON_DIR=. so we just need to put in in the same dir as the podman.exe or we could create a subdir like config and put the file there.

n1hility · 2024-02-20T16:54:46Z

@n1hility I was unable to figure out how to add this file to the windows installer, can you help me here? We can build podman with MACHINE_POLICY_JSON_DIR=. so we just need to put in in the same dir as the podman.exe or we could create a subdir like config and put the file there.

Sure. If you get the file into the podman-remote zip tasks which are in the main Makefile, that’s the first step. The installer build pulls from those. It likely needs a small update which I can sort out after it’s in those builds.

Luap99 · 2024-02-21T15:07:56Z

OK I think I added it now but I really do not know how to set MACHINE_POLICY_JSON_DIR there? This target seems to be used for other OS'es as well. I have hard time understanding what is and isn't used in the Makefile

ashley-cui

Building the pkginstaller fails with

CGO_ENABLED=0 GOOS=darwin GOARCH=arm64 go build \
		 \
		-ldflags '-X github.com/containers/podman/v5/libpod/define.gitCommit=e40f43f75b0c3352df5e87a66934a773e9cc06ca -X github.com/containers/podman/v5/libpod/define.buildInfo=1708530478 -X github.com/containers/podman/v5/libpod/config._installPrefix=/usr/local -X github.com/containers/podman/v5/libpod/config._etcDir=/etc -X github.com/containers/podman/v5/pkg/systemd/quadlet._binDir=/usr/local/bin -X github.com/containers/podman/v5/pkg/machine/ocipull.DefaultPolicyJSONPath= -X github.com/containers/common/pkg/config.additionalHelperBinariesDir=/opt/podman/bin MACHINE_POLICY_JSON_DIR=/opt/podman/config ' \
		-tags "remote exclude_graphdriver_btrfs btrfs_noversion exclude_graphdriver_devicemapper containers_image_openpgp" \
		-o bin/darwin/podman ./cmd/podman
# github.com/containers/podman/v5/cmd/podman
usage: link [options] main.o
...
make[1]: *** [bin/darwin/podman] Error 1
make: *** [pkginstaller] Error 2

ashley-cui · 2024-02-21T15:52:11Z

Makefile

@@ -119,6 +119,7 @@ LDFLAGS_PODMAN ?= \
 	-X $(LIBPOD)/config._installPrefix=$(PREFIX) \
 	-X $(LIBPOD)/config._etcDir=$(ETCDIR) \
 	-X $(PROJECT)/v5/pkg/systemd/quadlet._binDir=$(BINDIR) \
+	-X $(PROJECT)/v5/pkg/machine/ocipull.DefaultPolicyJSONPath=$(MACHINE_POLICY_JSON_DIR) \


It looks like during the pkginstaller build, $(MACHINE_POLICY_JSON_DIR) isn't passed from package.sh for some reason.

thanks for checking, yes the quoting in package.sh was incorrect please retry

ashley-cui · 2024-02-21T17:25:06Z

One last fix:

diff --git a/contrib/pkginstaller/Makefile b/contrib/pkginstaller/Makefile
index 3b40cebf9..35a86c9a6 100644
--- a/contrib/pkginstaller/Makefile
+++ b/contrib/pkginstaller/Makefile
@@ -36,8 +36,6 @@ packagedir: podman_version package_root Distribution welcome.html
        cp -r scripts $(PACKAGE_DIR)/
        cp -r $(PACKAGE_ROOT) $(PACKAGE_DIR)/
        cp package.sh $(PACKAGE_DIR)/
-       mkdir $(PACKAGE_DIR)/config
-       cp ../../pkg/machine/ocipull/policy.json $(PACKAGE_DIR)/config/policy.json
        cd $(PACKAGE_DIR) && pkgbuild --analyze --root ./root component.plist
        ../../test/version/version > $(PACKAGE_DIR)/VERSION
        echo -n $(ARCH) > $(PACKAGE_DIR)/ARCH
@@ -49,6 +47,8 @@ package_root: clean-pkgroot $(TMP_DOWNLOAD)/gvproxy $(TMP_DOWNLOAD)/vfkit
        cp $(TMP_DOWNLOAD)/gvproxy $(PACKAGE_ROOT)/podman/bin/
        cp $(TMP_DOWNLOAD)/vfkit $(PACKAGE_ROOT)/podman/bin/
        chmod a+x $(PACKAGE_ROOT)/podman/bin/*
+       mkdir $(PACKAGE_ROOT)/podman/config
+       cp ../../pkg/machine/ocipull/policy.json $(PACKAGE_ROOT)/podman/config/policy.json
 
 %: %.in podman_version
        @sed -e 's/__VERSION__/'$(shell ../../test/version/version)'/g' $< >$@

With this, policy.json is installed into /opt/podman/config/policy.json

Luap99 · 2024-02-21T17:35:49Z

Thanks @ashley-cui, I will apply this on the next push.

n1hility · 2024-02-21T19:08:10Z

OK I think I added it now but I really do not know how to set MACHINE_POLICY_JSON_DIR there? This target seems to be used for other OS'es as well. I have hard time understanding what is and isn't used in the Makefile

I think there is 3 ways this could work:

Change the $(SRCBINDIR)/podman$(BINSFX): task (underlying podman-remote) to have a conditional command line append for windows which sets MACHINE_POLICY_JSON_DIR to "." (see podman-remote-release-%.zip and install.remote for examples)
Instead of doing it on the podman-remote binary build this could just be specified in podman-remote-release-%.zip
Instead of relying on the Makefile change the golang code to hardcode this search location only for windows

The advantage of 1 over 2 is that a developer who does "make podman-remote" and then copies over the exe gets the same behavior as the zip build

The advantage of 3 over 1 and 2 is that it's the same no matter how the binary is built (including when the winemake powershell build is used).

Relatedly, If you go with 1 or 2, then then winmake.ps1 would also need an update as a followup to set this

Luap99 · 2024-02-22T15:07:32Z

@n1hility I went with 2, leave it to you to add it to the installer and winmake. I rather not touch more of this right now,

n1hility · 2024-02-22T16:04:34Z

@Luap99 sure SGTM

ashley-cui · 2024-02-22T16:41:04Z

contrib/pkginstaller/Makefile

+	mkdir $(PACKAGE_ROOT)/podman/config
+	cp ../../pkg/machine/ocipull/policy.json $(PACKAGE_ROOT)/podman/config/policy.json


These need to be moved into the package_root target in order to be correctly packaged.

n1hility · 2024-02-24T21:57:48Z

Makefile

 			clean-binaries podman-remote; \
 	else \
-		$(MAKE) $(GOPLAT) podman-remote; \
+		$(MAKE) $(GOPLAT) MACHINE_POLICY_JSON_DIR="." podman-remote; \


Note this is going to apply this setting to all machine zip archives whether darwin/win/linux etc, which is probably not what you want.

One other approach that could make the Makefile approach a bit easier is to move LDFLAGS_PODMAN down to be after the os check area where SRCBINDIR is defined, and use that to also set MACHINE_POLICY_JSON_DIR defaults.

Or you could this code wise and just change the logic in policyPath with a func in _windows.go that ignores the build flag and just uses a hard coded constant.

If you prefer I can also just address this aspect in the follow-up PR change that also updates the installer.

Note this is going to apply this setting to all machine zip archives whether darwin/win/linux etc, which is probably not what you want.

Why not? If you run from the zip we should not assume any hard coded locations so I think that is fine regardless of the OS that they are used on.

I rather not hard code the default into the binary only for windows as this should really left to packagers IMO.

Ah ok fair enough, I assumed you wanted something like ../../conf on linux but I agree no reason it can't all be the same.

There is one remaining issue which is that right now the policy is placed in the root, yet the binary is placed in usr/bin. For this strategy to work with the zips we would need to put the policy in /usr/bin, or alternatively we could do a search order, first check same dir, then fall back to ../..

rhatdan · 2024-02-27T13:59:14Z

@Luap99 @n1hility @ashley-cui This ready to merge?

Luap99 · 2024-02-27T14:04:06Z

From my side yes, it will need follow up changes for the windows installer but this is way beyond me so I cannot do it in this PR.

This PR adds the build option and fixes our mac installer so it should already be a good improvement.

ashley-cui · 2024-02-27T14:12:28Z

I had one unaddressed comment, some lines needed to be moved for proper mac installer fixes.

n1hility · 2024-02-27T14:15:47Z

LGTM just one remaining small issue (See thread above), but I'm fine with us dealing with it in a follow up PR

Luap99 · 2024-02-27T14:29:18Z

I had one unaddressed comment, some lines needed to be moved for proper mac installer fixes.

A sorry looks I screwed that up while rebasing, let me fix it real quick.

The default policy file /etc/containers/policy.json location does not work on windows and for packages that ship a default. Now we search for the policy.json in the following overwrite locations: macos and linux: - ~/.config/containers/policy.json - /etc/containers/policy.json windows: - %APPDATA%\containers\policy.json Also it offers an additional DefaultPolicyJSONPath var that should be overwritten at built time with the path of the file that is shipped by packagers. Thile file is used when none of the overwrite paths exist. [NO NEW TESTS NEEDED] Signed-off-by: Paul Holzinger <[email protected]>

Allow users to set MACHINE_POLICY_JSON_DIR to the policy.json directory which is used for podman machine pulls. Signed-off-by: Paul Holzinger <[email protected]>

So that this file can be inculded in our windows/macos packages and also by other packagers. Right now the default policy is allow everything but we plan to add signing in the future. Signed-off-by: Paul Holzinger <[email protected]>

Include a default policy.json file in the macos package so users do not have to add this manually. Signed-off-by: Paul Holzinger <[email protected]>

This config needs to be included for podman machine pulls to work and set MACHINE_POLICY_JSON_DIR so that the file should be located next to the binary. Signed-off-by: Paul Holzinger <[email protected]>

ashley-cui · 2024-02-27T15:00:07Z

LGTM, TY!

baude · 2024-02-27T15:33:20Z

/lgtm

openshift-ci bot added release-note do-not-merge/work-in-progress Indicates that a PR should not merge because it is a work in progress. labels Feb 20, 2024

openshift-ci bot added the approved Indicates a PR has been approved by an approver from all required OWNERS files. label Feb 20, 2024

github-actions bot added the machine label Feb 20, 2024

baude added the 5.0 label Feb 20, 2024

Luap99 force-pushed the machine-pull-policy branch from bb29f58 to 9fd1831 Compare February 20, 2024 15:59

Luap99 force-pushed the machine-pull-policy branch from 9fd1831 to e40f43f Compare February 21, 2024 14:52

ashley-cui reviewed Feb 21, 2024

View reviewed changes

Luap99 force-pushed the machine-pull-policy branch from e40f43f to 173b0d0 Compare February 21, 2024 16:05

Luap99 force-pushed the machine-pull-policy branch from 173b0d0 to ba3e3c5 Compare February 22, 2024 15:04

Luap99 marked this pull request as ready for review February 22, 2024 15:16

openshift-ci bot removed the do-not-merge/work-in-progress Indicates that a PR should not merge because it is a work in progress. label Feb 22, 2024

ashley-cui reviewed Feb 22, 2024

View reviewed changes

n1hility reviewed Feb 24, 2024

View reviewed changes

This was referenced Feb 26, 2024

Move ignition functions into Containerfiles #21778

Merged

Windows contribution problems #21821

Closed

Luap99 added 5 commits February 27, 2024 15:29

Makefile: add MACHINE_POLICY_JSON_DIR option

f5a2683

Allow users to set MACHINE_POLICY_JSON_DIR to the policy.json directory which is used for podman machine pulls. Signed-off-by: Paul Holzinger <[email protected]>

pkg/machine/ocipull: add default policy.json file

bed6180

So that this file can be inculded in our windows/macos packages and also by other packagers. Right now the default policy is allow everything but we plan to add signing in the future. Signed-off-by: Paul Holzinger <[email protected]>

macos installer: install default policy.json file

1e5b5a8

Include a default policy.json file in the macos package so users do not have to add this manually. Signed-off-by: Paul Holzinger <[email protected]>

Makefile: add machine policy.json to podman-remote.zip

e32c9bb

This config needs to be included for podman machine pulls to work and set MACHINE_POLICY_JSON_DIR so that the file should be located next to the binary. Signed-off-by: Paul Holzinger <[email protected]>

Luap99 force-pushed the machine-pull-policy branch from ba3e3c5 to e32c9bb Compare February 27, 2024 14:31

openshift-ci bot assigned baude Feb 27, 2024

openshift-ci bot added the lgtm Indicates that a PR is ready to be merged. label Feb 27, 2024

openshift-merge-bot bot merged commit 19d3329 into containers:main Feb 27, 2024

Luap99 deleted the machine-pull-policy branch February 27, 2024 15:46

benoitf mentioned this pull request Feb 28, 2024

include default policy.json when doing 'make podman-remote' #21855

Closed

Luap99 mentioned this pull request Feb 28, 2024

Mac machine init fails with Error: obtaining default signature policy: open /etc/containers/policy.json #21677

Closed

stale-locking-app bot added the locked - please file new issue/PR Assist humans wanting to comment on an old issue or PR with locked comments. label May 28, 2024

stale-locking-app bot locked as resolved and limited conversation to collaborators May 28, 2024

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

pkg/machine: add custom policy.json logic #21765

pkg/machine: add custom policy.json logic #21765

Luap99 commented Feb 20, 2024

openshift-ci bot commented Feb 20, 2024

Luap99 commented Feb 20, 2024

Luap99 commented Feb 20, 2024

n1hility commented Feb 20, 2024

Luap99 commented Feb 21, 2024

ashley-cui left a comment

ashley-cui Feb 21, 2024

Luap99 Feb 21, 2024

ashley-cui commented Feb 21, 2024 •

edited

Loading

Luap99 commented Feb 21, 2024

n1hility commented Feb 21, 2024

Luap99 commented Feb 22, 2024

n1hility commented Feb 22, 2024

ashley-cui Feb 22, 2024

n1hility Feb 24, 2024 •

edited

Loading

Luap99 Feb 26, 2024

n1hility Feb 27, 2024 •

edited

Loading

rhatdan commented Feb 27, 2024

Luap99 commented Feb 27, 2024

ashley-cui commented Feb 27, 2024

n1hility commented Feb 27, 2024

Luap99 commented Feb 27, 2024

ashley-cui commented Feb 27, 2024

baude commented Feb 27, 2024

		mkdir $(PACKAGE_ROOT)/podman/config
		cp ../../pkg/machine/ocipull/policy.json $(PACKAGE_ROOT)/podman/config/policy.json

pkg/machine: add custom policy.json logic #21765

pkg/machine: add custom policy.json logic #21765

Conversation

Luap99 commented Feb 20, 2024

Does this PR introduce a user-facing change?

openshift-ci bot commented Feb 20, 2024

Luap99 commented Feb 20, 2024

Luap99 commented Feb 20, 2024

n1hility commented Feb 20, 2024

Luap99 commented Feb 21, 2024

ashley-cui left a comment

Choose a reason for hiding this comment

ashley-cui Feb 21, 2024

Choose a reason for hiding this comment

Luap99 Feb 21, 2024

Choose a reason for hiding this comment

ashley-cui commented Feb 21, 2024 • edited Loading

Luap99 commented Feb 21, 2024

n1hility commented Feb 21, 2024

Luap99 commented Feb 22, 2024

n1hility commented Feb 22, 2024

ashley-cui Feb 22, 2024

Choose a reason for hiding this comment

n1hility Feb 24, 2024 • edited Loading

Choose a reason for hiding this comment

Luap99 Feb 26, 2024

Choose a reason for hiding this comment

n1hility Feb 27, 2024 • edited Loading

Choose a reason for hiding this comment

rhatdan commented Feb 27, 2024

Luap99 commented Feb 27, 2024

ashley-cui commented Feb 27, 2024

n1hility commented Feb 27, 2024

Luap99 commented Feb 27, 2024

ashley-cui commented Feb 27, 2024

baude commented Feb 27, 2024

ashley-cui commented Feb 21, 2024 •

edited

Loading

n1hility Feb 24, 2024 •

edited

Loading

n1hility Feb 27, 2024 •

edited

Loading