Feature: Provisioning (ix-iocage-plugin) #426

gronke · 2018-07-07T13:10:33Z

closes #1

introduces nested jail config properties
embeds provisioning methods:
- ix for ix-iocage-plugins
- ~~ansible as runner for ansible playbooks stored in a remote git repository~~ will be implemented in another PR

ix-iocage-plugin install stages

When a jail is configured with provisioning.method=ix, the following will happen when running the provisioning command with provisioning.source=jenkins:

Download https://github.com/freenas/iocage-ix-plugins/blob/master/jenkins.json
Read artifacts from jenkins.json
Clone artifacts repo URL to a child dataset of the jail
Start jail with artifacts mounted
Install list of pkgs in jenkinsk.json within the jail
Run post_install.sh from the artifacts folder if it exists

CLI Syntax

ioc create plextemplate \
    provisioning.method=ix \
    provisioning.source=plexmediaserver \
    template=yes

# run provisioner
ioc provision plextemplate

# create jail from the provisioned template
ioc create -t plextemplate plexjail

# configure network and run plexjail
# ...

Update: Providing a temporary network connecting is no longer required. The ix-iocage-plugin now installs packages via iocage.lib.Pkg which does not require internet connections within a jail.

gronke · 2018-07-11T22:32:22Z

The ioc provision option can be set multiple times and allows to temporary override jail configuration properties. When provisioning a template this can be used to leave the configuration untouched while installing packages.

libiocage does not automatically configure the network when provisioning jails or templates because they are potentially untrusted.

skarekrow · 2018-07-12T04:34:10Z

iocage/lib/Provisioning/ix.py

        yield event

+    try:


Why do a try: finally block when any of the try's above the finally statement would end execution. I think you can drop a whole nested level here.

It would, but instead each nested try block would require the same fallback. This top-level try/finally makes sure that the code in finally is executed independently of the success of its internal steps.

skarekrow · 2018-07-12T04:34:59Z

iocage/lib/Provisioning/ix.py

+        yield packageInstallEvent.begin()
+        try:
+            stdout, stderr, returncode = self.jail.exec(
+                ["pkg", "install", "-yield"] + pkg_packages,


pkg install -yield?

Hahahaha, damn you auto-correct. 😆(Have no other explanation)
Of course this must be just -y.

skarekrow · 2018-07-12T04:35:48Z

iocage/lib/Provisioning/ix.py

        zfs.delete_dataset_recursive(dataset)
-    except libzfs.ZFSException:
+    except libzfs.ZFSException as e:


You aren't using e, you can drop that

skarekrow · 2018-07-12T04:36:38Z

iocage/lib/Jail.py

@@ -375,6 +374,7 @@ def provisioner(self) -> 'iocage.lib.Provisioning.prototype.Provisioner':
        except AttributeError:
            pass

+        import iocage.lib.Provisioning


Why do you remove the top level import? Recursion issues?

Yes Python does not like circular dependencies when loading them at import time. And In long term I'd like to reduce the memory footprint and startup time by lazy-loading specific features. There is a relatively small penalty for re-importing dependencies, but this would only happen once per jail if the provisioner is used.

skarekrow · 2018-07-12T04:37:20Z

iocage/cli/provision.py

        yield event
+        if isinstance(event, iocage.lib.events.JailCommandExecution):


This should just be an additional line with \ as there isn't anything else to be done if the if statement below isn't True

I don't get it. Could you please further explain or leave a code snippet?

gronke · 2018-07-14T10:03:00Z

@william-gr we've talked about ix-iocage-plugin support at BSDCan. Finally here is the implementation to install the plugins using the provision command (or Jail method).

@skarekrow you have pointed out some plugin features that were not implemented in this branch. In my opinion another application layer than libiocage should implement UI specific features such as printing administration URLs. My recommendation here is to use custom jail config properties to store the desired port and applying it (or the plugins default) during jail start. At jail start you have such properties stored in the environment variables.

Can you spot any missing install steps besides the UI part, that seems FreeNAS specific to me.

gronke · 2018-07-19T20:27:31Z

This feature can be enhanced after #435 is merged, so that no network is required in jails for package installation. Most of the ix-iocage-plugins will then not require network to get provisioning.

igalic

some comments / output strings are wrong here…

and then there's a few bits that i don't quite understand!

igalic · 2018-07-19T12:09:04Z

iocage/cli/provision.py

+__rootcmd__ = True
+
+
+@click.command(name="start", help="Trigger provisionong of jails.")


typo: provisioning

igalic · 2018-07-19T12:09:20Z

iocage/cli/provision.py

+    "--option", "-o",
+    "temporary_config_override",
+    multiple=True,
+    help="Temporary override jail config options"


temporarily

igalic · 2018-07-19T12:09:42Z

iocage/cli/provision.py

+    jails: typing.Tuple[str, ...],
+    temporary_config_override: typing.Tuple[str, ...]
+) -> None:
+    """Start one or many jails."""


wrong documentation

igalic · 2018-07-19T12:11:07Z

iocage/cli/provision.py

+            failed_jails.append(jail)
+            continue
+
+        changed_jails.append(jail)


can we really tell here if a jail was changed (again)? or if an idempotent operation did, in fact, nothing?

No, at the current time plugins do not signal whether they applied changes or not. We only know that the provisioner ran and did not error.

puppet returns 1 on success, 0 on idempotent success, and 2 on failure

Then we can listen for the return code in the puppet provisioner. The ix-plugins do not differentiate, so that there is no need yet to respond here now.

igalic · 2018-07-19T12:13:42Z

iocage/cli/start.py

+    "--option", "-o",
+    "temporary_config_override",
+    multiple=True,
+    help="Temporary override jail config options"


"temporarily", or else, make it "for jail config options"

igalic · 2018-07-20T09:11:06Z

iocage/lib/Config/Jail/BaseConfig.py

+        key: str,
+        data: typing.Dict[str, typing.Any]
+    ) -> None:
+        if "." in key:


mumbles something about nested json

This is about CLI interaction. The internal storage occurs as nested JSON - just the property may be accessed via dot notation.

igalic · 2018-07-20T09:13:25Z

iocage/lib/Provisioning/ix.py

+        try:
+            if os.path.isfile(f"{plugin_dataset.mountpoint}/post_install.sh"):
+                stdout, stderr, returncode = self.jail.exec(
+                    ["/.ix-plugin/post_install.sh"]


is this file guaranteed to be executable?

It was created by ourselves. Yes, we can assume that this file is existing and executable.

igalic · 2018-07-20T09:13:58Z

iocage/lib/ResourceUpdater.py

@@ -441,6 +441,7 @@ class HardenedBSD(Updater):

    @property
    def _update_command(self) -> typing.List[str]:
+        return ["sleep", "2000"]


😮 Good question. Will be removed!

… provisioner

fixes an issue deleting properties from jails

instead of returning provisioning that points to a dict structure provisioning.source and its siblings are returned

gronke · 2018-07-28T11:13:48Z

This basic implementation of ix-iocage-plugins also lays the groundwork for libiocage provisioning.

As discussed offline with @skarekrow there are changes to the ix-iocage-plugin standard planned aiming for:

Compatibility with different OS versions
Idempotence

Now is the right time to start an open discussion around this topic, so that we end up with a solution that shares interfaces with Ansible and Puppet provisioning.

With support for provisioning of current ix-iocage-plugins and the base implementation for generalised provisioning in this branch is ready to get merged at this point.

gronke added new feature work in progress labels Jul 7, 2018

gronke force-pushed the feature/provisioning branch 13 times, most recently from dfe52be to 60d2e1a Compare July 11, 2018 18:13

gronke requested a review from skarekrow July 11, 2018 18:15

skarekrow suggested changes Jul 12, 2018

View reviewed changes

gronke force-pushed the feature/provisioning branch 2 times, most recently from 1a9271e to c20d993 Compare July 14, 2018 09:33

gronke changed the title ~~Feature: Provisioning~~ Feature: Provisioning (ix-iocage-plugin) Jul 14, 2018

gronke force-pushed the feature/provisioning branch from c20d993 to b406f6b Compare July 14, 2018 11:20

gronke removed the work in progress label Jul 15, 2018

gronke force-pushed the feature/provisioning branch from b406f6b to 001ac8e Compare July 15, 2018 23:00

igalic requested changes Jul 20, 2018

View reviewed changes

gronke mentioned this pull request Jul 20, 2018

Unknown jail config property handling #434

Merged

gronke force-pushed the feature/provisioning branch 2 times, most recently from c8f133d to a630e5c Compare July 22, 2018 06:08

gronke added 26 commits July 27, 2018 22:51

only set +e in start script when ipfw is enabled

63e7ba4

failed fork_exec commands exit with proper return code

eed5328

optional auto-creation of created fstab entries

bc28acc

a jail config name is a synonym for its id

2aaa30e

install ix-iocage-plugin pkgs from remote plugin definition

2c39278

CLI provision -o option overrides jail config temporary

06ad11e

CLI start command -o option temporary overrides a jails configuration

d6ff1e2

encapsulate ix-plugin provisioning in JailCommandExecution events

79add9c

omit duplicate / in relative fstab paths

e6718a5

decode stdout lines in spam log output as UTF-8

61ac7af

take mountpoints for teardown from a jails fstab file

9f872ff

optional postinstall commands run after Pkg.install

51ab16a

allow pkg packages to contain / characters

84c1b33

beautify Provisioning to use yield from

bb2ef08

utilize iocage.lib.Pkg in ix-iocage-plugin provisioning

674766f

enhance error propagation when deleting a dataset in ix-iocage-plugin…

67f0f6b

… provisioner

remove duplicate check for existing fstab lines

a946904

fstab add_line skip_existing and replace toggles

d1922da

refactor Fstab.line_exists method

ca476b0

jails can be instantiated with an existing Fstab object

fd58fdf

leave mountpoint creation up to Fstab class in Pkg

34c369c

call UnknownJailConfigProperty with jail argument

4198a16

fixes an issue deleting properties from jails

return None for unset resource limits

19eccf8

return stringified keys for BaseConfig.all_properties

1301f1b

instead of returning provisioning that points to a dict structure provisioning.source and its siblings are returned

leave jail config key deletion entirely up to BaseConfig

14f3da5

break out _is_known_property in JailConfig

0fa7c34

gronke force-pushed the feature/provisioning branch from 40bb603 to 0fa7c34 Compare July 27, 2018 20:51

gronke merged commit 8e5e924 into master Jul 28, 2018

gronke deleted the feature/provisioning branch July 28, 2018 11:14

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Feature: Provisioning (ix-iocage-plugin) #426

Feature: Provisioning (ix-iocage-plugin) #426

gronke commented Jul 7, 2018 •

edited

Loading

gronke commented Jul 11, 2018

skarekrow Jul 12, 2018

gronke Jul 12, 2018

skarekrow Jul 12, 2018

gronke Jul 12, 2018

skarekrow Jul 12, 2018

skarekrow Jul 12, 2018

gronke Jul 14, 2018

skarekrow Jul 12, 2018

gronke Jul 14, 2018

gronke commented Jul 14, 2018

gronke commented Jul 19, 2018

igalic left a comment

igalic Jul 19, 2018

igalic Jul 19, 2018

igalic Jul 19, 2018

igalic Jul 19, 2018

gronke Jul 22, 2018

igalic Jul 22, 2018

gronke Jul 22, 2018

igalic Jul 19, 2018

igalic Jul 20, 2018

gronke Jul 22, 2018

igalic Jul 20, 2018

gronke Jul 22, 2018

igalic Jul 20, 2018

gronke Jul 22, 2018

gronke commented Jul 28, 2018

		yield event
		if isinstance(event, iocage.lib.events.JailCommandExecution):

		__rootcmd__ = True


		@click.command(name="start", help="Trigger provisionong of jails.")

Feature: Provisioning (ix-iocage-plugin) #426

Feature: Provisioning (ix-iocage-plugin) #426

Conversation

gronke commented Jul 7, 2018 • edited Loading

ix-iocage-plugin install stages

CLI Syntax

gronke commented Jul 11, 2018

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

gronke commented Jul 14, 2018

gronke commented Jul 19, 2018

igalic left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

gronke commented Jul 28, 2018

gronke commented Jul 7, 2018 •

edited

Loading