Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
23 changes: 23 additions & 0 deletions modules/storage-persistent-storage-change-policy-overview.adoc
Original file line number Diff line number Diff line change
@@ -0,0 +1,23 @@
// Module included in the following assemblies:
//
// * storage/understanding-persistent-storage.adoc
//* microshift_storage/understanding-persistent-storage-microshift.adoc

:_mod-docs-content-type: CONCEPT
[id="storage_persistent_storage_change_policy_overview_{context}"]
= Reducing pod timeouts

Certain volume characteristics can cause pod startup delays and timeouts. The following table lists some volume issues that can cause pod timeout with a possible solution.

.Possible pod startup delay issues and solutions
[cols="2",options="header"]
|===
|Issue
|Solution

|Volume with lots of files
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🤖 [error] RedHat.TermsErrors: Use 'many' rather than 'lots of'. For more information, see RedHat.TermsErrors.

|Set `fsGroupChangePolicy` to stop recursively changing ownership and permissions to match the fsGroup specified in a pod’s `securityContext`.

|Using SELinux on large volumes
|Set `seLinuxChangePolicy` to avoid runtime recursively relabeling all files on a volume to match the pod’s SELinux context.
|===
54 changes: 54 additions & 0 deletions modules/storage-persistent-storage-fsGroup-namespace.adoc
Original file line number Diff line number Diff line change
@@ -0,0 +1,54 @@
// Module included in the following assemblies:
//
// * storage/understanding-persistent-storage.adoc
//* microshift_storage/understanding-persistent-storage-microshift.adoc

:_mod-docs-content-type: PROCEDURE
[id="using_fsGroup_namespace_{context}"]
= Changing fsGroup at the namespace level

After applying the desired setting for `fsGroupChangePolicy` at the namespace level, all subsequently created pods, and restarted pods, in that namespace inherit the setting. However, if desired, you can override the inherited `fsGroupChangePolicy` setting for individual pods. Setting `fsGroupChangePolicy` at the pod level overrides inheritance from the namespace level setting for that pod.

.Prerequisites

* Logged in to a running {product-title} cluster with administrator privileges.

* Access to the {product-title} console.

.Procedure

To set `fsGroupChangePolicy` per namespace:

. Select the desired namespace:

.. Click *Administration* > *Namespaces*.

.. On the *Namespaces* page, click the desired namespace. The *Namespace details* page appears.

. Add the `fsGroupChangePolicy` label to the namespace:

.. On the *Namespace details* page, next to *Labels*, click *Edit*.

.. In the *Edit labels* dialog, add the label `storage.openshift.io/fsgroup-change-policy` and set it equal to either:
+
* `OnRootMismatch`: Specifies skipping recursive permission change, thus helping to avoid pod timeout problems.
* `Always`: Specifies always changing permission and ownership of the volume when a volume is mounted.

.. Click *Save*.

.Verification
Start up a pod in the previously edited namespace and observe that the parameter `spec.securityContext.fsGroupChangePolicy` contains the value that you set for the namespace.

.Example pod YAML file showing `fsGroupChangePolicy` setting
[source,yaml]
----
securityContext:
seLinuxOptions:
level: 's0:c27,c24'
runAsNonRoot: true
fsGroup: 1000750000
fsGroupChangePolicy: OnRootMismatch <1>
...
----
<1> This value is inherited from the namespace.

64 changes: 64 additions & 0 deletions modules/storage-persistent-storage-fsGroup-pod.adoc
Original file line number Diff line number Diff line change
@@ -0,0 +1,64 @@
// Module included in the following assemblies:
//
// * storage/understanding-persistent-storage.adoc
//* microshift_storage/understanding-persistent-storage-microshift.adoc

:_mod-docs-content-type: PROCEDURE
[id="using_fsGroup_pod_{context}"]
= Changing fsGroup at the pod level

You can set the set the `fsGroupChangePolicy` parameter in a new or existing deployment, and then the pods that it manages will have this parameter value. You can similarly do this for a statefulset. You cannot edit an existing pod to set `fsGroupChangePolicy`; however, you can set this parameter when creating a new pod.

This procedure describes how to set the `fsGroupChangePolicy` parameter in an existing deployment.

.Prerequisites

* Access to the {product-title} console.

.Procedure

To set the `fsGroupChangePolicy` parameter in an existing deployment:

. Click *Workloads* > *Deployments*.

. On the *Deployment* page, click the desired deployment.

. On the *Deployment details* page, click the *YAML* tab.

. Edit the deployment's YAML file under `spec.template.spec.securityContext` as per the following example file:
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🤖 [error] RedHat.TermsErrors: Use 'according to', 'as', or 'as in' rather than 'as per'. For more information, see RedHat.TermsErrors.

+
.Example deployment YAML file setting `fsGroupChangePolicy`
[source,yaml]
----
...
spec:
replicas: 3
selector:
matchLabels:
app: my-app
template:
metadata:
creationTimestamp: null
labels:
app: my-app
spec:
containers:
- name: container
image: 'image-registry.openshift-image-registry.svc:5000/openshift/httpd:latest'
ports:
- containerPort: 8080
protocol: TCP
resources: {}
terminationMessagePath: /dev/termination-log
terminationMessagePolicy: File
imagePullPolicy: Always
restartPolicy: Always
terminationGracePeriodSeconds: 30
dnsPolicy: ClusterFirst
securityContext:
fsGroupChangePolicy: OnRootMismatch <1>
...
----
<1> `OnRootMismatch` specifies skipping recursive permission change, thus helping to avoid pod timeout problems.

. Click *Save*.
24 changes: 7 additions & 17 deletions modules/storage-persistent-storage-fsGroup.adoc
Original file line number Diff line number Diff line change
Expand Up @@ -3,11 +3,11 @@
// * storage/understanding-persistent-storage.adoc
//* microshift_storage/understanding-persistent-storage-microshift.adoc

:_mod-docs-content-type: CONCEPT
[id="using_fsGroup_overview_{context}"]
= Reducing pod timeouts using fsGroup

[id="using_fsGroup_{context}"]
= Using fsGroup to reduce pod timeouts

If a storage volume contains many files (~1,000,000 or greater), you may experience pod timeouts.
If a storage volume contains many files (for example, a million or more), you might experience pod timeouts.

This can occur because, by default, {product-title} recursively changes ownership and permissions for the contents of each volume to match the `fsGroup` specified in a pod's `securityContext` when that volume is mounted. For large volumes, checking and changing ownership and permissions can be time consuming, slowing pod startup. You can use the `fsGroupChangePolicy` field inside a `securityContext` to control the way that {product-title} checks and manages ownership and permissions for a volume.

Expand All @@ -17,19 +17,9 @@ This can occur because, by default, {product-title} recursively changes ownershi

* `Always`: Always change permission and ownership of the volume when a volume is mounted.

.`fsGroupChangePolicy` example
[source,yaml]
----
securityContext:
runAsUser: 1000
runAsGroup: 3000
fsGroup: 2000
fsGroupChangePolicy: "OnRootMismatch" <1>
...
----
<1> `OnRootMismatch` specifies skipping recursive permission change, thus helping to avoid pod timeout problems.

[NOTE]
====
The fsGroupChangePolicyfield has no effect on ephemeral volume types, such as secret, configMap, and emptydir.
The `fsGroupChangePolicy` field has no effect on ephemeral volume types, such as secret, configMap, and emptydir.
====

You can set `fsGroupChangePolicy` at either the namespace or pod level.
9 changes: 1 addition & 8 deletions modules/storage-persistent-storage-pv.adoc
Original file line number Diff line number Diff line change
Expand Up @@ -129,7 +129,7 @@ The following table lists the access modes:
|ReadWriteOnce
|`RWO`
|The volume can be mounted as read-write by a single node.
|ReadWriteOncePod ^[1]^
|ReadWriteOncePod
|`RWOP`
|The volume can be mounted as read-write by a single pod on a single node.
ifdef::openshift-enterprise,openshift-webscale,openshift-origin[]
Expand All @@ -141,13 +141,6 @@ ifdef::openshift-enterprise,openshift-webscale,openshift-origin[]
|The volume can be mounted as read-write by many nodes.
endif::[]
|===
--
1. RWOP uses the SELinux mount feature. This feature is driver dependent, and enabled by default in AWS EBS
ifndef::openshift-dedicated,openshift-rosa,openshift-rosa-hcp[]
, Azure Disk, GCP PD, IBM Cloud Block Storage volume, Cinder, vSphere,
endif::openshift-dedicated,openshift-rosa,openshift-rosa-hcp[]
and {rh-storage-first}. For third-party drivers, contact your storage vendor.
--

.Supported access modes for persistent volumes
[cols=",^v,^v,^v,^v", width="100%",options="header"]
Expand Down
Original file line number Diff line number Diff line change
@@ -0,0 +1,56 @@
// Module included in the following assemblies:
//
// * storage/understanding-persistent-storage.adoc
//* microshift_storage/understanding-persistent-storage-microshift.adoc

:_mod-docs-content-type: PROCEDURE
[id="using_selinuxChangePolicy_namespace_{context}"]
= Changing seLinuxChangePolicy at the namespace level

After applying the desired setting for `seLinuxChangePolicy` at the namespace level, all subsequently created pods, and restarted pods, in that namespace inherit the setting. However, if desired, you can override the inherited `SELinuxChangePolicy` setting for individual pods. Setting `seLinuxChangePolicy` at the pod level overrides inheritance from the namespace level setting for that pod.

.Prerequisites

* Logged in to a running {product-title} cluster with administrator privileges.

* Access to the {product-title} console.

.Procedure

To set `SELinuxChangePolicy` per namespace:

. Select the desired namespace:

.. Click *Administration* > *Namespaces*.

.. On the *Namespaces* page, click the desired namespace. The *Namespace details* page appears.

. Add the `seLinuxChangePolicy` label to the namespace:

.. On the *Namespace details* page, next to *Labels*, click *Edit*.

.. In the *Edit labels* dialog, add the label `storage.openshift.io/selinux-change-policy` and set it equal to either:
+
* `MountOption`: Specifies avoiding recursive relabeling of all files by attempting to mount the volume with the correct SELinux label directly using the -o context mount option, thus helping to avoid pod timeout problems.
* `Recursive`: Specifies recursively relabeling all files on all pod volumes to the appropriate SELinux context.

.. Click *Save*.

.Verification
Start up a pod in the previously edited namespace and observe that the parameter `spec.securityContext.seLinuxChangePolicy` contains the value that you set for the namespace.

.Example pod YAML file showing `seLinuxChangePolicy` setting
[source,yaml]
----
securityContext:
seLinuxOptions:
level: 's0:c27,c19'
runAsNonRoot: true
fsGroup: 1000740000
seccompProfile:
type: RuntimeDefault
seLinuxChangePolicy: MountOption <1>
...
----
<1> This value is inherited from the namespace.

40 changes: 40 additions & 0 deletions modules/storage-persistent-storage-selinuxChangePolicy-pod.adoc
Original file line number Diff line number Diff line change
@@ -0,0 +1,40 @@
// Module included in the following assemblies:
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🤖 [error] OpenShiftAsciiDoc.ModuleContainsContentType: Module is missing the '_mod-docs-content-type' variable.

//
// * storage/understanding-persistent-storage.adoc
//* microshift_storage/understanding-persistent-storage-microshift.adoc


[id="using_selinuxChangePolicy_pod_{context}"]
= Changing seLinuxChangePolicy at the pod level

You can set the set the `seLinuxChangePolicy` parameter in a new or existing deployment, and then the pods that it manages will have this parameter value. You can similarly do this for a statefulset. You cannot edit an existing pod to set `seLinuxChangePolicy`; however, you can set this parameter when creating a new pod.

This procedure describes how to set the `seLinuxChangePolicy` parameter in an existing deployment.

.Prerequisites

* Access to the {product-title} console.

.Procedure

To set the `seLinuxChangePolicy` parameter in an existing deployment:

. Click *Workloads* > *Deployments*.

. On the *Deployment* page, click the desired deployment.

. On the *Deployment details* page, click the *YAML* tab.

. Edit the deployment's YAML file under `spec.template.spec.securityContext` as per the following example file:
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🤖 [error] RedHat.TermsErrors: Use 'according to', 'as', or 'as in' rather than 'as per'. For more information, see RedHat.TermsErrors.

+
.Example deployment YAML file setting `seLinuxChangePolicy`
[source,yaml]
----
...
securityContext:
seLinuxChangePolicy: MountOption <1>
...
----
<1> `MountOption` specifies skipping recursive relabeling, thus helping to avoid pod timeout problems.

. Click *Save*.
31 changes: 31 additions & 0 deletions modules/storage-persistent-storage-selinuxChangePolicy.adoc
Original file line number Diff line number Diff line change
@@ -0,0 +1,31 @@
// Module included in the following assemblies:
//
// * storage/understanding-persistent-storage.adoc
//* microshift_storage/understanding-persistent-storage-microshift.adoc

:_mod-docs-content-type: CONCEPT
[id="using_selinuxChangePolicy_overview_{context}"]
= Reducing pod timeouts using seLinuxChangePolicy

SELinux (Security-Enhanced Linux) is a security mechanism that assigns security labels (contexts) to all objects (files, processes, network ports, etc.) on a system. These labels determine what a process can access. In {product-title}, SELinux helps prevent containers from escaping and accessing the host system or other containers.

ReadWRiteOnce (RWO), ReadWriteMany (RWX), and ReadWriteOncePod (RWOP) persistent volume (PV) access modes use the SELinux mount feature. This feature is driver dependent, and enabled by default in AWS EBS
ifndef::openshift-dedicated,openshift-rosa,openshift-rosa-hcp[]
, Azure Disk, GCP PD, IBM Cloud Block Storage volume, Cinder, vSphere,
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🤖 [error] OpenShiftAsciiDoc.SuggestAttribute: Use the AsciiDoc attribute '{ibm-cloud-title}' rather than the plain text product term 'IBM Cloud', unless your use case is an exception.

endif::openshift-dedicated,openshift-rosa,openshift-rosa-hcp[]
and {rh-storage-first}. For third-party drivers, contact your storage vendor.

:FeatureName: RWO/RWX SELinux mount
include::snippets/technology-preview.adoc[]

When a pod starts, the container runtime recursively relabels all files on a volume to match the pod's SELinux context. For large volumes, this can significantly increase pod startup times.

The `seLinuxChangePolicy` parameter addresses this performance issue by providing different strategies for applying SELinux labels to volumes.

The `seLinuxChangePolicy` parameter has two possible values:

* `MountOption`: This setting avoids the need for a recursive relabeling of all files by attempting to mount the volume with the correct SELinux label directly using the `-o context` mount option.

* `Recursive`: (Default) This setting recursively relabels all files on all pod volumes to the appropriate SELinux context.

You can set `seLinuxChangePolicy` at either the namespace or pod level.
16 changes: 15 additions & 1 deletion storage/understanding-persistent-storage.adoc
Original file line number Diff line number Diff line change
Expand Up @@ -48,4 +48,18 @@ include::modules/storage-persistent-storage-block-volume.adoc[leveloffset=+1]

include::modules/storage-persistent-storage-block-volume-examples.adoc[leveloffset=+2]

include::modules/storage-persistent-storage-fsGroup.adoc[leveloffset=+1]
include::modules/storage-persistent-storage-change-policy-overview.adoc[leveloffset=+1]

include::modules/storage-persistent-storage-fsGroup.adoc[leveloffset=+2]

include::modules/storage-persistent-storage-fsGroup-namespace.adoc[leveloffset=+3]

include::modules/storage-persistent-storage-fsGroup-pod.adoc[leveloffset=+3]


include::modules/storage-persistent-storage-selinuxChangePolicy.adoc[leveloffset=+2]

include::modules/storage-persistent-storage-selinuxChangePolicy-namespace.adoc[leveloffset=+3]

include::modules/storage-persistent-storage-selinuxChangePolicy-pod.adoc[leveloffset=+3]