Skip to content

Major documentation update #14

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
albatrossflavour merged 2 commits into from
Aug 7, 2018
Merged

Major documentation update #14

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
2 commits
Select commit Hold shift + click to select a range
d022b96
Documentation, changelog and metadata updates prior to release
albatrossflavour Aug 7, 2018
91c0326
Final doco updates
albatrossflavour Aug 7, 2018
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
9 changes: 9 additions & 0 deletions CHANGELOG.md
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,5 +1,14 @@
# Changelog

## Release 0.2.1

- Major rewrite on most areas of the module
- Patching task now written in ruby and has much better reporting and error handling
- Facter is now ruby based but still uses cache data from a cron job
- Blackout window functionality included
- Documentation updates
- SHOULD be backwards compatible

## Release 0.1.19

- Bugfix on the debian task which caused incorrect reporting of patching status
Expand Down
201 changes: 152 additions & 49 deletions README.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -3,19 +3,7 @@

This module contains a set of tasks and custom facts to allow the automation of and reporting on operating system patching, currently restricted to Redhat and Debian derivatives.

## Table of contents

- [os_patching](#ospatching)
- [Table of contents](#table-of-contents)
- [Description](#description)
- [Setup](#setup)
- [What os_patching affects](#what-ospatching-affects)
- [Beginning with os_patching](#beginning-with-ospatching)
- [Usage](#usage)
- [Reference](#reference)
- [Limitations](#limitations)
- [Development](#development)
- [Contributors](#contributors)
Under the hood it uses the OS level tools to carry out the actual patching.

## Description

Expand All @@ -25,56 +13,109 @@ If you're looking for a simple way to report on your OS patch levels, this modul

It also uses security metadata (where available) to determine if there are security updates. On Redhat, this is provided from Redhat as additional metadata in YUM. On Debian, checks are done for which repo the updates are coming from. There is a parameter to the task to only apply security updates.

Blackout windows enable the support for time based change freezes where no patching can happen. There can be multiple windows defined and each which will automatically expire after reaching the defined end date.

## Setup

### What os_patching affects

The module, when added to a node, installs a script to generate the patching facts and a cron job to run the script.
The module provides an additional fact (`os_patching`) and has a task to allow the patching of a server. When the `os_patching` manifest is added to a node it installs a script and cron job to generate cache data used by the `os_patching` fact.

### Beginning with os_patching

Install the module using the Puppetfile, include it on your nodes and then use the provided tasks to carry out patching.

## Usage

### Manifest
Include the module:
```puppet
include os_patching
```

Run a basic patching task from the command line:
```bash
# puppet task show os_patching::patch_server
More advanced usage:
```puppet
class { 'os_patching':
patch_window => 'Week3',
reboot_override => true,
blackout_windows => { 'End of year change freeze':
{
'start': '2018-12-15T00:00:00+1000',
'end': '2019-01-15T23:59:59+1000',
}
},
}
```

In that example, the node is assigned to a "patch window", will be forced to reboot regardless of the setting specified in the task and has a blackout window defined for the period of 2018-12-15 - 2019-01-15, during which time no patching through the task can be carried out.

os_patching::patch_server - Carry out OS patching on the server, optionally including a reboot
### Task
Run a basic patching task from the command line:
```
os_patching::patch_server - Carry out OS patching on the server, optionally including a reboot and/or only applying security related updates

USAGE:
$ puppet task run os_patching::patch_server reboot=<value> security_only=<value> <[--nodes, -n <node-names>] | [--query, -q <'query'>]>
$ puppet task run os_patching::patch_server [dpkg_params=<value>] [reboot=<value>] [security_only=<value>] [timeout=<value>] [yum_params=<value>] <[--nodes, -n <node-names>] | [--query, -q <'query'>]>

PARAMETERS:
- reboot : Boolean
Should the server reboot after patching has been applied?
- security_only : Boolean
Limit patches to those tagged as security related?
- dpkg_params : Optional[String]
Any additional parameters to include in the dpkg command
- reboot : Optional[Boolean]
Should the server reboot after patching has been applied? (Defaults to false)
- security_only : Optional[Boolean]
Limit patches to those tagged as security related? (Defaults to false)
- timeout : Optional[Integer]
How many seconds should we wait until timing out the patch run? (Defaults to 3600 seconds)
- yum_params : Optional[String]
Any additional parameters to include in the yum upgrade command (such as including/excluding repos)
```

Example:
```bash
$ puppet task run os_patching::patch_server reboot=true security_only=false --query="inventory[certname] { facts.os_patching.patch_window = 'Week3' and facts.os_patching.blocked = false and facts.os_patching.package_update_count > 0}"
```

This will run a patching task against all nodes which have facts matching:

* `os_patching.patch_window` of 'Week3'
* `os_patching.blocked` equals `false`
* `os_patching.package_update_count` greater than 0

The task will apply all patches (`security_only=false`) and will reboot the node after patching (`reboot=true`).

## Reference

### Facts

Most of the reporting is driven off the custom fact `os_patching_data`, for example:

```yaml
# facter os_patching
# facter -p os_patching
{
package_update_count => 0
package_update_count => 0,
package_updates => [],
security_package_updates => [],
security_package_update_count => 0,
blocked => false,
blocked_reasons => [],
blackouts => {},
patch_window = 'Week3',
pinned_packages => [],
last_run => {
date => "2018-08-07T21:55:20+10:00",
message => "Patching complete",
return_code => "Success",
post_reboot => "false",
security_only => "false",
job_id => "60"
}
}
```

This shows there are no updates which can be applied to this server. When there are updates to add, you will see similar to this:

```yaml
# facter os_patching
# facter -p os_patching
{
package_update_count => 6,
package_updates => [
Expand All @@ -85,51 +126,113 @@ This shows there are no updates which can be applied to this server. When there
"procps-ng.x86_64",
"python-perf.x86_64"
]
security_package_updates => [],
security_package_update_count => 0,
blocked => false,
blocked_reasons => [],
blackouts => {
Test change freeze 2 => {
start => "2018-08-01T09:17:10+1000",
end => "2018-08-01T11:15:50+1000"
}
},
pinned_packages => [],
patch_window = 'Week3',
last_run => {
date => "2018-08-07T21:55:20+10:00",
message => "Patching complete",
return_code => "Success",
post_reboot => "false",
security_only => "false",
job_id => "60"
}
}
```

Where it shows 6 packages with available updates, along with an array of the package names.
Where it shows 6 packages with available updates, along with an array of the package names. None of the packges are tagged as security related (requires Debian or a subscription to RHEL). There are no blockers to patching and the blackout window defined is not in effect.

The pinned packages entry lists any packages which have been specifically excluded from being patched, from [version lock](https://access.redhat.com/solutions/98873) on Red Hat or by [pinning](https://wiki.debian.org/AptPreferences) in Debian.

Last run shows a summary of the information from the last `os_patching::patch_server` task.

The fact `os_patching.patch_window` can be used to assign nodes to an arbitrary group. The fact can be used as part of the query fed into the task to determine which nodes to patch:

```bash
$ puppet task run os_patching::patch_server --query="inventory[certname] {facts.os_patching.patch_window = 'Week3'}"
```

### Task output

If there is nothing to be done, the task will report:

```puppet
```json
{
"date" : "Mon May 28 12:23:33 AEST 2018",
"fqdn" : "example.lab.vm",
"reboot" : "true",
"message" : "yum dry run shows no patching work to do",
"return-code" : "success",
"securityonly" : "false"
"pinned_packages" : [ ],
"security" : false,
"return" : "Success",
"start_time" : "2018-08-08T07:52:28+10:00",
"debug" : "",
"end_time" : "2018-08-08T07:52:46+10:00",
"reboot" : false,
"packages_updated" : "",
"job_id" : "",
"message" : "No patches to apply"
}
```

If patching was executed, the task will report:
If patching was executed, the task will report similar to below:

```puppet
```json
{
"date": "Mon May 28 12:29:23 AEST 2018",
"fqdn": "example.lab.vm",
"reboot": "true",
"message": "Patching complete",
"return-code": "Success",
"packagesupdated": [
"kernel-tools-3.10.0-862.2.3.el7.x86_64",
"kernel-tools-libs-3.10.0-862.2.3.el7.x86_64",
"postfix-2:2.10.1-6.el7.x86_64",
"procps-ng-3.3.10-17.el7.x86_64",
"python-perf-3.10.0-862.2.3.el7.x86_64"
],
"securityonly": "false"
"pinned_packages" : [ ],
"security" : false,
"return" : "Success",
"start_time" : "2018-08-07T21:55:20+10:00",
"debug" : "TRIMMED DUE TO LENGTH FOR THIS EXAMPLE, WOULD NORMALLY CONTAIN FULL COMMAND OUTPUT",
"end_time" : "2018-08-07T21:57:11+10:00",
"reboot" : false,
"packages_updated" : [ "NetworkManager-1:1.10.2-14.el7_5.x86_64", "NetworkManager-libnm-1:1.10.2-14.el7_5.x86_64", "NetworkManager-team-1:1.10.2-14.el7_5.x86_64", "NetworkManager-tui-1:1.10.2-14.el7_5.x86_64", "binutils-2.27-27.base.el7.x86_64", "centos-release-7-5.1804.el7.centos.2.x86_64", "git-1.8.3.1-13.el7.x86_64", "gnupg2-2.0.22-4.el7.x86_64", "kernel-tools-3.10.0-862.3.3.el7.x86_64", "kernel-tools-libs-3.10.0-862.3.3.el7.x86_64", "perl-Git-1.8.3.1-13.el7.noarch", "python-2.7.5-68.el7.x86_64", "python-libs-2.7.5-68.el7.x86_64", "python-perf-3.10.0-862.3.3.el7.centos.plus.x86_64", "selinux-policy-3.13.1-192.el7_5.3.noarch", "selinux-policy-targeted-3.13.1-192.el7_5.3.noarch", "sudo-1.8.19p2-13.el7.x86_64", "yum-plugin-fastestmirror-1.1.31-45.el7.noarch", "yum-utils-1.1.31-45.el7.noarch" ],
"job_id" : "60",
"message" : "Patching complete"
}
```

If patching was blocked, the task will report similar to below:

```json
Error: Task exited : 100
Patching blocked
```
A summary of the patch run is also written to `/etc/os_patching/run_history`, the last line of which is used by the `os_patching.last_run` fact.

```
2018-08-07T14:47:24+10:00|No patches to apply|Success|false|false|
2018-08-07T14:56:56+10:00|Patching complete|Success|false|false|121
2018-08-07T15:04:42+10:00|yum timeout after 2 seconds : Loaded plugins: versionlock|1|||
2018-08-07T15:05:51+10:00|yum timeout after 3 seconds : Loaded plugins: versionlock|1|||
2018-08-07T15:10:16+10:00|Patching complete|Success|false|false|127
2018-08-07T21:31:47+10:00|Patching blocked |100|||
2018-08-08T07:53:59+10:00|Patching blocked |100|||
```

### `/etc/os_patching` directory

This directory contains the various control files needed for the fact and task to work correctly. They are managed by the manifest.

* `/etc/os_patching/blackout_windows` - contains name, start and end time for all blackout windows
* `/etc/os_patching/package_updates` - a list of all package updates available, populated by `/usr/local/bin/os_patching_fact_generation.sh`, triggered through cron
* `/etc/os_patching/security_package_updates` - a list of all security_package updates available, populated by `/usr/local/bin/os_patching_fact_generation.sh`, triggered through cron
* `/etc/os_patching/run_history` - a summary of each run of the `os_patching::patch_server` task, populated by the task
* `/etc/os_patching/reboot_override` - if present, overrides the `reboot=` parameter to the task
* `/etc/os_patching/patch_window` - if present, sets the value for the fact `os_patching.patch_window`


## Limitations

This module is for PE2018+ with agents capable of running tasks. It is currently limited to the Red Hat and Debian operating system.

Debian nodes currently do not allow `security_only` patch tasks to be set to `true`, a fix for this is being worked on.

## Development

Fork, develop, submit a pull request
Expand Down
56 changes: 53 additions & 3 deletions manifests/init.pp
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,6 +1,56 @@
# @summary This manifest sets up the script and cron job to write
# custom structured facts that contain patching data.
# @example include os_patching
# @summary This manifest sets up a script and cron job to populate
# the `os_patching` fact.
#
# @param [String] patch_data_owner User name for the owner of the patch data
# @param [String] patch_data_group Group name for the owner of the patch data
# @param [String] patch_cron_user User name to run the cron job as (defaults to patch_data_owner)
# @param [Boolean] install_delta_rpm Should the deltarpm package be installed on RedHat family nodes
# @param [Boolean] reboot_override Controls on a node level if a reboot should/should not be done after patching.
# This overrides the setting in the task
# @param [Hash] blackout_windows A hash containing the patch blackout windows, which prevent patching.
# The dates are in full ISO8601 format.
# @param [String] patch_window A freeform text entry used to allocate a node to a specific patch window (Optional)
# @param [String] patch_cron_hour The hour(s) for the cron job to run (defaults to absent, which means '*' in cron)
# @param [String] patch_cron_month The month(s) for the cron job to run (defaults to absent, which means '*' in cron)
# @param [String] patch_cron_monthday The monthday(s) for the cron job to run (defaults to absent, which means '*' in cron)
# @param [String] patch_cron_weekday The weekday(s) for the cron job to run (defaults to absent, which means '*' in cron)
# @param [String] patch_cron_min The min(s) for the cron job to run (defaults to a random number between 0 and 59)
#
# @example assign node to 'Week3' patching window, force a reboot and create a blackout window for the end of the year
# class { 'os_patching':
# patch_window => 'Week3',
# reboot_override => true,
# blackout_windows => { 'End of year change freeze':
# {
# 'start': '2018-12-15T00:00:00+1000',
# 'end': '2019-01-15T23:59:59+1000',
# }
# },
# }
#
# @example An example profile to setup patching, sourcing blackout windows from hiera
# class profiles::soe::patching (
# $patch_window = undef,
# $blackout_windows = undef,
# $reboot_override = undef,
# ){
# # Pull any blackout windows out of hiera
# $hiera_blackout_windows = lookup('profiles::soe::patching::blackout_windows',Hash,hash,{})
#
# # Merge the blackout windows from the parameter and hiera
# $full_blackout_windows = $hiera_blackout_windows + $blackout_windows
#
# # Call the os_patching class to set everything up
# class { 'os_patching':
# patch_window => $patch_window,
# reboot_override => $reboot_override,
# blackout_windows => $full_blackout_windows,
# }
# }
#
# @example JSON hash to specify a change freeze from 2018-12-15 to 2019-01-15
# {"End of year change freeze": {"start": "2018-12-15T00:00:00+1000", "end": "2019-01-15T23:59:59+1000"}}
#
class os_patching (
String $patch_data_owner = 'root',
String $patch_data_group = 'root',
Expand Down
2 changes: 1 addition & 1 deletion metadata.json
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
{
"name": "albatrossflavour-os_patching",
"version": "0.1.19",
"version": "0.2.1",
"author": "Tony Green",
"summary": "Perform OS level patching through tasks and report on patch levels through facts",
"license": "Apache-2.0",
Expand Down