Skip to content

Commit 53bb4c2

Browse files
cgoncalvescgoncalves
committed
Add support to sysctl node-exporter collector
Signed-off-by: Carlos Goncalves <[email protected]>
1 parent d81d23d commit 53bb4c2

File tree

9 files changed

+157
-0
lines changed

9 files changed

+157
-0
lines changed

‎CHANGELOG.md

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -3,6 +3,7 @@
33
## 4.20
44

55
- [#2595](https://github.com/openshift/cluster-monitoring-operator/pull/2595) Multi-tenant support for KSM's CRS feature-set downstream.
6+
- [#2339](https://github.com/openshift/cluster-monitoring-operator/pull/2339) Add support to sysctl node-exporter collector
67

78
## 4.18
89

‎Documentation/api.md

Lines changed: 19 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -34,6 +34,7 @@ Configuring Cluster Monitoring is optional. If the config does not exist or is e
3434
* [NodeExporterCollectorNetClassConfig](#nodeexportercollectornetclassconfig)
3535
* [NodeExporterCollectorNetDevConfig](#nodeexportercollectornetdevconfig)
3636
* [NodeExporterCollectorProcessesConfig](#nodeexportercollectorprocessesconfig)
37+
* [NodeExporterCollectorSysctlConfig](#nodeexportercollectorsysctlconfig)
3738
* [NodeExporterCollectorSystemdConfig](#nodeexportercollectorsystemdconfig)
3839
* [NodeExporterCollectorTcpStatConfig](#nodeexportercollectortcpstatconfig)
3940
* [NodeExporterConfig](#nodeexporterconfig)
@@ -249,6 +250,7 @@ The `NodeExporterCollectorConfig` resource defines settings for individual colle
249250
| mountstats | [NodeExporterCollectorMountStatsConfig](#nodeexportercollectormountstatsconfig) | Defines the configuration of the `mountstats` collector, which collects statistics about NFS volume I/O activities. Disabled by default. |
250251
| ksmd | [NodeExporterCollectorKSMDConfig](#nodeexportercollectorksmdconfig) | Defines the configuration of the `ksmd` collector, which collects statistics from the kernel same-page merger daemon. Disabled by default. |
251252
| processes | [NodeExporterCollectorProcessesConfig](#nodeexportercollectorprocessesconfig) | Defines the configuration of the `processes` collector, which collects statistics from processes and threads running in the system. Disabled by default. |
253+
| sysctl | [NodeExporterCollectorSysctlConfig](#nodeexportercollectorsysctlconfig) | Defines the configuration of the `sysctl` collector, which collects sysctl metrics. Disabled by default. |
252254
| systemd | [NodeExporterCollectorSystemdConfig](#nodeexportercollectorsystemdconfig) | Defines the configuration of the `systemd` collector, which collects statistics on the systemd daemon and its managed services. Disabled by default. |
253255

254256
[Back to TOC](#table-of-contents)
@@ -344,6 +346,23 @@ The `NodeExporterCollectorProcessesConfig` resource works as an on/off switch fo
344346

345347
[Back to TOC](#table-of-contents)
346348

349+
## NodeExporterCollectorSysctlConfig
350+
351+
#### Description
352+
353+
The `NodeExporterCollectorSysctlConfig` resource works as an on/off switch for the `sysctl` collector of the `node-exporter` agent. Caution! Exposing metrics like kernel.random.uuid can disrupt Prometheus, as it generates new data series with every scrape. Use this option judiciously! By default, the `sysctl` collector is disabled.
354+
355+
356+
<em>appears in: [NodeExporterCollectorConfig](#nodeexportercollectorconfig)</em>
357+
358+
| Property | Type | Description |
359+
| -------- | ---- | ----------- |
360+
| enabled | bool | A Boolean flag that enables or disables the `sysctl` collector. |
361+
| includeSysctlMetrics | []string | A list of numeric sysctl values. Note that a sysctl can contain multiple values, for example: `net.ipv4.tcp_rmem = 4096\t131072\t6291456`. Using `includeSysctlMetrics: ['net.ipv4.tcp_rmem']` the collector will expose: `node_sysctl_net_ipv4_tcp_rmem{index=\"0\"} 4096`, `node_sysctl_net_ipv4_tcp_rmem{index=\"1\"} 131072`, `node_sysctl_net_ipv4_tcp_rmem{index=\"2\"} 6291456`. If the indexes have defined meaning like in this case, the values can be mapped to multiple metrics: `includeSysctlMetrics: ['net.ipv4.tcp_rmem:min,default,max']`. The collector will expose these metrics as such: `node_sysctl_net_ipv4_tcp_rmem_min 4096`, `node_sysctl_net_ipv4_tcp_rmem_default 131072`, `node_sysctl_net_ipv4_tcp_rmem_max 6291456`. |
362+
| includeInfoSysctlMetrics | []string | A list of string sysctl values. For example: `includeSysctlMetrics: ['kernel.core_pattern', 'kernel.seccomp.actions_avail = kill_process kill_thread']`. The collector will expose these metrics as such: `node_sysctl_info{name=\"kernel.core_pattern\", value=\"core\"} 1`, `node_sysctl_info{name=\"kernel.seccomp.actions_avail\", index=\"0\", value=\"kill_process\"} 1`, `node_sysctl_info{name=\"kernel.seccomp.actions_avail\", index=\"1\", value=\"kill_thread\"} 1`, ... |
363+
364+
[Back to TOC](#table-of-contents)
365+
347366
## NodeExporterCollectorSystemdConfig
348367

349368
#### Description

‎Documentation/openshiftdocs/index.adoc

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -54,6 +54,7 @@ The configuration file itself is always defined under the `config.yaml` key in t
5454
* link:modules/nodeexportercollectornetclassconfig.adoc[NodeExporterCollectorNetClassConfig]
5555
* link:modules/nodeexportercollectornetdevconfig.adoc[NodeExporterCollectorNetDevConfig]
5656
* link:modules/nodeexportercollectorprocessesconfig.adoc[NodeExporterCollectorProcessesConfig]
57+
* link:modules/nodeexportercollectorsysctlconfig.adoc[NodeExporterCollectorSysctlConfig]
5758
* link:modules/nodeexportercollectorsystemdconfig.adoc[NodeExporterCollectorSystemdConfig]
5859
* link:modules/nodeexportercollectortcpstatconfig.adoc[NodeExporterCollectorTcpStatConfig]
5960
* link:modules/nodeexporterconfig.adoc[NodeExporterConfig]

‎Documentation/openshiftdocs/modules/nodeexportercollectorconfig.adoc

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -34,6 +34,8 @@ Appears in: link:nodeexporterconfig.adoc[NodeExporterConfig]
3434

3535
|processes|link:nodeexportercollectorprocessesconfig.adoc[NodeExporterCollectorProcessesConfig]|Defines the configuration of the `processes` collector, which collects statistics from processes and threads running in the system. Disabled by default.
3636

37+
|sysctl|link:nodeexportercollectorsysctlconfig.adoc[NodeExporterCollectorSysctlConfig]|Defines the configuration of the `sysctl` collector, which collects sysctl metrics. Disabled by default.
38+
3739
|systemd|link:nodeexportercollectorsystemdconfig.adoc[NodeExporterCollectorSystemdConfig]|Defines the configuration of the `systemd` collector, which collects statistics on the systemd daemon and its managed services. Disabled by default.
3840

3941
|===

‎Documentation/openshiftdocs/modules/nodeexportercollectorsysctlconfig.adoc

Lines changed: 29 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,29 @@
1+
// DO NOT EDIT THE CONTENT IN THIS FILE. It is automatically generated from the
2+
// source code for the Cluster Monitoring Operator. Any changes made to this
3+
// file will be overwritten when the content is re-generated. If you wish to
4+
// make edits, read the docgen utility instructions in the source code for the
5+
// CMO.
6+
:_content-type: ASSEMBLY
7+
8+
== NodeExporterCollectorSysctlConfig
9+
10+
=== Description
11+
12+
The `NodeExporterCollectorSysctlConfig` resource works as an on/off switch for the `sysctl` collector of the `node-exporter` agent. Caution! Exposing metrics like kernel.random.uuid can disrupt Prometheus, as it generates new data series with every scrape. Use this option judiciously! By default, the `sysctl` collector is disabled.
13+
14+
15+
16+
Appears in: link:nodeexportercollectorconfig.adoc[NodeExporterCollectorConfig]
17+
18+
[options="header"]
19+
|===
20+
| Property | Type | Description
21+
|enabled|bool|A Boolean flag that enables or disables the `sysctl` collector.
22+
23+
|includeSysctlMetrics|[]string|A list of numeric sysctl values. Note that a sysctl can contain multiple values, for example: `net.ipv4.tcp_rmem = 4096\t131072\t6291456`. Using `includeSysctlMetrics: ['net.ipv4.tcp_rmem']` the collector will expose: `node_sysctl_net_ipv4_tcp_rmem{index=\"0\"} 4096`, `node_sysctl_net_ipv4_tcp_rmem{index=\"1\"} 131072`, `node_sysctl_net_ipv4_tcp_rmem{index=\"2\"} 6291456`. If the indexes have defined meaning like in this case, the values can be mapped to multiple metrics: `includeSysctlMetrics: ['net.ipv4.tcp_rmem:min,default,max']`. The collector will expose these metrics as such: `node_sysctl_net_ipv4_tcp_rmem_min 4096`, `node_sysctl_net_ipv4_tcp_rmem_default 131072`, `node_sysctl_net_ipv4_tcp_rmem_max 6291456`.
24+
25+
|includeInfoSysctlMetrics|[]string|A list of string sysctl values. For example: `includeSysctlMetrics: ['kernel.core_pattern', 'kernel.seccomp.actions_avail = kill_process kill_thread']`. The collector will expose these metrics as such: `node_sysctl_info{name=\"kernel.core_pattern\", value=\"core\"} 1`, `node_sysctl_info{name=\"kernel.seccomp.actions_avail\", index=\"0\", value=\"kill_process\"} 1`, `node_sysctl_info{name=\"kernel.seccomp.actions_avail\", index=\"1\", value=\"kill_thread\"} 1`, ...
26+
27+
|===
28+
29+
link:../index.adoc[Back to TOC]

‎pkg/manifests/config.go

Lines changed: 5 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -387,6 +387,11 @@ func defaultClusterMonitoringConfiguration() ClusterMonitoringConfiguration {
387387
Systemd: NodeExporterCollectorSystemdConfig{
388388
Enabled: false,
389389
},
390+
Sysctl: NodeExporterCollectorSysctlConfig{
391+
Enabled: false,
392+
IncludeSysctlMetrics: []string{},
393+
IncludeInfoSysctlMetrics: []string{},
394+
},
390395
},
391396
},
392397
}

‎pkg/manifests/manifests.go

Lines changed: 31 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -879,6 +879,25 @@ func (f *Factory) updateNodeExporterArgs(args []string) ([]string, error) {
879879
args = setArg(args, "--no-collector.tcpstat", "")
880880
}
881881

882+
if f.config.ClusterMonitoringConfiguration.NodeExporterConfig.Collectors.Sysctl.Enabled {
883+
includeSysctlMetrics := f.config.ClusterMonitoringConfiguration.NodeExporterConfig.Collectors.Sysctl.IncludeSysctlMetrics
884+
includeInfoSysctlMetrics := f.config.ClusterMonitoringConfiguration.NodeExporterConfig.Collectors.Sysctl.IncludeInfoSysctlMetrics
885+
886+
args = setArg(args, "--collector.sysctl", "")
887+
888+
sysctlSet := uniqueSet(includeSysctlMetrics)
889+
for _, sysctl := range sysctlSet {
890+
args = append(args, fmt.Sprintf("--collector.sysctl.include=%s", sysctl))
891+
}
892+
893+
sysctlSet = uniqueSet(includeInfoSysctlMetrics)
894+
for _, sysctl := range sysctlSet {
895+
args = append(args, fmt.Sprintf("--collector.sysctl.include-info=%s", sysctl))
896+
}
897+
} else {
898+
args = setArg(args, "--no-collector.sysctl", "")
899+
}
900+
882901
var excludedDevices string
883902
if f.config.ClusterMonitoringConfiguration.NodeExporterConfig.Collectors.NetDev.Enabled ||
884903
f.config.ClusterMonitoringConfiguration.NodeExporterConfig.Collectors.NetClass.Enabled {
@@ -2358,6 +2377,18 @@ func setArg(args []string, argName string, argValue string) []string {
23582377
return args
23592378
}
23602379

2380+
func uniqueSet(input []string) []string {
2381+
uniqueMap := make(map[string]struct{})
2382+
var unique []string
2383+
for _, str := range input {
2384+
if _, ok := uniqueMap[str]; !ok {
2385+
uniqueMap[str] = struct{}{}
2386+
unique = append(unique, str)
2387+
}
2388+
}
2389+
return unique
2390+
}
2391+
23612392
func (f *Factory) PrometheusRuleValidatingWebhook() (*admissionv1.ValidatingWebhookConfiguration, error) {
23622393
return f.NewValidatingWebhook(f.assets.MustNewAssetSlice(AdmissionWebhookRuleValidatingWebhook))
23632394
}

‎pkg/manifests/manifests_test.go

Lines changed: 34 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -3381,6 +3381,7 @@ func TestNodeExporterCollectorSettings(t *testing.T) {
33813381
name: "default config",
33823382
config: "",
33833383
argsPresent: []string{"--no-collector.cpufreq",
3384+
"--no-collector.sysctl",
33843385
"--no-collector.tcpstat",
33853386
"--collector.netdev",
33863387
"--collector.netclass",
@@ -3393,6 +3394,7 @@ func TestNodeExporterCollectorSettings(t *testing.T) {
33933394
"--no-collector.systemd",
33943395
},
33953396
argsAbsent: []string{"--collector.cpufreq",
3397+
"--collector.sysctl",
33963398
"--collector.tcpstat",
33973399
"--no-collector.netdev",
33983400
"--no-collector.netclass",
@@ -3558,6 +3560,38 @@ nodeExporter:
35583560
"--collector.systemd.unit-include=^(network.+|nss.+)$"},
35593561
argsAbsent: []string{"--no-collector.systemd"},
35603562
},
3563+
{
3564+
name: "disable sysctl collector",
3565+
config: `
3566+
nodeExporter:
3567+
collectors:
3568+
sysctl:
3569+
enabled: false
3570+
`,
3571+
argsPresent: []string{"--no-collector.sysctl"},
3572+
argsAbsent: []string{"--collector.sysctl"},
3573+
},
3574+
{
3575+
name: "enable sysctl collector",
3576+
config: `
3577+
nodeExporter:
3578+
collectors:
3579+
sysctl:
3580+
enabled: true
3581+
includeSysctlMetrics:
3582+
- net.ipv4.tcp_rmem:min,default,max
3583+
- net.ipv4.tcp_mem
3584+
includeInfoSysctlMetrics:
3585+
- kernel.core_pattern
3586+
- kernel.seccomp.actions_avail
3587+
`,
3588+
argsPresent: []string{"--collector.sysctl",
3589+
"--collector.sysctl.include=net.ipv4.tcp_rmem:min,default,max",
3590+
"--collector.sysctl.include=net.ipv4.tcp_mem",
3591+
"--collector.sysctl.include-info=kernel.core_pattern",
3592+
"--collector.sysctl.include-info=kernel.seccomp.actions_avail"},
3593+
argsAbsent: []string{"--no-collector.sysctl"},
3594+
},
35613595
}
35623596

35633597
for _, test := range tests {

‎pkg/manifests/types.go

Lines changed: 35 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -372,6 +372,9 @@ type NodeExporterCollectorConfig struct {
372372
// Defines the configuration of the `processes` collector, which collects statistics from processes and threads running in the system.
373373
// Disabled by default.
374374
Processes NodeExporterCollectorProcessesConfig `json:"processes,omitempty"`
375+
// Defines the configuration of the `sysctl` collector, which collects sysctl metrics.
376+
// Disabled by default.
377+
Sysctl NodeExporterCollectorSysctlConfig `json:"sysctl,omitempty"`
375378
// Defines the configuration of the `systemd` collector, which collects statistics on the systemd daemon and its managed services.
376379
// Disabled by default.
377380
Systemd NodeExporterCollectorSystemdConfig `json:"systemd,omitempty"`
@@ -397,6 +400,38 @@ type NodeExporterCollectorTcpStatConfig struct {
397400
Enabled bool `json:"enabled,omitempty"`
398401
}
399402

403+
// The `NodeExporterCollectorSysctlConfig` resource works as an on/off switch for
404+
// the `sysctl` collector of the `node-exporter` agent.
405+
// Caution! Exposing metrics like kernel.random.uuid can disrupt Prometheus, as it generates new data series with every scrape. Use this option judiciously!
406+
// By default, the `sysctl` collector is disabled.
407+
type NodeExporterCollectorSysctlConfig struct {
408+
// A Boolean flag that enables or disables the `sysctl` collector.
409+
Enabled bool `json:"enabled,omitempty"`
410+
// A list of numeric sysctl values.
411+
// Note that a sysctl can contain multiple values, for example:
412+
// `net.ipv4.tcp_rmem = 4096 131072 6291456`.
413+
// Using `includeSysctlMetrics: ['net.ipv4.tcp_rmem']` the collector will expose:
414+
// `node_sysctl_net_ipv4_tcp_rmem{index="0"} 4096`,
415+
// `node_sysctl_net_ipv4_tcp_rmem{index="1"} 131072`,
416+
// `node_sysctl_net_ipv4_tcp_rmem{index="2"} 6291456`.
417+
// If the indexes have defined meaning like in this case, the values can be mapped to multiple metrics:
418+
// `includeSysctlMetrics: ['net.ipv4.tcp_rmem:min,default,max']`.
419+
// The collector will expose these metrics as such:
420+
// `node_sysctl_net_ipv4_tcp_rmem_min 4096`,
421+
// `node_sysctl_net_ipv4_tcp_rmem_default 131072`,
422+
// `node_sysctl_net_ipv4_tcp_rmem_max 6291456`.
423+
IncludeSysctlMetrics []string `json:"includeSysctlMetrics,omitempty"`
424+
// A list of string sysctl values.
425+
// For example:
426+
// `includeSysctlMetrics: ['kernel.core_pattern', 'kernel.seccomp.actions_avail = kill_process kill_thread']`.
427+
// The collector will expose these metrics as such:
428+
// `node_sysctl_info{name="kernel.core_pattern", value="core"} 1`,
429+
// `node_sysctl_info{name="kernel.seccomp.actions_avail", index="0", value="kill_process"} 1`,
430+
// `node_sysctl_info{name="kernel.seccomp.actions_avail", index="1", value="kill_thread"} 1`,
431+
// ...
432+
IncludeInfoSysctlMetrics []string `json:"includeInfoSysctlMetrics,omitempty"`
433+
}
434+
400435
// The `NodeExporterCollectorNetDevConfig` resource works as an on/off switch for
401436
// the `netdev` collector of the `node-exporter` agent.
402437
// By default, the `netdev` collector is enabled.

0 commit comments

Comments
 (0)