You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Adds a Nomad Autoscaler target plugin that fails a conflicting active
deployment before changing a task group count (Nomad rejects scaling
while a deployment is in progress). Scaling is serialized per namespaced
job and retried from fresh state on races.
The plugin is a no-op when the durable count already matches, and the
convergence check ignores the rollout deployment spawned by its own
scale, so in-flight rollouts converging to the right count are never
failed.
Copy file name to clipboardExpand all lines: packages/nomad-nodepool-apm/README.md
+30-9Lines changed: 30 additions & 9 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -1,10 +1,15 @@
1
-
# Nomad NodePool APM Plugin
1
+
# Nomad Node Pool Autoscaler Plugins
2
2
3
-
A custom APM (Application Performance Monitoring) plugin for the [Nomad Autoscaler](https://github.com/hashicorp/nomad-autoscaler) that provides the count of nodes in a Nomad node pool.
3
+
Custom plugins for the [Nomad Autoscaler](https://github.com/hashicorp/nomad-autoscaler) that keep a service-job allocation count aligned with the number of nodes in a Nomad node pool.
4
4
5
5
## Purpose
6
6
7
-
This plugin enables service jobs to scale their count based on the number of nodes in a specific node pool, effectively replicating the behavior of system jobs while benefiting from proper rolling update support.
7
+
The package builds two plugins because Nomad Autoscaler supports only one plugin type per external binary:
8
+
9
+
-`nomad-nodepool-apm` reports the number of ready, eligible nodes in a pool.
10
+
-`nomad-deployment-aware-target` fails an active deployment before applying the requested task-group count.
11
+
12
+
Together they let service jobs replicate system-job placement while retaining rolling updates. The deployment-aware target is used by the `orchestrator-ee` service job. When the task-group count must change, scaling intentionally abandons a conflicting in-progress rollout (Nomad rejects scaling while a deployment is active); Nomad records that deployment as `failed`, rather than `cancelled`. The rollout spawned by the scale itself is left to run, and no deployment is touched when the count already matches. The target requires `auto_revert = false` on the scaled group (otherwise failing a deployment would restore an older job version and ping-pong); it refuses to scale groups with `auto_revert = true` and returns an error on every attempt.
8
13
9
14
## Usage
10
15
@@ -25,14 +30,23 @@ GCP_PROJECT_ID=your-project-id make build-and-upload
25
30
In your Nomad Autoscaler configuration:
26
31
27
32
```hcl
28
-
apm "nomad-nodepool" {
29
-
driver = "nomad-nodepool"
33
+
apm "nomad-nodepool-apm" {
34
+
driver = "nomad-nodepool-apm"
30
35
config = {
31
36
nomad_address = "http://localhost:4646" # Optional, uses NOMAD_ADDR env var
32
37
nomad_token = "your-token" # Optional, uses NOMAD_TOKEN env var
33
38
nomad_region = "global" # Optional
34
39
}
35
40
}
41
+
42
+
target "nomad-deployment-aware-target" {
43
+
driver = "nomad-deployment-aware-target"
44
+
config = {
45
+
nomad_address = "http://localhost:4646"
46
+
nomad_token = "your-token"
47
+
nomad_region = "global"
48
+
}
49
+
}
36
50
```
37
51
38
52
### In Job Scaling Policy
@@ -47,9 +61,11 @@ scaling {
47
61
evaluation_interval = "30s"
48
62
cooldown = "2m"
49
63
64
+
target "nomad-deployment-aware-target" {}
65
+
50
66
check "match_node_count" {
51
-
source = "nomad-nodepool"
52
-
query = "build" # Node pool name
67
+
source = "nomad-nodepool-apm"
68
+
query = "orchestrator" # Node pool name
53
69
54
70
strategy "pass-through" {} # Use node count directly as desired count
55
71
}
@@ -65,12 +81,18 @@ scaling {
65
81
- Scheduling eligibility: `eligible`
66
82
3. Returns the count as a metric for the autoscaler to use
67
83
4. With the `pass-through` strategy, this count becomes the desired number of allocations
84
+
5. The target serializes scaling per namespaced job and is a no-op when the durable count already matches
85
+
6. When the count must change it fails a conflicting active deployment, rereads the job, and scales with the current job modify index
86
+
7. It verifies the final durable count; the rollout spawned by the scale proceeds normally
87
+
8. Concurrent job or deployment changes are retried from fresh state with a bounded attempt count
88
+
89
+
Dry-run actions do not fail deployments or write task-group counts.
68
90
69
91
## Query Format
70
92
71
93
The `query` parameter should be the name of the node pool you want to count nodes from.
72
94
73
-
Example: `query = "build"`will count all ready, eligible nodes in the "build" node pool.
95
+
Example: `query = "orchestrator"`counts all ready, eligible nodes in the `orchestrator` node pool.
74
96
75
97
## Configuration Options
76
98
@@ -80,4 +102,3 @@ Example: `query = "build"` will count all ready, eligible nodes in the "build" n
80
102
|`nomad_token`| Nomad ACL token |`NOMAD_TOKEN` env var |
0 commit comments