Skip to content

Commit 073890e

Browse files
openshift-merge-bot[bot]openshift-merge-bot[bot]
authored
Merge pull request #170 from dorzel/main
Blog post for build scheduler features
2 parents dc692b4 + 1ed5f58 commit 073890e

File tree

1 file changed

+220
-0
lines changed

1 file changed

+220
-0
lines changed

‎content/docs/blog/posts/2025-5-20-build-scheduler-features.md

Lines changed: 220 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,220 @@
1+
---
2+
title: "Shipwright build scheduler features"
3+
date: 2025-05-20T11:39:00-06:00
4+
draft: false
5+
author: "Dylan Orzel ([@dorzel](https://github.com/dorzel))"
6+
---
7+
8+
## New Build Scheduler Features
9+
10+
A new set of build scheduling features introduced in [v0.15](https://shipwright.io/docs/blog/posts/2025-02-28-release-v0.15) allows users to specify node selectors, custom schedulers, and tolerations for builds.
11+
12+
These make it easier to schedule builds on clusters with nodes of multiple CPU architectures, use a scheduler that is tuned to a certain workflow, or just more general control of which nodes builds run on.
13+
14+
### CLI flags to use build scheduler features
15+
16+
With these features also comes new CLI flags (introduced in v0.16) that allow specifying nodeSelectors and custom schedulers on the command line when using `shp` with `Builds` or `BuildRuns`.
17+
18+
In the following commands, `--node-selector` and `--scheduler-name` sets these fields on the `Build` or `BuildRun` objects:
19+
20+
- `shp build create`
21+
- `shp build run`
22+
- `shp build upload`
23+
- `shp buildrun create`
24+
25+
### Example with `shp build run` and `--node-selector`
26+
27+
We can specify a build to be scheduled to a node with certain labels by using a node selector. Here, we'll schedule a build to a node with an ARM CPU architecture. Starting with an example build:
28+
29+
```bash
30+
$ shp build create test-golang-build \
31+
--output-image=kind.local/test/test-golang-build \
32+
--source-git-url=https://github.com/shipwright-io/sample-go \
33+
--source-context-dir=docker-build \
34+
--strategy-name=buildah-shipwright-managed-push
35+
```
36+
37+
We'll run the build and specify the arm64 architecture in the node selector flag:
38+
39+
```bash
40+
$ shp build run test-golang-build --node-selector=kubernetes.io/arch=arm64
41+
```
42+
43+
and see that the created BuildRun now has a nodeSelector specified and has been scheduled on the arm64 node:
44+
45+
```yaml
46+
apiVersion: shipwright.io/v1beta1
47+
kind: BuildRun
48+
metadata:
49+
creationTimestamp: "2025-05-20T17:27:01Z"
50+
generateName: test-golang-build-
51+
generation: 1
52+
labels:
53+
build.shipwright.io/generation: "1"
54+
build.shipwright.io/name: test-golang-build
55+
name: test-golang-build-glghq
56+
namespace: default
57+
resourceVersion: "31512392"
58+
uid: 19f02c99-1d24-44c0-a5f9-ab544fdf55ae
59+
spec:
60+
build:
61+
name: test-golang-build
62+
nodeSelector:
63+
kubernetes.io/arch: arm64
64+
```
65+
66+
```bash
67+
$ kubectl get pod test-golang-build-glghq-tfhnn-pod -o jsonpath='{.spec.nodeName} {.spec.nodeSelector}'
68+
69+
arm-node.compute.internal {"kubernetes.io/arch":"arm64"}
70+
71+
$ kubectl get node arm-node.compute.internal -o jsonpath='{.metadata.labels.kubernetes\.io/arch}'
72+
73+
arm64
74+
```
75+
76+
### Example with `shp build create` and `--scheduler-name`
77+
78+
This time we'll specify a scheduler name `test-scheduler` that is assumed to be a custom scheduler that has been deployed to the cluster already.
79+
80+
Instead of specifying these options when running the build, we can also specify them when creating the build:
81+
82+
```bash
83+
$ shp build create test-golang-build \
84+
--output-image=kind.local/test/test-golang-build \
85+
--source-git-url=https://github.com/shipwright-io/sample-go \
86+
--source-context-dir=docker-build \
87+
--strategy-name=buildah-shipwright-managed-push \
88+
--scheduler-name=test-scheduler
89+
```
90+
91+
and see that the scheduler name appears in the Build yaml:
92+
93+
```yaml
94+
apiVersion: shipwright.io/v1beta1
95+
kind: Build
96+
metadata:
97+
creationTimestamp: "2025-05-20T17:49:07Z"
98+
generation: 1
99+
name: test-golang-build
100+
namespace: default
101+
resourceVersion: "31518385"
102+
uid: bd2333fb-71eb-4228-bc5f-5a466032fcc5
103+
spec:
104+
output:
105+
image: kind.local/test/test-golang-build
106+
schedulerName: test-scheduler
107+
source:
108+
contextDir: docker-build
109+
git:
110+
url: https://github.com/shipwright-io/sample-go
111+
type: Git
112+
strategy:
113+
kind: ClusterBuildStrategy
114+
name: buildah-shipwright-managed-push
115+
status:
116+
message: all validations succeeded
117+
reason: Succeeded
118+
registered: "True"
119+
```
120+
121+
After running the build with `shp build run test-golang-build`, we see that the schedulerName got picked up by the build pod:
122+
123+
```bash
124+
$ kubectl get pods test-golang-build-j9vbd-qr9cr-pod -o jsonpath='{.spec.schedulerName}'
125+
126+
test-scheduler
127+
```
128+
129+
### Example with setting tolerations on a `BuildRun`
130+
131+
We'll start with the same example build as above:
132+
133+
```bash
134+
$ shp build create test-golang-build \
135+
--output-image=kind.local/test/test-golang-build \
136+
--source-git-url=https://github.com/shipwright-io/sample-go \
137+
--source-context-dir=docker-build \
138+
--strategy-name=buildah-shipwright-managed-push
139+
```
140+
141+
```yaml
142+
apiVersion: shipwright.io/v1beta1
143+
kind: Build
144+
metadata:
145+
creationTimestamp: "2025-05-20T18:29:43Z"
146+
generation: 1
147+
name: test-golang-build
148+
namespace: default
149+
resourceVersion: "31529978"
150+
uid: f393de53-e389-4e25-a29a-a434505bd82e
151+
spec:
152+
output:
153+
image: kind.local/test/test-golang-build
154+
source:
155+
contextDir: docker-build
156+
git:
157+
url: https://github.com/shipwright-io/sample-go
158+
type: Git
159+
strategy:
160+
kind: ClusterBuildStrategy
161+
name: buildah-shipwright-managed-push
162+
```
163+
164+
In this example we have a three node cluster, so let's taint all of the nodes with `test-key` and `test-value`. This will prevent any pod from scheduling on these nodes unless it tolerates this taint:
165+
166+
```bash
167+
$ kubectl get nodes -o name
168+
node/test-node-1
169+
node/test-node-2
170+
node/test-node-3
171+
172+
$ kubectl taint nodes test-node-1 test-key=test-value:NoSchedule
173+
node/test-node-1 tainted
174+
$ kubectl taint nodes test-node-2 test-key=test-value:NoSchedule
175+
node/test-node-2 tainted
176+
$ kubectl taint nodes test-node-3 test-key=test-value:NoSchedule
177+
node/test-node-3 tainted
178+
```
179+
180+
We see that if we create a BuildRun now, it will fail to schedule:
181+
182+
```bash
183+
$ shp build run test-golang-build
184+
185+
$ kubectl get events
186+
LAST SEEN TYPE REASON OBJECT MESSAGE
187+
...
188+
2m54s Normal Pending taskrun/test-golang-build-bqw9s-hbkds pod status "PodScheduled":"False"; message: "0/3 nodes are available: 3 node(s) had untolerated taint {test-key: test-value}. preemption: 0/3 nodes are available: 3 Preemption is not helpful for scheduling."
189+
...
190+
```
191+
192+
Let's patch the Build with the following toleration in order to have it tolerate the node taint:
193+
194+
```yaml
195+
tolerations:
196+
- key: "test-key"
197+
operator: "Equal"
198+
value: "test-value"
199+
effect: "NoSchedule"
200+
```
201+
202+
```bash
203+
$ kubectl patch Build test-golang-build --type=merge -p '{"spec":{"tolerations":[{"key":"test-key","operator":"Equal","value":"test-value"}]}}'
204+
205+
$ shp build run test-golang-build
206+
```
207+
208+
Now we see that the build is successfully scheduled:
209+
210+
```bash
211+
$ kubectl get events
212+
LAST SEEN TYPE REASON OBJECT MESSAGE
213+
...
214+
109s Normal Scheduled pod/test-golang-build-k7zc4-492d2-pod Successfully assigned default/test-golang-build-k7zc4-492d2-pod to ip-10-0-2-69.us-east-2.compute.internal
215+
...
216+
```
217+
218+
### Conclusion
219+
220+
Shipwright's new build scheduling options make it much easier to control where builds get placed in the cluster and give more options to those who are operating in multi-arch environments or with other constraints.

0 commit comments

Comments
 (0)