Skip to content

Commit 01b0bdd

Browse files
mcgrofmcgrof
committed
docs/kernel-ci: add specific CI examples
Extend documentation by providing the subsystems for which we have a kernel-ci for. Signed-off-by: Luis Chamberlain <[email protected]>
1 parent 683610d commit 01b0bdd

8 files changed

+521
-113
lines changed

‎docs/kernel-ci/README.md

Lines changed: 26 additions & 113 deletions
Original file line numberDiff line numberDiff line change
@@ -11,8 +11,11 @@ mailing lists
1111
[kernel-patches-daemon](https://github.com/facebookincubator/kernel-patches-daemon)
1212
(kpd) can optionally be used and is supported.
1313

14-
Both github and gitlab actions are optionally supported so to leverage
15-
triggering tests on custom repositories with custom branches. Either kpd
14+
The CI part of github and gitlab are called Github actions and GitLab Pipelines
15+
respectively. To keep things simple we refer to both as ``git CI solutions``.
16+
Both ``git CI solutions`` are optionally supported to leverage
17+
triggering tests on custom repositories with custom branches. Either
18+
[kpd](https://github.com/facebookincubator/kernel-patches-daemon)
1619
can push custom branches to test or developers can manually push custom
1720
branches to test. Once confidence on a CI pipeline is high, tests against
1821
new Linux releases can also be automated.
@@ -84,117 +87,27 @@ representation of what this looks like.
8487
+--------------------------+
8588
```
8689

87-
# What's possible at a glance
88-
Here's what's possible. The variability that kdevops supports through
89-
the adoption of Linux kconfig, it let's you expose that variability directly
90-
from the github action. You can git push a tree, and then customize specifics
91-
of your workflow test.
92-
93-
Support for enabling ``Run workflow`` to be available to your CI requires the default
94-
branch used for the repository to have in place an existing `.github/*/*.yml` with the
95-
`workflow_dispatch` enabled. Only privileged users of the repository will be able to
96-
trigger manual workflows.
97-
98-
<img src="/docs/kernel-ci/fstests-workflow/0001-fstests-run-workflow.png" width=1024 align=center alt="001-fstests-run-workflow.png">
99-
100-
101-
<table>
102-
<tr>
103-
<th>Description</th>
104-
<th>Options</th>
105-
</tr>
106-
<tr>
107-
<td>
108-
Here is the first drop down menu once you click on "Run workflow" button.
109-
With it you can now see different test variability options. Most
110-
of the test variability options you've defined through Kconfig on kdevops
111-
can become live realtime test variability options you can now configure from your CI.
112-
</td>
113-
<td>
114-
<img src="/docs/kernel-ci/fstests-workflow/0002-drop-down-menu.png" alt="First drop down menu">
115-
</td>
116-
</tr>
117-
<tr>
118-
<td>
119-
You can pick the branch which you want the test to run. Each branch in turn can
120-
have custom input options of its own. So for example a tree which lacks LBS can
121-
support can avoid having the options to test 16k, 32k and 64k block size filesystems.
122-
</td>
123-
<td>
124-
<img src="/docs/kernel-ci/fstests-workflow/0003-pick-branch.png" alt="0003-pick-branch.png">
125-
</td>
126-
</tr>
127-
<tr>
128-
<td>
129-
You can configure a custom soak duration, by default no soak duration is used.
130-
If enabled kdevops manages to enable soak duraiton by setting an environment variable
131-
which will be read at `make defconfig` time. Support for this was added to kdevops
132-
through commit
133-
<a href="https://github.com/linux-kdevops/kdevops/commit/e6294385d2b3b6">
134-
commit e6294385d2b3b6 ("fstests: add support for overriding soak duration through CLI")
135-
</a>.
136-
See also related fix
137-
<a href="https://github.com/linux-kdevops/kdevops/commit/59aee5697ea0">
138-
commit 59aee5697ea0 ("fstests/Kconfig: use int-specific default for CLI SOAK_DURATION")
139-
</a>.
140-
</td>
141-
<td>
142-
<img src="/docs/kernel-ci/fstests-workflow/0004-pick-soak.png" alt="Pick soak duration">
143-
</td>
144-
</tr>
145-
<tr>
146-
<td>
147-
You can pick the specific test group you want to test, as well, the default in kdevops is to
148-
test the group "auto".
149-
</td>
150-
<td>
151-
<img src="/docs/kernel-ci/fstests-workflow/0005-pick-group.png" alt="Pick test group">
152-
</td>
153-
</tr>
154-
<tr>
155-
<td>
156-
You can select one of all suported tests to run as well, when you do this only that specific
157-
test will run, that's it. By default all test are run which are part of the "auto" group as
158-
that is the default test group kdevops uses.
159-
</td>
160-
<td>
161-
<img src="/docs/kernel-ci/fstests-workflow/0006-pick-test.png" alt="Pick your test">
162-
</td>
163-
</tr>
164-
<tr>
165-
<td>
166-
If you are working on something specific, you may know you want to only test
167-
a few select tests, but you may not want to run a full test group or only
168-
pick one test. From the drop down menu on "Select additional test coverage" pick
169-
"custom". In the field "Enter custom test" put the list of tests you want to test
170-
each test separated by a space. In this example we'll run a test with only
171-
generic/003 and generic/750.
172-
</td>
173-
<td>
174-
<img src="/docs/kernel-ci/fstests-workflow/0007-custom-tests.png" alt="Custom tests">
175-
</td>
176-
</tr>
177-
</table>
178-
179-
Then just click on the green button which says `Run workflow`. You can either just
180-
wait for the test to complete or you can also access the console of the runner
181-
and watch live with a status of each machine being tested, a different target
182-
node will run each test filesystem profile.
183-
184-
![0008-run-tests-wait.png](/docs/kernel-ci/fstests-workflow/0008-run-tests-wait.png)
185-
186-
Below is a real example of an interaction with an existing ongoing fstests test
187-
for XFS development. kdevops leverages a custom test specific watchdog in
188-
kdevops for fstests the
189-
[fstests_watchdog.py](https://github.com/linux-kdevops/kdevops/blob/main/scripts/workflows/fstests/fstests_watchdog.py).
190-
For documentation refer to the
191-
[fstests Kconfig](https://github.com/linux-kdevops/kdevops/blob/main/workflows/fstests/Kconfig)
192-
on the related `FSTESTS_WATCHDOG` options and `SOAK_DURATION`. Live monitoring
193-
is only useful for tests which take a long time, like fstests, or blktests.
194-
Each test which takes a long time to run should consider implementing its own
195-
watchdog to allow test progress monitoring.
196-
197-
![example kdevops fstests CI watchdog live run](/docs/kernel-ci/fstests-workflow/0009-watchdog-example-2024-11-12.png)
90+
# Active kdevops CIs
91+
92+
The below are active kdevops CIs used to help developers proactively test
93+
patches either from the mailing list for different Linux kernel subsystems,
94+
or to help increase confidence in a branch to be used as a pull request sent
95+
to Linus. We also have kdevops's own CI. Each CI has its own use case and
96+
documented as such. The kdevops tree has its CI built-in to the main branch
97+
of the development tree. However, to enable to Linux kernel development trees
98+
to *use* kdevops to run tests, the [kdevops-ci tree](kdevops-ci-tree.md) is
99+
can merged into development tree before pushing onto kdevops CI enabled trees.
100+
Read first the [kdevops-ci tree documentation](kdevops-ci-tree.md). The list
101+
of active CI trees leveraging all this and their respective documentation
102+
are:
103+
104+
* [kdevops CI](kdevops-ci.md)
105+
* [Linux kernel filesystem kdevops CI](linux-filesystems-kdevops-CI-testing.md)
106+
* [Linux kernel XFS kdevops CI](linux-xfs-kdevops-ci.md)
107+
* Linux kernel simple subsystems:
108+
* [Linux kernel modules kdevops CI](linux-modules-kdevops-ci.md)
109+
* [Linux kernel firmware loader kdevops CI](linux-firmware-kdevops-ci.md)
110+
* [Linux kernel mm kdevops CI](linux-mm-kdevops-ci.md)
198111

199112
# kernel-patches-daemon support
200113

‎docs/kernel-ci/kdevops-ci-tree.md

Lines changed: 122 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,122 @@
1+
# kdevops-ci tree
2+
3+
The different ``git CI solutions`` supported (github actions and gitlab
4+
pipelines) are supported to enable Linux kernel testing using kdevops
5+
through a specific git tree:
6+
7+
* [kdevops-ci](https://github.com/linux-kdevops/kdevops-ci)
8+
9+
The main branch is intended to be generic enough for all simple Linux
10+
kernel tests which don't have complex tests and which also use and leverage
11+
the Linux kernel selftests. Topic branches allow further subsystem CI test
12+
customizations which we don't yet generalize. Topic branches may also be
13+
inevitable since since each CI may have custom test variability options. It
14+
might be possible to address the variability for each test in one main tree in
15+
the future, however this would require a bit more R&D to investigate if this
16+
is possible.
17+
18+
Test results are pushed onto the
19+
[kdevops-results-archive](https://github.com/linux-kdevops/kdevops-results-archive)
20+
and each results has a commit log intended to be descriptive enough to help
21+
developers easily understand the final status of a test. Topic branches also
22+
allow each subsystem developer to fine to results status messaging for tests.
23+
24+
## kdevops-ci branches
25+
26+
The following branches track different subsystems:
27+
28+
* [main](https://github.com/linux-kdevops/kdevops-ci/tree/main): tracks simple Linux kernel tests and [Linux kernel selftests](https://www.kernel.org/doc/html/latest/dev-tools/kselftest.html)
29+
* [fstests](https://github.com/linux-kdevops/kdevops-ci/tree/fstests): example branch you should use to start your filesystem specific support
30+
* [xfs](https://github.com/linux-kdevops/kdevops-ci/tree/xfs): the latest XFS CI
31+
* [mm](https://github.com/linux-kdevops/kdevops-ci/tree/mm): CI for Linux kernel memory management
32+
33+
# kdevops-ci file
34+
35+
We try to break down work into different files. This allows us to share as much
36+
code as possible on the main branch. The main branch has these files for
37+
instance:
38+
39+
* [.github/workflows/kdevops-init.yml](https://github.com/linux-kdevops/kdevops-ci/blob/main/.github/workflows/kdevops-init.yml)
40+
* [.github/workflows/kdevops-cleanup.yml](https://github.com/linux-kdevops/kdevops-ci/blob/main/.github/workflows/kdevops-generic.yml)
41+
* [.github/workflows/kdevops-generic.yml](https://github.com/linux-kdevops/kdevops-ci/blob/main/.github/workflows/kdevops-generic.yml)
42+
43+
However a filesystem would replace the kdevops-generic.yml file with:
44+
45+
* [.github/workflows/kdevops-fstests.yml](https://github.com/linux-kdevops/kdevops-ci/blob/fstests/.github/workflows/kdevops-fstests.yml)
46+
47+
And the mm subsystem CI would replace the same kdevops-generic.yml file with:
48+
49+
* [.github/workflows/kdevops-mm.yml](https://github.com/linux-kdevops/kdevops-ci/blob/mm/.github/workflows/kdevops-mm.yml)
50+
51+
The delta between the main subsystem worker file is pretty minor to the
52+
[kdevops-fstests.yml](https://github.com/linux-kdevops/kdevops-ci/blob/fstests/.github/workflows/kdevops-fstests.yml) file,
53+
mostly test results parsing modifications for the
54+
[kdevops-results-archive](https://github.com/linux-kdevops/kdevops-results-archive)
55+
and CI variability on the web graphical user interface when you are customizing
56+
a test. It may be possible in the future to completely avoid all this delta,
57+
that however will require more CI R&D.
58+
59+
# Consideration of long running tests
60+
61+
github actions lets you customize when you should run tests. These can be
62+
triggered by pull requests, pushes, or manual runs. The main branch of
63+
kdevops-ci by default enables all methods to trigger tests. But for some
64+
subsystems, running a full test can take 4-8 hours. For this reason some
65+
topic branches on kdevops-ci may disable automatic test runs on git pushes.
66+
67+
For example, the fstests branch for kdevops-ci disables running tests on
68+
git push. You can however force to enable this in your CI workflow easily,
69+
with just three lines of github action code:
70+
71+
```
72+
diff --git a/.github/workflows/kdevops-fstests.yml b/.github/workflows/kdevops-fstests.yml
73+
index b2da84e96f42..eafc9c59b394 100644
74+
--- a/.github/workflows/kdevops-fstests.yml
75+
+++ b/.github/workflows/kdevops-fstests.yml
76+
@@ -14,9 +14,9 @@ name: Run fstests
77+
#
78+
79+
on:
80+
-# push:
81+
-# branches:
82+
-# - '**'
83+
+ push:
84+
+ branches:
85+
+ - '**'
86+
pull_request:
87+
branches:
88+
- '**'
89+
```
90+
91+
You should consider how long tests take before enabling automatic tests
92+
to run on all git pushes.
93+
94+
# Merging kdevops-ci into Linux kernel trees
95+
96+
To leverage kdevops for a CI all you need to do merge the kdevops-ci topic
97+
branch for your subsystem into your Linux kernel tree and git push the branch
98+
to a kdevops CI tree dedicated for your subsystem. The only thing special
99+
about the kdevops CI tree you push your changes onto is that it has a github
100+
self-hosted runner which is capable of running kdevops.
101+
102+
For example if working on XFS testing, you would use the xfs branch of
103+
kdevops-ci, and so you would just do something like this:
104+
105+
```
106+
# Merge the kdevops-ci xfs branch
107+
git remote add kdevops-ci https://github.com/linux-kdevops/kdevops-ci.git
108+
git merge kdevops-ci/xfs --allow-unrelated-histories --squash
109+
git commit -a -s -m "CI: add kdevops actions from $(date -I)"
110+
111+
# Push onto linux-kdevops-xfs
112+
git remote add linux-xfs-kpd [email protected]:linux-kdevops/linux-xfs-kpd.git
113+
git push -u linux-xfs-kpd example-xfs:example-xfs
114+
```
115+
116+
For most subsystem, git pushes of new branches trigger automatic test runs.
117+
However, since filesystems take long, by default you'd have to go and trigger
118+
the run manually after a git push. You can view live tests running in realtime
119+
and you can also trigger test runs by going to the respective subsystem
120+
github actions page. For example to view existing test runs for XFS or to
121+
trigger a new custom XFS test you can go to
122+
[linux-xfs-kpd actions page](https://github.com/linux-kdevops/linux-xfs-kpd/actions).

‎docs/kernel-ci/kdevops-ci.md

Lines changed: 63 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,63 @@
1+
# kdevops's own CI
2+
3+
kdevops has its own CI to help allow testing of kdevops patches posted
4+
to the [email protected] mailing list and also allow developers to
5+
push development test branches to test. Two repositories are used for testing:
6+
7+
* [kdevops](https://github.com/linux-kdevops/kdevops) - Main kdevops development tree
8+
* [kdevops-kpd](https://github.com/linux-kdevops/kdevops-kpd) - CI tree used by [kpd](https://github.com/facebookincubator/kernel-patches-daemon)
9+
10+
We also have a patchwork instance where run time test results are listed
11+
as part of the column with "Success/Warning/Failures" "S/W/F":
12+
13+
* [kdevops patchwork](https://patchwork.kernel.org/project/kdevops/list/)
14+
15+
Testing is done through both [kdevops](https://github.com/linux-kdevops/kdevops)
16+
and [kdevops-kpd](https://github.com/linux-kdevops/kdevops-kpd) trees,
17+
however the main purpose of the
18+
[kdevops-kpd](https://github.com/linux-kdevops/kdevops-kpd) tree is to
19+
proactively test mailing list patches and not pollute the
20+
[kdevops](https://github.com/linux-kdevops/kdevops) with random test branches.
21+
22+
If you are a kdevops developers you can use either tree to also proactively
23+
test a branch. Each tree uses its own self hosted server.
24+
25+
# kdevops CI testing
26+
27+
The [kdevops](https://github.com/linux-kdevops/kdevops) is where you get
28+
the latest and greatest kdevops. To be sure we don't regress workloads
29+
we proactively test the main tree. The work is defined on the directory
30+
.github/workflows/ and we currently have two workloads:
31+
32+
* docker-tests.yml - limited docker tests, there's only so much you can do with
33+
containers on kdevops so this is bare bones testing
34+
* fstests.yml: build Linus' tree, xfstests-dev, and runs generic/003, collects
35+
test results
36+
37+
With time this should be expanded to also run similar simple tests for each
38+
supported workload with a limited test scope so to ensure testing will not
39+
regress.
40+
41+
## kdevops CI testing results
42+
43+
You can see kdevops test results here:
44+
45+
* [Ephemeral interactive kdevops CI results](https://github.com/linux-kdevops/kdevops/actions)
46+
* [Persistent kdevops-results-archive kdevops test results](https://github.com/search?q=repo%3Alinux-kdevops%2Fkdevops-results-archive+is%3Acommit+%22kdevops%22+NOT+%22kdevops-kpd%3A%22&type=commits)
47+
48+
# kdevops-kpd CI testing
49+
50+
[kpd](https://github.com/facebookincubator/kernel-patches-daemon) requires
51+
a tree to push branches with patches it gathers from series from patchwork.
52+
To not pollute the branch space on the main development
53+
[kdevops](https://github.com/linux-kdevops/kdevops) tree we use a separate
54+
tree for testing branches which
55+
[kpd](https://github.com/facebookincubator/kernel-patches-daemon) puts
56+
together for us from patches posted to the [email protected] mailing list.
57+
58+
## kdevops-kpd CI testing results
59+
60+
You can see kdevops-kpd test results here:
61+
62+
* [Ephemeral interactive kdevops-kpd CI results](https://github.com/linux-kdevops/kdevops-kpd/actions)
63+
* [Persistent kdevops-results-archive kdevops-kpd test results](https://github.com/search?q=repo%3Alinux-kdevops%2Fkdevops-results-archive+is%3Acommit+%22kdevops-kpd%3A%22&type=commits)

0 commit comments

Comments
 (0)