Skip to content

Commit c214b37

Browse files
committed
lws_spawn: add cgroup support if on linux
1 parent e8c1b9c commit c214b37

File tree

8 files changed

+585
-2
lines changed

8 files changed

+585
-2
lines changed

READMEs/README.lws_spawn.md

Lines changed: 90 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,90 @@
1+
# Spawning and Managing Child Processes with `lws_spawn`
2+
3+
The `lws_spawn` API provides a robust, platform-agnostic way to create and manage child processes directly from your C application, fully integrated with the libwebsockets event loop.
4+
5+
## Overview
6+
7+
The `lws_spawn` API is designed to securely run external executables as child processes. Its key features include:
8+
9+
* **Event-Loop Integration**: The child process lifecycle is managed without blocking the main lws event loop.
10+
* **Standard I/O Redirection**: The child's `stdin`, `stdout`, and `stderr` are automatically redirected to pipes. These pipes are represented as standard `struct lws` connection instances (`wsi`), allowing you to interact with the child process using familiar lws protocol callbacks.
11+
* **Automatic Lifecycle Management**: Lws handles waiting for and "reaping" the terminated child process, providing a callback with its exit status and resource accounting information.
12+
* **Security**: Provides options for setting a `chroot()` jail and the working directory for the child process.
13+
* **Timeout Management**: An optional timeout can be specified to automatically kill a child process that runs for too long.
14+
* **Linux Cgroup Containment**: On Linux, spawned processes can be automatically placed into a new, dedicated cgroup for resource control and isolation. The cgroup is automatically removed when the process is reaped.
15+
16+
## Core API Usage
17+
18+
The primary entrypoint for this API is `lws_spawn_piped()`. It takes a single argument: a pointer to a `struct lws_spawn_piped_info` which you populate to describe the child process you want to create.
19+
20+
Key members of `struct lws_spawn_piped_info`:
21+
22+
- `exec_array`: A `NULL`-terminated array of strings for the executable path and its arguments (equivalent to `argv`).
23+
- `env_array`: A `NULL`-terminated array of strings for the child's environment variables (e.g., `{"VAR1=VALUE1", "VAR2=VALUE2", NULL}`).
24+
- `vh`: The `lws_vhost` the new stdio `wsi` should be associated with.
25+
- `protocol_name`: The name of the lws protocol that will handle events for the stdio `wsi`. This is **mandatory** for proper cleanup.
26+
- `reap_cb`: A **mandatory** callback function of type `lsp_cb_t` that lws will call after the child process has terminated and been reaped.
27+
- `opaque`: A user pointer that will be passed to your `reap_cb` and will also be available on the stdio `wsi` via `lws_get_opaque_user_data()`.
28+
- `timeout_us`: Optional timeout in microseconds. If the process runs longer than this, it will be sent a `SIGTERM`.
29+
30+
### Lifecycle and Cleanup
31+
32+
Proper cleanup is essential. When the child process exits, its stdio pipes are closed by the operating system. This generates a `LWS_CALLBACK_RAW_FILE_CLOSE` event on each of the three stdio `wsi`.
33+
34+
Your protocol handler **must** implement a case for this reason and call `lws_spawn_stdwsi_closed()`:
35+
36+
```c
37+
static int my_spawn_protocol_cb(struct lws *wsi, enum lws_callback_reasons reason, ...)
38+
{
39+
struct my_spawn_state *st = (struct my_spawn_state *)
40+
lws_get_opaque_user_data(wsi);
41+
42+
switch (reason) {
43+
case LWS_CALLBACK_RAW_FILE_CLOSE:
44+
if (st && st->lsp)
45+
lws_spawn_stdwsi_closed(st->lsp, wsi);
46+
break;
47+
/* ... other cases ... */
48+
}
49+
return 0;
50+
}
51+
```
52+
53+
When lws has been notified that all three stdio `wsi` have closed, it will proceed to reap the child process and invoke your `reap_cb`.
54+
55+
## Linux Cgroup Support
56+
57+
On Linux, `lws_spawn` can automatically create a transient cgroup v2 for each spawned process, providing resource isolation. The cgroup is automatically removed when the process is reaped.
58+
59+
### Permissions and One-Time Setup
60+
61+
By default, creating new cgroups in `/sys/fs/cgroup/` requires `root` privileges. A non-root user will get a "Permission Denied" error.
62+
63+
The recommended solution is to have an administrator (or a CI setup script) perform a **one-time setup** to delegate control of a subdirectory to the user running the lws application:
64+
65+
```sh
66+
# Run these commands as root once
67+
sudo mkdir -p /sys/fs/cgroup/lws
68+
sudo chown myuser:mygroup /sys/fs/cgroup/lws
69+
echo "+cpu +memory +pids +io" | sudo tee /sys/fs/cgroup/lws/cgroup.subtree_control
70+
```
71+
72+
For applications that start as `root` and then drop privileges, lws provides a helper function to perform this setup programmatically: `int lws_spawn_cgroup_admin_init(const char *toplevel_name, const char *owner_or_NULL, const char *group_or_NULL);`. Call this function once at startup while your process still has root privileges. The cgroup directory will take on the ownership and group given in the args, NULL means to skip the change. If `toplevel_name` is NULL, it defaults to the builtin one "lws" as the toplevel token. It returns success (0) if the toplevel entry already existed as well as if successfully created.
73+
74+
Typically it will called by a daemon during init while it is still root, and configured to prepare the cgroup dir ownership for the less-privileged user / group it subsequently runs under.
75+
76+
### API Usage
77+
78+
To enable cgroup containment for a spawned process, set the following members in your `struct lws_spawn_piped_info`:
79+
80+
- `cgroup_name_suffix`: A string that will be used to name the cgroup directory. For example, a suffix of `"lws/my-task"` will create a cgroup at `/sys/fs/cgroup/lws/my-task`. This must be unique for concurrent spawns.
81+
82+
- `p_cgroup_ret`: An optional pointer to an `int`. If provided, lws will write `0` to this integer on successful cgroup creation, and `1` on failure.
83+
84+
If cgroup creation fails (e.g., due to permissions), `lws_spawn_piped()` will **not** fail. It will log a warning and proceed to spawn the process without cgroup containment. Use `p_cgroup_ret` to confirm the outcome.
85+
86+
## Example
87+
88+
For a complete, working example that demonstrates these concepts, including cgroup setup and verification, please refer to the API test located at:
89+
90+
`minimal-examples-lowlevel/api-tests/api-test-lws_spawn/`

include/libwebsockets/lws-misc.h

Lines changed: 28 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -1053,6 +1053,8 @@ typedef void (*lsp_cb_t)(void *opaque, lws_usec_t *accounting, siginfo_t *si,
10531053
* \p timeout: optional us-resolution timeout, or zero
10541054
* \p reap_cb: callback when child process has been reaped and the lsp destroyed
10551055
* \p tsi: tsi to bind stdwsi to... from opt_parent if given
1056+
* \p cgroup_name_suffix: for Linux, encapsulate spawn into this new cgroup
1057+
* \p p_cgroup_ret: NULL, or pointer to int to show if cgroups applied OK (0 = OK)
10561058
*/
10571059
struct lws_spawn_piped_info {
10581060
struct lws_dll2_owner *owner;
@@ -1078,6 +1080,9 @@ struct lws_spawn_piped_info {
10781080
const struct lws_role_ops *ops; /* NULL is raw file */
10791081

10801082
uint8_t disable_ctrlc;
1083+
1084+
const char *cgroup_name_suffix;
1085+
int *p_cgroup_ret;
10811086
};
10821087

10831088
/**
@@ -1154,6 +1159,29 @@ lws_spawn_get_stdfd(struct lws *wsi);
11541159
*/
11551160
LWS_VISIBLE LWS_EXTERN int
11561161
lws_spawn_get_fd_stdxxx(struct lws_spawn_piped *lsp, int std_idx);
1162+
1163+
/**
1164+
* lws_spawn_cgroup_admin_init() - Create lws parent cgroup
1165+
*
1166+
* \p toplevel_name: NULL (chooses name 'lws') or the name fragment to
1167+
* try to create.
1168+
* \p user: NULL, or the user name that will use the toplevel cgroup
1169+
* \p group: NULL, or the group name that will use the toplevel cgroup
1170+
*
1171+
* This helper should be called once at startup by a process that has root
1172+
* privileges. It will create and configure the master cgroup directory
1173+
* `/sys/fs/cgroup/<toplevel_name>`.
1174+
*
1175+
* After this has been called successfully, the process can drop privileges
1176+
* to a non-root user, and subsequent calls to lws_spawn_piped() with a
1177+
* cgroup_name_suffix will succeed as long as that user has write permission
1178+
* in the master cgroup directory (which can be arranged via chown).
1179+
*
1180+
* Returns 0 on success. On non-Linux platforms, it's a no-op that returns 1.
1181+
*/
1182+
LWS_VISIBLE LWS_EXTERN int
1183+
lws_spawn_cgroup_admin_init(const char *toplevel_name,
1184+
const char *user, const char *group);
11571185
#endif
11581186

11591187
struct lws_fsmount {

lib/core-net/private-lib-core-net.h

Lines changed: 4 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -926,6 +926,10 @@ struct lws_spawn_piped {
926926
lws_sorted_usec_list_t sul;
927927
lws_sorted_usec_list_t sul_reap;
928928

929+
#if defined(__linux__)
930+
char cgroup_path[256];
931+
#endif
932+
929933
struct lws_context *context;
930934
struct lws *stdwsi[3];
931935
lws_filefd_type pipe_fds[3][2];

lib/core/context.c

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -1757,7 +1757,7 @@ lws_system_cpd_set(struct lws_context *cx, lws_cpd_result_t result)
17571757
return;
17581758

17591759
#if !defined(LWS_WITH_NO_LOGS)
1760-
lwsl_cx_notice(cx, "setting CPD result %s", cname[result]);
1760+
lwsl_cx_info(cx, "setting CPD result %s", cname[result]);
17611761
#endif
17621762

17631763
cx->captive_portal_detect = (uint8_t)result;

lib/plat/unix/unix-spawn.c

Lines changed: 178 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -28,6 +28,13 @@
2828

2929
#include "private-lib-core.h"
3030
#include <unistd.h>
31+
#include <sys/types.h>
32+
#include <pwd.h>
33+
#include <grp.h>
34+
35+
#if defined(__linux__)
36+
#include <sys/stat.h>
37+
#endif
3138

3239
#if defined(__OpenBSD__) || defined(__NetBSD__)
3340
#include <sys/resource.h>
@@ -51,7 +58,7 @@ lws_spawn_sul_reap(struct lws_sorted_usec_list *sul)
5158
struct lws_spawn_piped *lsp = lws_container_of(sul,
5259
struct lws_spawn_piped, sul_reap);
5360

54-
lwsl_notice("%s: reaping spawn after last stdpipe, tries left %d\n",
61+
lwsl_info("%s: reaping spawn after last stdpipe, tries left %d\n",
5562
__func__, lsp->reap_retry_budget);
5663
if (!lws_spawn_reap(lsp) && !lsp->pipes_alive) {
5764
if (--lsp->reap_retry_budget) {
@@ -212,6 +219,21 @@ lws_spawn_reap(struct lws_spawn_piped *lsp)
212219

213220
lws_sul_cancel(&lsp->sul);
214221

222+
#if defined(__linux__)
223+
if (lsp->cgroup_path[0]) {
224+
/*
225+
* The child has been reaped, we can remove the cgroup dir.
226+
* This will only work if the cgroup is empty, which it should
227+
* be now.
228+
*/
229+
if (rmdir(lsp->cgroup_path))
230+
lwsl_warn("%s: unable to rmdir cgroup %s, errno %d\n",
231+
__func__, lsp->cgroup_path, errno);
232+
else
233+
lwsl_info("%s: reaped cgroup %s\n", __func__, lsp->cgroup_path);
234+
}
235+
#endif
236+
215237
/*
216238
* All the stdwsi went down, nothing more is coming... it's over
217239
* Collect the final information and then reap the dead process
@@ -350,6 +372,13 @@ lws_spawn_piped(const struct lws_spawn_piped_info *i)
350372
lsp->info = *i;
351373
lsp->reap_retry_budget = 20;
352374

375+
#if defined(__linux__)
376+
lsp->cgroup_path[0] = '\0';
377+
#endif
378+
379+
if (i->p_cgroup_ret)
380+
*i->p_cgroup_ret = 1; /* Default to cgroup failed */
381+
353382
/*
354383
* Prepare the stdin / out / err pipes
355384
*/
@@ -445,6 +474,31 @@ lws_spawn_piped(const struct lws_spawn_piped_info *i)
445474
lsp->stdwsi[LWS_STDIN]->desc.sockfd,
446475
lsp->stdwsi[LWS_STDOUT]->desc.sockfd,
447476
lsp->stdwsi[LWS_STDERR]->desc.sockfd);
477+
478+
#if defined(__linux__)
479+
if (i->cgroup_name_suffix && i->cgroup_name_suffix[0]) {
480+
lws_snprintf(lsp->cgroup_path, sizeof(lsp->cgroup_path),
481+
"/sys/fs/cgroup/%s", i->cgroup_name_suffix);
482+
483+
/*
484+
* This is the step that requires the process to either be root,
485+
* or for an admin to have delegated control of /sys/fs/cgroup/a
486+
* to the user running the process, if the suffix is "a/b".
487+
*/
488+
if (mkdir(lsp->cgroup_path, 0755)) {
489+
lwsl_warn("%s: Failed to create cgroup %s, errno %d. "
490+
"Continuing without cgroup.\n",
491+
__func__, lsp->cgroup_path, errno);
492+
lsp->cgroup_path[0] = '\0';
493+
/* Do not abort, just clear the path and continue */
494+
} else {
495+
lwsl_info("%s: created cgroup %s\n", __func__, lsp->cgroup_path);
496+
if (i->p_cgroup_ret)
497+
/* Report cgroup success to caller */
498+
*i->p_cgroup_ret = 0;
499+
}
500+
}
501+
#endif
448502

449503
/* we are ready with the redirection pipes... do the (v)fork */
450504
#if defined(__sun) || !defined(LWS_HAVE_VFORK) || !defined(LWS_HAVE_EXECVPE)
@@ -510,6 +564,32 @@ lws_spawn_piped(const struct lws_spawn_piped_info *i)
510564
* process is OK. Stuff that happens after the execvpe() is OK.
511565
*/
512566

567+
#if defined(__linux__)
568+
if (lsp->cgroup_path[0]) {
569+
char path[300], pid_str[20];
570+
int fd, len;
571+
572+
/*
573+
* We are the new child process. We must move ourselves into
574+
* the cgroup created for us by the parent.
575+
*/
576+
lws_snprintf(path, sizeof(path) - 1, "%s/cgroup.procs", lsp->cgroup_path);
577+
fd = open(path, O_WRONLY);
578+
if (fd >= 0) {
579+
len = lws_snprintf(pid_str, sizeof(pid_str) - 1, "%d", (int)getpid());
580+
if (write(fd, pid_str, (size_t)len) != (ssize_t)len) {
581+
/*
582+
* using lwsl_err here is unsafe in vfork()
583+
* child, just exit with a special code
584+
*/
585+
_exit(121);
586+
}
587+
close(fd);
588+
} else
589+
_exit(122);
590+
}
591+
#endif
592+
513593
if (i->chroot_path && chroot(i->chroot_path)) {
514594
lwsl_err("%s: child chroot %s failed, errno %d\n",
515595
__func__, i->chroot_path, errno);
@@ -603,6 +683,21 @@ lws_spawn_stdwsi_closed(struct lws_spawn_piped *lsp, struct lws *wsi)
603683
{
604684
int n;
605685

686+
/*
687+
* This is part of the normal cleanup path, check if the lsp has already
688+
* been destroyed by a timeout or other error path. If the stdwsi that
689+
* is closing has already been nulled out, we have already been through
690+
* destroy.
691+
*/
692+
for (n = 0; n < 3; n++)
693+
if (lsp->stdwsi[n] == wsi)
694+
goto found;
695+
696+
/* Not found, so must have been destroyed already */
697+
return;
698+
699+
found:
700+
606701
assert(lsp);
607702
lsp->pipes_alive--;
608703
lwsl_debug("%s: pipes alive %d\n", __func__, lsp->pipes_alive);
@@ -621,6 +716,88 @@ lws_spawn_get_stdfd(struct lws *wsi)
621716
return wsi->lsp_channel;
622717
}
623718

719+
int
720+
lws_spawn_cgroup_admin_init(const char *toplevel_name, const char *user, const char *group)
721+
{
722+
#if defined(__linux__)
723+
uid_t uid = (uid_t)-1;
724+
gid_t gid = (gid_t)-1;
725+
char path[128];
726+
int cfd;
727+
728+
if (!toplevel_name)
729+
toplevel_name = "lws";
730+
731+
lws_snprintf(path, sizeof(path), "/sys/fs/cgroup/%s", toplevel_name);
732+
733+
/*
734+
* Create the lws parent cgroup directory. This will only succeed if
735+
* the process has sufficient privileges (e.g., root).
736+
*/
737+
if (mkdir(path, 0755) && errno != EEXIST) {
738+
lwsl_warn("%s: Failed to mkdir %s: %s\n",
739+
__func__, path, strerror(errno));
740+
return 1;
741+
}
742+
743+
/*
744+
* Now, change the ownership of the directory to the user/group that
745+
* lws will drop privileges to. This allows the non-root user to
746+
* create sub-cgroups inside.
747+
*/
748+
if (user) {
749+
struct passwd *pwd;
750+
751+
pwd = getpwnam(user);
752+
if (pwd)
753+
uid = pwd->pw_uid;
754+
else
755+
lwsl_warn("%s: user '%s' not found\n", __func__, user);
756+
}
757+
if (group) {
758+
struct group *grp;
759+
760+
grp = getgrnam(group);
761+
if (grp)
762+
gid = grp->gr_gid;
763+
else
764+
lwsl_warn("%s: group '%s' not found\n", __func__, group);
765+
}
766+
767+
lwsl_notice("%s: switching cgroup toplevel owner to %d:%d\n",
768+
__func__, uid, gid);
769+
770+
if (chown("/sys/fs/cgroup/lws", uid, gid) < 0)
771+
lwsl_warn("%s: failed to chown /sys/fs/cgroup/lws: %s\n",
772+
__func__, strerror(errno));
773+
774+
lws_snprintf(path, sizeof(path), "/sys/fs/cgroup/%s/cgroup.subtree_control",
775+
toplevel_name);
776+
777+
/*
778+
* We must enable controllers for the parent before they can be
779+
* used by children. This makes them available for delegation.
780+
* This step might require root.
781+
*/
782+
cfd = lws_open(path, LWS_O_WRONLY);
783+
if (cfd < 0) {
784+
/* May fail if user doesn't own the file, that's okay */
785+
lwsl_info("%s: cannot open subtree_control: %s\n",
786+
__func__, strerror(errno));
787+
return 0; /* Still a success if dir exists */
788+
}
789+
790+
if (write(cfd, "+cpu +memory +pids +io", 22) != 22)
791+
/* ignore, may be there already or fail due to perms */
792+
lwsl_debug("%s: setting admin cgroup options failed\n", __func__);
793+
close(cfd);
794+
lwsl_notice("%s: lws cgroup parent configured\n", __func__);
795+
796+
return 0;
797+
#endif
798+
return 1; /* Not supported on this platform */
799+
}
800+
624801
int
625802
lws_spawn_get_fd_stdxxx(struct lws_spawn_piped *lsp, int std_idx)
626803
{

0 commit comments

Comments
 (0)