Repository navigation
[RFC] Wayland sessions: wlup module for sway and GNOME, started by sesman - #3904
Open
congzhangzh wants to merge 61 commits into
Open
congzhangzh wants to merge 61 commits into
congzhangzh wants to merge 61 commits into
Conversation
This was referenced Oct 10, 2026
Contributor
Author
Tested on a real host with mstsc
Until now this was only tested in containers with FreeRDP. Today I installed it on a real Debian testing host (no GPU) and used it from Windows' own Remote Desktop client (mstsc), at 2560x1440 over a link of about 160 ms round trip. Everything below was checked in that session, mostly from the server's logs.
Not checked on the real host: resizing (the client window stayed at one size; the GNOME suite covers it), and a local login of the same user at the same time (#2491, still open). Two new commits61cfcc1 wlup: let the RDP client draw GNOME's pointer. Over 160 ms the pointer trailed the mouse, because it was drawn into the frames: every movement waited for input → new frame → encode → network. The GNOME backend now asks Mutter for the cursor as metadata (
cb8183a wlup: add make-deb.sh to build a test package. For trying this on a Debian or Ubuntu host and removing it cleanly ( CIAll three suites pass again (the GNOME suite has 30 checks now): https://github.com/congzhangzh/xrdp/actions/runs/38069317254 Observations, not blocking
|
congzhangzh
force-pushed
the
wayland-wlup
branch
from
October 13, 2026 23:23
cb8183a to
a2f87bc
Compare
Contributor
Author
|
Rebased onto current devel, and added:
Over a 4 Mbit/s link with 80 ms each way and half of the screen changing, a typed change used to take more than 20 s to show with bitmap updates while xrdp grew by about 2 MB/s; now it shows after about 2 s with flat memory, and nothing is sent while the client window is minimised. The GFX planar path also needs xrdp-side changes, which I am sending as separate PRs. run-all.sh: sway 22/0, sway with systemd 31/0, GNOME 36/0. |
Adds an optional backend module (--enable-wlup) that connects xrdp to a running Wayland compositor as an ordinary Wayland client, instead of to Xorg/xorgxrdp: - frames: ext-image-copy-capture-v1 into a wl_shm buffer; only the regions the compositor reports as damaged are painted - keyboard: zwp_virtual_keyboard_v1 with an XKB keymap from libxkbcommon; RDP scancodes are mapped to evdev with the existing scancode tables, and the modifier state is sent explicitly - mouse: zwlr_virtual_pointer_v1, absolute motion, buttons and wheel These protocols are offered by wlroots-based compositors (sway, labwc). There is no session management or authentication yet: the module attaches to the compositor named by wayland_display in xrdp.ini. wlup/test contains a container test bed which runs headless sway, connects xfreerdp3 to xrdp and checks that typed input runs in the Wayland session.
xrdp passes its stored pointer position with wheel events. That position is only updated by move PDUs, not by the coordinates of button PDUs, so after a click without a preceding move it is stale, and moving the pointer there sent the scroll to the wrong window. Scroll goes to the surface under the pointer, so wheel events now send only the axis. The test bed now runs wev next to foot (focus_follows_mouse off) and checks the left/right/middle button and both wheel directions arrive in the Wayland session.
Used to check what a headless gnome-shell offers to a Wayland client. It needs systemd/logind: run the image with systemd as PID 1.
Uses wlr-output-management to give the captured output a custom mode: on connect, so the output matches the size the client asked for, and on a dynamic resize. A resize reports in_progress to xrdp and calls server_monitor_resize_done() once the capture session delivers a buffer of the new size. Every head is listed in the configuration, as the protocol requires. The test bed starts sway at 1024x768, connects at 1280x800 and then resizes the client window to 1600x900, checking the output size and that input still works afterwards. It uses 24 bpp by default: at 32 bpp FreeRDP's client keeps its planar decoder at the initial width after a deactivate-reactivate resize and disconnects.
XFREERDP selects the client binary. wlup/test/freerdp builds an image with FreeRDP's build dependencies, used to A/B test a FreeRDP fix for resizing at 32 bpp.
SCP_SESSION_TYPE_WAYLAND is selected with code=30 in xrdp.ini. xrdp asks sesman for such a session like for Xorg, and passes the module the display server connection sesman returns instead of a port.
After the PAM login, sesexec runs the command in the new [Wayland] section of sesman.ini as the user (default: startwayland.sh, which runs sway headless at the client's size). The compositor plays the part of the window manager: the session ends when it exits. Compositors pick their own socket name, so sesexec finds it from the "wayland-N.lock" file libwayland keeps locked: the child reports its XDG_RUNTIME_DIR (from the PAM environment, or the xrdp socket directory if there is none) over a pipe, and the lock held by the compositor's PID is looked up in /proc/locks. This needs no ptrace rights, unlike reading /proc/<pid>/fd. On each connect sesexec, which is privileged, connects to the socket and hands the connection to xrdp as the display server fd. No chansrv is started for Wayland sessions yet.
If xrdp passes a connected fd (session from sesman), use it with wl_display_connect_to_fd(). wayland_display= is still used otherwise.
The test now starts xrdp-sesman, logs in as 'tester' with a password, and lets sesman start sway through startwayland.sh. The user's sway config runs foot and wev. A new check reconnects and expects the same sway process and working input.
wlup/test/systemd builds the test bed with systemd, libpam-systemd and dbus. Run as PID 1 in a privileged container, sesman's PAM login goes through pam_systemd, and the test then also checks that sway runs in a logind session of the user, with XDG_RUNTIME_DIR=/run/user/<uid> and its Wayland socket there.
GNOME does not offer the wlroots capture and virtual input protocols to Wayland clients. With backend=mutter, wlup reaches the same functions through Mutter's D-Bus APIs on the session bus: - RemoteDesktop.CreateSession, then ScreenCast.CreateSession tied to it - ScreenCast.Session.RecordVirtual adds a virtual monitor; its size is the size wlup prefers when it connects to the PipeWire stream - frames arrive as shared-memory PipeWire buffers (BGRx/BGRA); only the regions in the video damage metadata are copied and painted - input goes through libei on the fd from RemoteDesktop.Session.ConnectToEIS The sd-bus connection, the PipeWire loop fd and the libei fd all go into xrdp's poll loop, so no extra threads are needed. sd-bus was chosen as it maps directly onto such a loop, and Mutter needs systemd anyway. Built with --enable-wlup-mutter (libsystemd, libpipewire-0.3, libei-1.0). Not done yet: resizing, sesman integration, cursor as RDP pointer.
Runs headless gnome-shell 50 under systemd (PID 1 in the container), xrdp with backend=mutter as the session user, and xfreerdp3 in Xvfb. Checks the PipeWire stream size, the EIS devices, the picture, typing into foot, and buttons and wheel through wev.
SCP_SESSION_TYPE_GNOME is selected with code=31 in xrdp.ini. Like a Wayland session, xrdp asks sesman for it and passes the module what sesman returns instead of a port. A GNOME session needs two connections: the session bus, which goes in the display server fd, and the user's PipeWire daemon. The SCP connect response gets an optional third fd for the second one, and xrdp gives it to the module with the display_aux_fd parameter before connecting.
After the PAM login, sesexec runs the command in the new [GNOME] section of sesman.ini as the user (default: startgnome.sh, which sets the variables a display manager sets for a GNOME Wayland session and execs gnome-session). gnome-session leads the session; it starts GNOME Shell through the user's systemd instance. The logind session has no seat, so GNOME Shell runs headless by itself. On each connect, sesexec connects to the user's session bus and PipeWire sockets from a child process running as the user, and hands both connections to xrdp. A Unix socket keeps the credentials of the process that connected, so the D-Bus daemon and PipeWire see the user, while sesexec and xrdp stay privileged and never use the user's runtime directory themselves. The [Wayland] and [GNOME] sections share one reader for command lines. No chansrv is started for GNOME sessions yet.
With sesman, the Mutter backend gets a connection to the session bus as the display server fd and one to PipeWire as display_aux_fd, both made as the session user. sd-bus authenticates with EXTERNAL without claiming a uid, so the bus sees the user even though xrdp is privileged; PipeWire takes its connection with pw_context_connect_fd(). GNOME Shell may still be starting when xrdp connects, so wait up to a minute for org.gnome.Mutter.RemoteDesktop on the bus. sd-bus works on a copy of the bus fd, as xrdp closes the original if connecting fails.
The Mutter test now runs xrdp and sesman as root, logs in with a password, and lets sesman start gnome-session (gnome-session is added to the image). It checks that GNOME runs as the user in a remote logind session, that GNOME Shell chose headless mode itself, that xrdp is not running as the user, and that a reconnect reaches the same GNOME Shell with working input.
Offering a new preferred size in the PipeWire stream's EnumFormat renegotiates the format, and Mutter resizes the virtual monitor to it. A resize reports in_progress to xrdp and finishes when the stream format of the new size arrives; if Mutter does not answer within 5 seconds, the resize is finished anyway and the picture is clipped. wlup's resize completion is now shared by both backends.
GNOME's pointer starts at (0,0), its hot corner, and the first motion of the remote pointer then opens the Activities overview, which takes the keyboard. Move the pointer to the middle of the screen as soon as the EIS pointer can be used.
The client now resizes from 1280x800 to 1600x900 and the test checks the virtual monitor and the stream follow, and that input still works. The test no longer turns hot corners off.
Sesman sessions always used chansrv, so channel data never reached the module. Wayland and GNOME sessions have no chansrv; their module handles the channels it supports itself, as the VNC module does when it is used without sesman.
chansrv's clipboard works with X11 selections, so wlup speaks [MS-RDPECLIP] itself, following the VNC module. The session side is UTF-8 text, converted to and from the client's CF_UNICODETEXT (UTF-16, CR LF line ends): - wlroots: ext-data-control-v1. A new selection is read through a pipe once events are dispatched, and not while our own source holds it, as the compositor would then ask us to write to that same pipe. - Mutter: RemoteDesktop.Session EnableClipboard, SetSelection and SelectionWrite for the client's text, SelectionOwnerChanged and SelectionRead for the session's. GNOME Shell 50 sends the mime-types option as "(as)", not the documented "as"; both are accepted. The client's text also becomes the copy of the session clipboard, so it is not announced back to the client when the session offers it.
clipboard-tests.sh is shared by both tests. A two-line text with non-ASCII characters is copied in the session with foot's OSC 52 and read from the client's X clipboard with xclip, then set on the client with xclip and pasted into foot. xclip is added to the images.
Wayland and GNOME sessions now have chansrv for audio, drives and the other channels, but its clipboard needs X11, so wlup provides the clipboard. Instead of sending all channels to either chansrv or the module, xrdp asks whether the module owns a channel: in Wayland and GNOME sessions it owns cliprdr. Its data goes to the module, the module may send on it, and chansrv is not told about it. This replaces the previous change, which turned chansrv off for these sessions altogether.
The session's display name is now "wlup-<pid>" or "gnome-<pid>", from the PID of the process leading it, instead of the compositor's socket name, which is not known before the compositor runs. chansrv takes the name from WAYLAND_DISPLAY when there is no X display, and names its sockets after it. The leading process sets XRDP_PULSE_SINK_SOCKET and XRDP_PULSE_SOURCE_SOCKET from its own PID before exec, so the session's audio module (pipewire-module-xrdp) finds chansrv's audio sockets. GNOME loads that module from its XDG autostart entry; startwayland.sh loads it for sway, which does not run autostart entries.
The tests play three seconds of audio into the session's xrdp sink with pw-cat, and check that chansrv runs, that PipeWire has the xrdp sink, and that the client received Wave PDUs (FreeRDP's rdpsnd debug log). The sway test does this when systemd is PID 1, as the user's PipeWire comes from systemd --user; the systemd image gets PipeWire and pipewire-module-xrdp.
A Wayland compositor has no colour depth of its own: wlup serves a client of any bpp from the same session. With the default policy (UB) a client asking for another bpp than the session was started with got a new session instead, and for GNOME that second session of the same user fails at once. Clients do change bpp between connections: a GFX client connects at 32 bpp, the same client without GFX at 24. Ignore the B policy for Wayland and GNOME sessions.
wlup painted with server_paint_rect(), so xrdp sent its frames as planar bitmaps every 40 ms: the H.264 encoder was created but never got a frame, and nothing slowed the frames down on a slow link. Over a 160 ms link a playing video filled the link, and sound and input waited behind it. When xrdp has chosen GFX H.264 for the client (capture code CC_GFX_A2), frames now go the way xorgxrdp sends them: - the damage collected since the last frame is converted to NV12 (BT.709, full range, XRDP_nv12_709fr) in a new anonymous mapping. Only the damaged rectangles, widened to even coordinates, are filled in, as the encoder copies just those into its own picture - STARTFRAME, WIRETOSURFACE_1 (AVC420) and ENDFRAME go to server_egfx_cmd(), which hands the mapping to the encoder thread and unmaps it once encoded - the next frame waits for mod_frame_ack(), which xrdp sends once the frame is encoded and the client is at most frames_in_flight frames behind. Damage keeps collecting meanwhile, so a slow link gets fewer, newer frames instead of a queue A lost ack (xrdp drops frames while it resizes) is given up after 2 s, and a resize starts over with a full frame. Other clients keep using server_paint_rect().
Build with x264 and connect a third time with a 32 bpp GFX AVC420 client. It must reach the same session, its frames must go through the H.264 encoder and be acknowledged without a timeout, and typing must work. In the GNOME suite, a band of the desktop must look the same through H.264 as through planar (PSNR > 35 dB, measured 43.8), which catches a wrong colour matrix or chroma layout.
The version was <date>.git<hash>, so two builds from the same day were ordered by their hashes: a newer commit could sort lower, and apt refused to install it as a downgrade. Use the commit time to the second instead of the date.
A client that minimises its window, or that is in the middle of a deactivate-reactivate, sends Suppress Output. Logging it makes it visible why the client gets no frames, and lets a test check that the request reached wlup.
Does wlup send data nobody needs, and does a busy screen on a slow link pile up frames that delay everything after them? xrdp queues what the socket does not take in user space without a limit, so too many frames do not show as more bytes on the wire but as a growing delay before the client sees a change, and as a growing xrdp process. run-backpressure.sh runs xfreerdp3 against xrdp + wlup + headless sway over lo, which tc netem turns into a 4 Mbit/s link with 80 ms each way and a 100 packet queue (lo gets a 1500 byte MTU and no GSO, so the queue holds 150 KB, not megabytes). A terminal printing random text keeps half of the screen changing. Typing a command in another terminal sets its background to the next of five colours; the time from the keystroke to that colour showing in the client window, read back from Xvfb, is the delay a backlog makes longer. For each way frames go to the client (GFX H.264, GFX planar, bitmap updates) it measures the bytes sent while the screen is still, the delay, xrdp's size and CPU and the send queue on the slow link, the delay of the first change once the link is fast again, the bytes sent while the client window is unmapped (Suppress Output), and how soon the client shows a change made meanwhile once it is mapped again. It needs CAP_NET_ADMIN and the host's sch_netem module, so it is not part of run-all.sh. The image gets iproute2 for tc and ss.
wlroots captures went on whatever the client did with them: the next frame was asked for as soon as one was ready. With output suppressed (the client window minimised), every frame was copied and thrown away. Without the H.264 encoder, every frame was painted, and xrdp queued what the network did not take without a limit: over a 4 Mbit/s link with an 80 ms delay each way and a terminal printing random text over half of the screen, xrdp grew by 2 MB/s, a change typed in had not shown after 20 s, and with the client window minimised xrdp went on sending the backlog. Ask for the next frame only if it would be used: not while output is suppressed, as the compositor collects the damage and the first capture after the resume carries it, and not while xrdp still holds more than 50000 bytes of what wlup painted, as neutrinordp does. For that, what xrdp queues while it runs wlup counts as wlup's (XRDP_SOURCE_MOD in source_info); lib_mod_check_wait_objs() runs again as the queue drains, and goes on capturing. In the same test xrdp's size stays flat, a change shows after 2.2 s (0.7 s with net.ipv4.tcp_notsent_lowat=131072, as the rest waits in the kernel's send buffer), and a minimised client gets nothing.
send_frame() converted the capture buffer to NV12 when xrdp acknowledged the last frame, but by then frame_ready() had already attached the buffer to the next capture: the compositor may write to it from then on, while wlup reads it. ext-image-copy-capture-v1 leaves the buffer to the client only between ready and the next capture. wlroots captures now copy the damaged rectangles, widened to even coordinates as send_frame() converts them, to a copy of the buffer in frame_ready(), and frames are made from that copy. Every capture's damage is copied when it arrives and none arrives while output is suppressed, so the copy is up to date whenever a frame is sent. If xrdp has acknowledged the last frame, the damage of a capture goes out in frame_ready(), before the next capture starts. Mutter's pixels are wlup's own already. The capture no longer has to wait for frame acks, so a frame carries the newest picture: over a 4 Mbit/s link with an 80 ms delay each way and a terminal printing random text over half of the screen, a change shows after 580-770 ms (median of three runs), against 780 ms before and 1010 ms with the capture paused until the ack. It costs a copy of the buffer (4 MB at 1280x800) and the CPU of copying damage that the next frame replaces.
xrdp's own Xorg sessions use the same encoder and frame acks in xrdp as wlup's H.264 frames. To compare the two, and to catch changes in xrdp that slow down or stall Xorg sessions, run-backpressure.sh takes BP_SESSION=xorg: in the new xrdp-wlup-xorgxrdp image it builds xorgxrdp against the xrdp under test, so both speak the same protocol, and runs an Xorg session (no window manager) whose probe and busy terminals are xterms instead of foot. What the two sessions do differently is in a few functions; the measurements are shared. The image adds the Xorg server and xorgxrdp's source at a fixed commit. sesman runs /usr/lib/xorg/Xorg directly, as Debian's wrapper only lets console users start X. xorgxrdp logs Suppress Output only at TRACE level, so for Xorg the hidden-client check runs without seeing it. Over a 4 Mbit/s link with an 80 ms delay each way, an Xorg session on xrdp devel shows a typed change after 760 ms with H.264, 1.3 s with RFX and 2.0 s with bitmap updates, with flat memory, and sends nothing while the client window is unmapped.
Contributor
Author
|
Hope some day, we can focus on improve xrdp (we have already pick it, so will be truth someday |
Experiment for the design proposed in neutrinolabs#2637: the module keeps one interface towards xrdp, and what each compositor family needs sits behind struct wlup_backend_ops (wlup_backend.h): - wlup.c, the core: the module entry points, painting and the GFX H.264 frames, the clipboard channel, keyboard scancodes, and whether frames are wanted at all (suppressed output, output queued for the client) - backends/wlroots: Wayland protocols (ext-image-copy-capture, virtual keyboard and pointer, wlr-output-management), moved out of wlup.c - backends/mutter: D-Bus, PipeWire and libei, moved from wlup_mutter.c and reached through the same table The 16 places where wlup.c asked "which backend?" are now calls through the table. Behaviour is meant to be unchanged; the wlroots state still lives in struct wlup for now.
The Wayland objects, capture state, output heads and xkb state of the wlroots backend move from struct wlup to struct wlroots, in backends/wlroots/wlroots.h, which only the backend's files include. struct wlup keeps the core's state and the desktop image the backends fill in, and a backend_data pointer for each backend's own state, which the Mutter backend now uses too. The core keeps v->ops only after a successful connect, so no other op runs on a backend whose connect failed; that failure calls disconnect to release what was made.
wlup_clip.c, the backend-neutral clipboard, still chose between wlup_mutter_set_clipboard_text() and wlup_wlr_set_clipboard_text() itself. That is now the optional set_clipboard_text op, and each backend's clipboard functions are declared in its own header, so no header of the core names a backend.
Moving the state to struct wlroots left a check of v after WLR(v) had already used it (cppcheck: nullPointerRedundantCheck). The core only calls the ops of a connected backend; checking w keeps the guard meaningful.
wlup_backend.h makes the core testable without a compositor. tests/wlup links the core as built for libwlup (wlup.lo, wlup_gfx.lo, wlup_clip.lo) with a backend that only records calls, defined under the name the core looks up by default, and a mock xrdp side. The tests check what the core promises a backend: connect and disconnect, no op after a failed connect, when frames are wanted (suppressed output, xrdp's queue to the client), asking for frames again after a frame ack and after events, painting clipped to the desktop image and repainting all of it on resume, keys as evdev codes, pointer events, and the resize handshake with xrdp. Built only with --enable-wlup.
The backends and tests/wlup include the module's headers by name, as the build adds -I for wlup; cppcheck needs the same, or it reports them as missing includes and fails.
trans_write_copy_s() queued what the socket did not take by walking the queue to its end for every PDU, so queueing n PDUs took O(n^2) steps. With a slow client and a busy screen sent as GFX planar tiles, one PDU per 64x64 tile, the queue reached many thousands of PDUs and xrdp spent all of its CPU there (gdb: 6 of 6 samples at the loop), so it also stopped reading the client's input. A pointer to the last stream makes appending constant time. trans_delete() also freed nothing of the queue, and left its bytes counted for their source in source_info. It now frees the queue and takes its bytes off their sources. Over a 4 Mbit/s link with 80 ms each way and half of the screen changing (GFX planar, without "xrdp_mm: don't flush the GFX screen while the client is behind"), xrdp now uses 16% CPU instead of 100%, and input and Suppress Output get through. As xrdp is no longer held back by the loop, the queue then grows much faster (380 MB in 20 s instead of 13 MB), which that change prevents. (cherry picked from commit c19bd9aa3dee5c65d90ac847a8114b57d6c74305)
Damage up to 64 rectangles and then their bounding box, a frame only once the one before is acknowledged and the waiting damage at once on the ack, a lost ack not stopping the frames for good and the timeout that notices it, acks beyond the last frame, reset, and the copy for the frames widened to even coordinates for the NV12 conversion.
The tests play the RDP client: they send [MS-RDPECLIP] PDUs to wlup_clip_process_channel_data() and look at the PDUs wlup sends back and at the text the mock backend is given. They cover the exchange starting when the channel opens, text from the client reaching the session as UTF-8 (CR LF as LF, a character outside the BMP from its surrogate pair), text from the session reaching the client as UTF-16 (LF as CR LF), unchanged session text being announced once, a PDU in two channel chunks, and wlup_clip_read_fd().
The build workflow does not build wlup, so tests/wlup ran nowhere in CI. The wlup workflow now runs them too, in its own job, and also when only tests/wlup changes. The test image gets check and cmocka for that.
When encoding a WireToSurface1 frame failed, the frame was dropped without a word. If x264 encoded the picture but the output did not fit, the frames after it are predicted from a picture the client never gets, so the client can show wrong pixels until a key frame. Log a warning with the encoder's error and the room it had.
Once in four GNOME runs over a slow link, H.264 frames kept going out but a change in one part of the screen did not show for 20 s. These show whether such damage reached wlup and went out in a frame.
wlup used H.264 only when the client had one monitor: xrdp makes a GFX surface per monitor and keeps an H.264 stream for each, but wlup sent the whole desktop to surface 0. With more monitors it fell back to planar bitmaps. Send the damage as a frame per monitor instead, as xorgxrdp does: the monitor's index as the surface id and in the top 4 bits of the WIRETOSURFACE_1 flags, its damage relative to it, and an NV12 picture the size of the monitor. The monitors are read from the client info at the time of each frame, with the non-negative coordinates xrdp made the surfaces from (minfo_wm). The next update waits for the ack of the last frame. Damage between monitors is dropped rather than kept, as no surface shows it. Unit tests check the commands for two monitors: one frame per damaged monitor, the surface id, the flags, the rects relative to the monitor, the pixels, and the ack of the last frame.
Connect the client from an Xorg with two 800x600 monitors side by side (dummy driver, Xinerama, RandR off so that FreeRDP finds both; Xvfb puts its Xinerama screens on top of each other). Check that sway's output spans both, that xrdp maps a surface per monitor and wlup sends frames for both, and that a focus change shows on both: the title bars must swap colours on the left and the right monitor. H.264 is lossy, so the colours are compared with a tolerance.
The message for a compositor without ext-image-copy-capture-v1 only reached the login window, so xrdp.log gave no reason why a session on sway 1.10 (wlroots 0.18) or older ended at once. Log the missing globals as an error, with the wlroots version that has them.
A compositor that could not be run (for instance exec failing with EPERM, or a missing library) or that gave up at once only showed as "Timed out waiting for the Wayland compositor" after 10 seconds. Log why exec failed, wait for the compositor without blocking while looking for its socket, and log its exit status or signal as soon as it ends. It has then been waited for, so the caller no longer signals and waits for its PID.
…tory The version came from HEAD's commit time. A branch rebuilt by cherry-picking with the original commit times can keep the time of the build before while its hash changes, and the hash does not order: apt then sees the newer package as a downgrade. Use the newest commit time in the history, which a new, amended or rebased commit moves forward.
A start-up script such as startwayland.sh execs the compositor; when that exec fails (permissions, file capabilities the session lacks, a missing program) the shell exits with 126 or 127 and its error goes to the script's standard error, not to the log. Say what the status means.
congzhangzh
force-pushed
the
wayland-wlup
branch
from
October 19, 2026 14:39
a2f87bc to
f361d2a
Compare
Contributor
Author
|
A messy but workable sample for now, if anyone wants to try it:
Not asking for review, just something to play with :) |
This branch has not been deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
This adds Wayland sessions to xrdp, for a wlroots compositor (sway) and for GNOME, as discussed in #2637. It is a working prototype: the design is the part I'd most like feedback on, and I'm happy to split, squash or rework it in whatever shape suits review (the history is the prototype's history, not a reviewable series yet).
Overview
A new backend module,
wlup, loaded likexup/vnc, and two new session types started by sesman:Wayland(code=30): swayGNOME(code=31): Mutterext-image-copy-capture-v1, damaged regions onlyRecordVirtual→ PipeWirezwlr_virtual_pointer_v1,zwp_virtual_keyboard_v1ConnectToEIS→ libeiwlr-output-managementcustom modeext-data-control-v1startwayland.sh→sway(headless backend)startgnome.sh→gnome-session(GNOME Shell goes headless itself: no seat)Audio, microphone and drive redirection go through chansrv as today: xrdp now routes channels one by one, giving
cliprdrto the module (chansrv's clipboard needs X11) and everything else to chansrv.How the session is connected
pam_systemdthis gives a normal logind session (Remote=yes,Type=wayland,Service=xrdp-sesman) and/run/user/<uid>.display_fdof the SCP connect response; wlup uses it withwl_display_connect_to_fd(). xrdp needs no access to/run/user/<uid>. sesexec finds the socket through thewayland-N.lockheld by the compositor's PID (/proc/locks, Linux only).initgroups/setgid/setuid), connects to$XDG_RUNTIME_DIR/busandpipewire-0, and passes both fds back. The bus and PipeWire see the user's credentials while xrdp keeps running as root. SCP gets an optional extra fd for this.mod_get_wait_objs(Wayland fd; or D-Bus, PipeWire and EIS fds). No threads.New build options:
--enable-wlup(libwayland-client, libxkbcommon) and--enable-wlup-mutter(libsystemd for sd-bus, libpipewire-0.3, libei-1.0).Testing
wlup/test/run-all.shbuilds container images and runs three end-to-end suites in parallel: headless compositor, sesman + xrdp, andxfreerdp3in Xvfb driven byxdotool. The checks are objective: a typed command must write a file inside the session,wevmust report the buttons and wheel, clipboard text must match byte for byte, a file must cross the redirected drive both ways, the client must receive audio, and a 440 Hz tone from the client's microphone must be recorded inside the session./dev/fuse).github/workflows/wlup-e2e.ymlruns the same script in GitHub Actions. Latest run, on cb8183a: https://github.com/congzhangzh/xrdp/actions/runs/38069317254. Its artifact holds each suite's logs and screenshots of the client's screen.It has also been tested on a real Debian testing host with mstsc: clipboard, sound, microphone, drive redirection and reconnecting all work there (the picture is still sent as planar, not H.264), see #3904 (comment).
wlup/make-deb.shbuilds a test package for trying it on Debian or Ubuntu.xrdp's own
build and testworkflow passes on the fork too (formatting with astyle 3.4.14, cppcheck, distcheck, the Linux builds and three FreeBSD builds), except FreeBSD 15.0 with LibreSSL. That job fails at link time withundefined reference: recallocarray@FBSD_1.9in/usr/local/lib/libssl.so, which also happens on other recent PRs, so it looks unrelated to this change. #3905 moves that job to FreeBSD 15.1, which fixes it.The test builds use
CFLAGS=-Wno-error=nonnull: GCC 16 reports-Wnonnullin existing code inxrdp/xrdp_mm.c, unrelated to this change.Direction
The aim is CPU-only servers: no GPU needed. That is why
wlupkeeps frames in shared memory and uses the compositor's damage information. A later software video encoder (H.264/AV1) still encodes whole frames, but damage lets it skip unchanged frames entirely and limit colour conversion to changed tiles.Known gaps
systemd --user, so a remote and a local GNOME session of the same user would clash ([RFE] enforce single graphical session on systemd platforms #2491). This needs a decision.server_paint_rect, so xrdp sends them as planar bitmaps, not through the GFX encoder (H.264), andmod_frame_ackis ignored, so nothing paces them on a slow link. Moving toserver_paint_rects_exwith frame acks, as xorgxrdp does, is in progress.Related: GNOME's
SelectionOwnerChangedsends"mime-types"as(as)rather than the documentedas; wlup accepts both (reported as https://gitlab.gnome.org/GNOME/mutter/-/work_items/5116).