You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
refactor: streamline Android display streaming around shared video (#76)
* feat: Enhance Android H.264 streaming support and refactor related components
- Updated StreamQualityLimits struct to be cloneable and public.
- Introduced stream_quality_limits_for_payload function to resolve quality limits from payload.
- Refactored Android H.264 socket handling to utilize new quality resolution logic.
- Renamed native encoder function to encode BGRA frames instead of RGBA.
- Removed deprecated RGBA WebRTC transport handling in favor of H.264.
- Implemented shared video frame encoding for Android using native H.264 encoder.
- Improved error handling and logging for stream quality updates and encoding failures.
- Updated SimDeck documentation to reflect changes in Android live viewing capabilities.
* fix: improve android display streaming
* fix: harden android and webrtc ci
* fix: stabilize simulator integration retry
* fix: retry webrtc reference screenshot in ci
|`GET`|`/api/simulators/{udid}/input`| Input WebSocket fallback for controls |
172
+
|`GET`|`/api/simulators/{udid}/control`| Alias for input control WebSocket |
173
+
|`POST`|`/api/simulators/{udid}/refresh`| Request a fresh frame or keyframe |
174
+
175
+
For normal clients, copy the browser behavior instead of hand-coding a raw decoder. The UI uses the WebRTC offer endpoint for live video. Android emulator IDs use the same WebRTC endpoint; their H.264 frames are produced from the emulator `-share-vid` display surface, not screenshot polling.
Copy file name to clipboardExpand all lines: docs/guide/architecture.md
+1-1Lines changed: 1 addition & 1 deletion
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -26,7 +26,7 @@ This is why a long-lived service feels faster than repeatedly calling lower-leve
26
26
27
27
## Video flow
28
28
29
-
The browser opens a live stream for the selected device. SimDeck sends fresh frames, drops stale ones when a client falls behind, and lets the browser request refreshes. The UI can use WebRTC or H.264-over-WebSocket fallback depending on browser support and network behavior.
29
+
The browser opens a live WebRTC stream for the selected device. SimDeck sends fresh frames, drops stale ones when a client falls behind, and lets the browser request refreshes. iOS frames come from the native display bridge and are encoded on the Mac; Android frames come from the emulator `-share-vid` shared display surface and are encoded on the Mac.
If `frames_dropped_server` keeps climbing, the client or network cannot keep up. Move closer to the host, reduce quality, or switch to software encoding.
139
139
140
+
For Android emulator streams, SimDeck uses the emulator `-share-vid` shared display surface. If Android video never starts, confirm `adb devices` shows the emulator as `device`, that it has fully booted, and that externally launched emulators were started with `-share-vid`. SimDeck-owned Android boots add the flag automatically and default to `--android-gpu host`; try `simdeck service restart --android-gpu auto` or `--android-gpu swiftshader_indirect` only when host rendering is unstable.
141
+
140
142
### Browser cannot establish WebRTC
141
143
142
-
Force the H.264 WebSocket fallback while testing:
144
+
Use `?stream=webrtc` while testing to make transport selection explicit:
143
145
144
146
```text
145
-
http://127.0.0.1:4310?stream=h264
147
+
http://127.0.0.1:4310?stream=webrtc
146
148
```
147
149
148
150
For routed remote sessions, configure TURN as described in [Video and streaming](/guide/video#remote-browsers).
Copy file name to clipboardExpand all lines: docs/guide/video.md
+31-8Lines changed: 31 additions & 8 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -1,14 +1,19 @@
1
1
# Video and streaming
2
2
3
-
SimDeck streams live device video to the browser. Local sessions default to high quality. Remote or constrained sessions can trade detail for lower CPU and latency.
3
+
SimDeck streams live device video to the browser. Local sessions default to full-resolution 60 fps. Remote or constrained sessions can trade detail for lower CPU and latency.
4
4
5
5
iOS simulator H.264 uses VideoToolbox for hardware encoding and x264 for software encoding.
6
+
Android emulator H.264 uses the emulator `-share-vid` display surface. SimDeck reads BGRA frames from the `videmulator<console-port>` shared memory region and encodes them on the Mac, so normal Android live video stays on the native shared display path.
6
7
7
8
## When encoding runs
8
9
9
-
SimDeck starts encoding when a browser stream needs H.264 frames. The server
10
-
requests an initial keyframe to answer the WebRTC or H.264 WebSocket viewer,
11
-
then keeps a shared refresh pump active while frame subscribers exist.
10
+
SimDeck starts encoding when a browser stream needs H.264 frames. For iOS, the
11
+
server requests an initial keyframe to answer the WebRTC viewer, then keeps a
12
+
shared refresh pump active while frame subscribers exist.
13
+
For Android, SimDeck starts emulators with `-share-vid`, maps the shared display
14
+
region, and feeds changed BGRA frames into the native host H.264 encoder.
15
+
SimDeck-owned Android boots also default to `-gpu host`, matching the native
16
+
emulator app's accelerated renderer while staying in headless shared-video mode.
12
17
13
18
The browser reports whether the page and stream canvas are foreground. When all
14
19
known viewers are hidden or the last frame subscriber disconnects, the native
0 commit comments