Change to async SDR acquire thread

[zuckschwerdt]

This changes the SDR data acquisition to run in a thread.

The idea is: if using live inputs we already depend on threads (vie SoapySDR or libusb). Currently the main event loop "lives" in the SDR callback, this is awkward and the event loop can't run while gathering a frame (which is almost always).

The benefits include:

• reduced nesting (event loop is now top-level)
• event loop is much more responsive and can handle more throughput
• replace crude alarm() signal with fine grained timers
• reduced complexity controlling the SDR (most commands can't be done from a callback)
• truely instant frequency hopping
• we could now easy handle SDR dropouts, the event loop will continue to run.

Thoughts and comments much welcome!

[merbanan]

Well as long as local playback (-r) is single threaded then this is fine. Any embedded platform we target has the required functionality.

[zuckschwerdt]

Yes, this is all with best compatibility in mind. But really I only know that SoapySDR and rtlsdr (via libusb) use threads, so we should be fine.

To quickly summarize the overall architecture: The program will stay single threaded and unaware of threads, just the live inputs will deliver frames as events, replacing the blocking and abuse of the SDR callback to run a (actually very short) event loop.

Rather silly, but that means we could service multiple SDR sources from one program instance ;)
Also we could change the file (non-live) inputs to use the event loop now, but I don't see any obvious benefit.

[gdt]

I'd like to see this finished/merged (but have not read it yet). NetBSD doesn't have async usb read, which is a missing feature to be added, but it does have a streaming ugen read that I added back in 2006 for USRP, where the kernel reads another chunk ahead so that the user process doesn't block. Or rather, as soon as the thing you read is available, the kernel starts the next read and then returns, so that after you do whatever and call read again, there is no gap in read activity on the bus, so you can stream. With a thread doing the reading, it should be easy to use the read-ahead method instead. And yes, I know I should just implement USB async anyway.

[zuckschwerdt]

Broadly speaking this change just runs the acquire loop (to the RTL-SDR or SoapySDR lib) in a thread which uses mg_broadcast() to shuttle received data to the main thread. There is no true concurrency, just the blocking while waiting for a new data is removed.
The main thread (which used to process the data in the acquire callback) is free to process for longer then, without the need to get out of the acquire callback timely enough to continue the acquire loop.

The most visible benefit of this untangling would be the option to start/crash/restart/switch/stop the inputs at leisure.
Currently stopping or crashing the input ends the event loop and thus the application.

[gdt]

I don't follow no true concurrency; this lets the read thread and the process thread be indepdent. I get it that it does not allow paralllel reads or multiple CPUs for decoding.

I agree that this decoupling is good; I see it as a bug that rtl_433 exists with a rtl dongle hiccup rather than retrying. I also think it woudl be easier to add in the NetBSD read method.

I hope mg_broadcast() preserves ordering. Probably doesn't matter 99% of the time, but seems like it should be guaranteed.

[zuckschwerdt]

I should have said: no actual concurrency is desired or purposefully enabled with this. The acquire thread blocks until a buffer is available, then notifies the main thread (which is likely blocked in the event loop) with that data. We expect the processing to usually finish before the acquire thread gets unblocked with the next buffer, but that's not a hard limit anymore.

But as a bonus we could run on smaller CPUs as long as accumulated processing keeps up with the incoming data (e.g. with frames "squelch"ed others can take longer to process). Currently every single frame needs to be faster than the acquire loop.

But really I just wanted to say that this feature goes towards decoupling not a new execution logic model. As you noted decoupling is the value here.

Yes, mg_broadcast() uses a socket pair and thus is a well behaved stream.

[zuckschwerdt]

I've now added the idea of restartable SDR devices to this, quick and dirty hack, but it just works.
Some clean up and refactoring needed, but this feature will land soon now.

[endmarsfr]

Hi Christian,

I encountered a failure with this commit.
Compiling after this commit caused an error with rtl_433 stopping, even when I used -D restart.
I did my test on an Odroid C2 using a fresh DietPi v8.21.1 bookworm.
The previous commit works fine on my device
There are many files changed in this commit.
Could you help me?
Kind Regards

Trying conf file at "rtl_433.conf"... Trying conf file at "/home/dietpi/.config/rtl_433/rtl_433.conf"... Trying conf file at "/usr/local/etc/rtl_433/rtl_433.conf"... Trying conf file at "/etc/rtl_433/rtl_433.conf"... [Protocols] Registered 212 out of 246 device decoding protocols [ 1-4 8 10-12 15-17 19-23 25-26 29-36 38-60 63 67-71 73-100 102-105 108-116 119-121 124-128 130-149 151-161 163-168 170-175 177-197 199 201-215 217-228 230-232 234-241 243-244 246 ] Detached kernel driver Found Rafael Micro R820T tuner [SDR] Using device 1: Realtek, RTL2838UHIDIR, SN: 00000001, "Generic RTL2832U OEM" Exact sample rate is: 250000.000414 Hz [R82XX] PLL not locked! Allocating 15 zero-copy buffers [Input] Input device start failed, exiting! Reattached kernel driver