acarsdec next? :-) #117
acarsdec next? :-) #117
Conversation
- avoid special-casing IPv6 (AF_UNSPEC already handles it) - avoid keeping multiple copies of the Rawaddr parameter - avoid redundant (and useless) error checks - avoid redundant checks in Netout*() - avoid running strlen() over buffer when snprintf() is guaranteed to return the buffer size - avoid duplicating the prepared JSON buffer, send it natively instead (tested working with airframes.io) - use static buffers for Netoutpp() and Netoutsv() to reduce stack pressure
Fixes: TLeconte#112
This fixes the following problem: Found 3 device(s): 0: Realtek, RTL2838UHIDIR, SN: 00000003 1: Realtek, RTL2838UHIDIR, SN: 00000002 2: Realtek, RTL2838UHIDIR, SN: 00000001 `acarsdec -r 00000002` would select the device with serial 00000001 `acarsdec -r 00000001` would select the device with serial 00000002 This patch disambiguates explicit serials by skipping matching device ID number when the passed argument is exactly 8 characters long, like a fully qualified serial. This patch is designed to be as short a diff as possible while preserving existing behaviour for all but 8-character long arguments.
- move common global variables out of #define scope - exit with an error if more than one input is selected - reformat the usage help text - rename rtlMult -> rateMult (in line with soapy) - adjust rtl and soapy gain routines to work with floats cmake -Drtl=ON -Dsoapy=ON -Dairspy=ON works and builds
This reverts commit 22f0c81. Does this solve an actual problem? dumpvdl2 doesn't have similar code. This runs an extraneous thread and uses nonportable pthread API. -Drtl=ON now builds on macOS
Does this solve an actual problem? Neither dumpvdl2 nor common examples have similar code. This runs an extraneous thread and uses nonportable pthread API. -Dsoapy=ON now builds on macOS Additionally: - add some error checking - retry on SOAPY_SDR_OVERFLOW (previous code wouldn't work either without it) - gracefully retry on zero-reads - don't free in buffer while read() may still be in-flight - remove unnecessary global variables PS: this needs so much more work :/
- No functional change. - Some unused vars have been removed.
- No functional change. - Some unused vars have been removed.
- Simplify the logic (use the median value of min and max frequency) - Switch all Fc/minFc/maxFc references to unsigned int This also "fixes" the following warning: warning: taking the absolute value of unsigned type 'unsigned int' has no effect [-Wabsolute-value]
First pass at cleaning the mess, using Linux kernel clang-format rules (without ColumnLimit) and a handful of manual adjustements. No code change.
don't explicitly zero R initializers (zeroing is implicit)
allows decoding an arbitrary number of channels, limited only by CPU and memory. channels.Fr is now unsigned int, adjust casts accordingly until further cleanup.
Remove malloc() casts. No code change.
find it through pkg-config since we're already using that for libacars and make it a hard dependency for now.
We no longer need the various -D<radio>=On since we can build them all simultaneously. Instead, print an informative message when the corresponding library is or isn't found.
also make MQTT support depend on it.
This commit enables the use of multiple concurrent outputs, using the following syntax: acarsdec --output format:destination:parameters [--output ...] Where "format" is one of: - "oneline" for single line text decoding - "full" for full text decoding - "monitor" for live decoding - "pp" for PlanePlotter - "native" for Acarsdec native format With CJSON support enabled: - "json" for JSON output - "routejson" for flight route output in JSON format and "destination" is one of: - "file" for file (including stdout) output - "udp" for network output over UDP With MQTT support enabled: - "mqtt" for MQTT output Not all combinations of format and destination are valid, acarsdec will complain if an invalid combination is chosen. params are described in respective source file (TODO: update usage()) Fixes: TLeconte#76
No functional change
GCC is apparently not as smart as clang to identify uninitialized uses Fixes: d6b48ae
for the if (ch->MskS & 1) block, using a ternary op enables the compiler to efficiently optimize the sign adjustment and real/imag part assignment using (on x86) a single branch instead of 5. for the if (ch->MskS & 2) block, using a ternary inside the function call reduces code size by having only one inlined copy and allows further integration with the results of the previous if block. this also makes the code slightly more compact/readable
dump all channels in a more efficient manner than doing it sample by sample
As dm_buffer from SDR input only contains positive values (cabsf()), i.e. half of the power scale (just like the matched filter), this normalization skews the actual dB scale readout and prevents reporting clipping (i.e. overloads). This change has been matched with a readout of an audio dump of a live SDR capture normalized in an audio processing software: the reported value of the normalized file is '-0.0' as expected, vs '-6.0' with the previous code. The lowest reported values for SDR input are now also more in line with the expected SNR. In order to keep soundfile input values consistent with this change, the audio samples are "normalized" to half scale. That means that audio files generated through DEBUG will appear 6dB quieter than the original signal (i.e. the readback will report the same value as was reported for SDR input *before* this patch). The rationale is to show "true" values for "standard" audio input.
This measures signal level averaged mostly over the PRESYNC key, which should typically be the "loudest" part.
|
Thank you so much for this. I maintain a docker container that originally was running TLeconte's acarsdec, and I've switched to using your version 4.0/4.1 for my users. It works wonderfully, and I appreciate your work on this. |
Thanks for the feedback, much appreciated. I have a few more improvements brewing but have left them on the backburner especially considering there was zero feedback here. This gives new motivation to complete that work 👍 |
|
In case the brave and adventurous are interested, testing of this branch is most welcome: https://github.com/f00b4r0/acarsdec/commits/demod/ There are a few changes to the decoder for which the most user-side visible impact is a change in reported signal levels (which should be closer to "reality") and the addition of a (very basic and not very accurate) noise floor estimate (shown in plain text output: NB: I'm running that code myself but don't assume any/all of these changes are release-worthy (i.e.: will be merged into master), this is work in progress left as it was when I put things on hold. FWIW I'm also successfully using a much lower multiplier ( I still want to improve the MSK routines (need to better understand the PLL lock speed and a couple tricks used there) before I move forward. The filter also needs improvement as I'm occasionally seeing messages spillover across adjacent channels (this was already the case with stock acarsdec so probably not related to my changes). HTH |
This narrows the IF filter band to improve rejection of nearby unwanted signals.
See d389b4
|
demod branch seems to work fine. master branch of your fork is in use for https://github.com/sdr-enthusiasts/docker-dumpvdl2 which in turn is used by adsb.im Might be useful to open issues on your fork just because. |
Thanks for the feedback. I'll merge into master soon I guess. I'd still want to improve the matched filter (there's too much crosstalk between adjacent channels on loud signals - unrelated to my changes) but didn't find the time to wrap my head around it yet. I need to educate myself a little more about SDR signal processing tricks.
Cool! I suppose you meant https://github.com/sdr-enthusiasts/docker-acarsdec (even though the README mentions some other fork) ;)
Have to admit being reluctant to do that as I'm not overly keen on dealing with the typical GH issue onslaught. ;P At least PRs tend to be more productive. But I guess I'll open the issues now that this is getting more audience.
Great. Thanks again for the feedback, much appreciated. |
This introduces a basic, low cpu-cost NF estimator which works by using an exponential moving average over the magnitude of the received signal updated only outside of ACARS message processing. The computational overhead of this extra estimator is virtually nil, as the signal power estimator has been reworked to use the same input as the noise floor estimator. The known limitation is that this estimate will be skewed in case of a large burst of ACARS messages as the average will no longer converge towards the noisefloor. To be improved on later. Fixes: TLeconte#84
use a ceiling macro to compute correct BITLEN regardless of INTRATE
This commit enables correct support of arbitrary INTRATE values (as long as they respect Nyquist-Shannon) This corrects d721086
Integer multiple of MSK shift freq, this should result in less bit phase drift (cf BITLEN). This also enables "native" 48kHz audio processing.
Compute the minimum usable sample rate multiplier, flooring at 80 (960kS/s) to account for the RTL sampling rate hole.
Default to 100 (1.2MS/s) which should be plenty enough for most situations while reducing the CPU load from the previous default, until proper automation is implemented (requires parsing the target device capabilities).
|
-m auto i've tested 80 and lower is probably not terribly useful as you approach the limits of the rtl-sdr anyhow. |
No problem at all, in fact: f00b4r0@e8a27d6 :-)
nod. I'm flooring at 80 for that reason. |
|
I was just gonna take a look at the code, i've been in there before to do minor CPU optimizations and stuff but you're obviously much more familiar after all the changes you've made. |
|
No worries, I'll merge and will open issues. We should take these conversations to the right repo. |
Hi,
This large PR proposes what could be acarsdec 4.x
It'll be too long to document each commit separately (that's what the commit log is for anyway, I tried to use meaningful commit messages) so I'll focus on main changes:
This PR includes #116, #107, #106 and bits of #111, in more or less reworked form. It fixes #112, #76 and presumably #100 (through inclusion of #107).
The bulk of this PR consists of:
-march=nativecompilation flag (which can be added back on cmake invocation, as detailed in README), should make acarsdec packageable by distributions if necessaryOh yes and now there's also a bit of statistics collection and reporting available via a simple StatsD-compatible interface. It's loosely based on how dumpvdl2 reports its own statistics. This helps to e.g. keep track of the performance of the SDR dongle, or to find which ACARS channels are the busiest.
The code reorganization (building all supported input backends together) and the ability to use multiple output combinations prompted me to change quite drastically how command line options are processed. This means in particular that from this point forward, acarsdec is no longer compatible with previous command line parameters (another reason for increasing the version major number, if all the above changes weren't enough already 😉):
--rtlsdrinstead of-ror--airspyinstead of-s, loosely based on dumpvdl2), yet their argument remains unchanged: migration is easy.--output FORMAT:DESTINATION:PARAMETERSallowing mix and match of output formats and destinations (e.g. it's now possible to log planeplotter format to file, or send full text via UDP). This is detailed in the README and the supported formats and destinations can be listed viaacarsdec --output help.I tried to revamp the help text (shown with
acarsdec -h) to make everything as clear as possible, as well as correctly handling the cases where not all optional libraries were available at build time.Finally the things I didn't touch (beyond code cleanup) at this stage are:
The latter two I might revisit later as I see some room for improvement (I'd like to add a noisefloor measurement among other things), but I thought that a 100-commit PR was a nice round number for a start 😅
I have tried to add some meaningful documentation where applicable, I'm by no means an SDR expert so I hope I didn't write any blunder. Should probably still be better than nothing 😊
Sucessfully built: everything except the sdrplay backend (I don't have the required dependencies)
Tested:
Needs testing:
(though I shouldn't have brought any functional changes to the latter two)
Tests performed on macOS x86_64, Linux x86_64 and armhf.
I think that sums it up. I'm happy to help maintain acarsdec in the future, if that can help 😊
I hope all of this is useful.