statefulness detector

[phi-go]

Add a statefulness detection step as part of coverage measurement to detect if the fuzz target is stateful. Compare the coverage measurement output by running the corpus in two different orders, if the target is not stateful there should be no difference.

Comparisons can be done by comparing the coverage summary data which is less computationally expensive and requires less memory, however it is also much less accurate. Another option is implemented comparing the number of executions for each region. Though anything provided by the coverage measurement result should be straightforward to implement.

To make use of this information in later stages, the log file and json data file with the differences are written into a subdirectory of the $LOGS_DIR.

Here is a sample output:

/src/sudo/plugins/sudoers/toke.c: 6347:5   - 657005 != 657004
/src/sudo/plugins/sudoers/toke.c: 6347:9   - 657005 != 657004
/src/sudo/plugins/sudoers/toke.c: 6347:25  - 657005 != 657004
/src/sudo/plugins/sudoers/toke.c: 6347:26  -  8132 != 8131
/src/sudo/plugins/sudoers/toke.c: 6348:2   -  8132 != 8131
/src/sudo/plugins/sudoers/toke.c: 6348:18  - 657005 != 657004
/src/sudo/plugins/sudoers/toke.c: 6351:1   - 657005 != 657004
/src/sudo/plugins/sudoers/toke.c: 6352:5   - 657005 != 657004
/src/sudo/plugins/sudoers/toke.c: 6352:21  - 657005 != 657004
Target IS stateful.

[maflcko]

Just for reference, any project that has afl enabled should have a metric "stabilty" on https://oss-fuzz.com/fuzzer-stats. The statistic will tell how good or bad the stability of a given fuzz target is.

[phi-go]

Yes, fair point. However, to my understanding oss-fuzz mostly uses libFuzzer where this is not available. Nor is it easy to extract where exactly this instability happens for the afl variant, which seems quite useful if one wants to investigate the cause.