fuzz coverage fails with clang?

[maflcko]

I get:

/filter-lcov.py", line 19, in <module>
    with open(tracefile, 'r', encoding="utf8") as f:
         ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
FileNotFoundError: [Errno 2] No such file or directory: 'fuzz.info'
Combining tracefiles.
lcov: ERROR: (missing) 'fuzz_filtered.info' found from pattern 'fuzz_filtered.info' is not a readable file
	(use "lcov --ignore-errors missing ..." to bypass this error)
lcov: ERROR: (missing) 'baseline_filtered.info' found from pattern 'baseline_filtered.info' is not a readable file
	(use "lcov --ignore-errors missing ..." to bypass this error)
genhtml: ERROR: (missing) 'fuzz_coverage.info' found from pattern 'fuzz_coverage.info' is not a readable file
	(use "genhtml --ignore-errors missing ..." to bypass this error)

I used maflcko/DrahtBot@1b699e8

I can try to add easier steps to reproduce tomorrow.

[hebasto]

What are the OS and the lcov version?

[maflcko]

Either Ubuntu 24.04 or 24.10 should reproduce:

git checkout fa680ac6c016533ac512857647555c688c071b63 && mkdir -p empty_temp && cmake -B bld -DBUILD_FOR_FUZZING=ON -DSANITIZERS=fuzzer -DCMAKE_C_COMPILER=clang -DCMAKE_CXX_COMPILER=clang++ -DCMAKE_BUILD_TYPE=Coverage && cmake --build ./bld -j$( nproc ) && cmake -DJOBS=$( nproc ) -DFUZZ_SEED_CORPUS_DIR=$PWD/empty_temp -DLCOV_OPTS='--rc branch_coverage=1 --ignore-errors mismatch,inconsistent' -P bld/CoverageFuzz.cmake

[maflcko]

The error for 24.04 is slightly different:

Processing src/util/CMakeFiles/bitcoin_util.dir/fs_helpers.cpp.gcda
Processing src/util/CMakeFiles/bitcoin_util.dir/strencodings.cpp.gcda
Processing src/util/CMakeFiles/bitcoin_util.dir/signalinterrupt.cpp.gcda
Processing src/util/CMakeFiles/bitcoin_util.dir/fs.cpp.gcda
Processing src/util/CMakeFiles/bitcoin_util.dir/exception.cpp.gcda
Processing src/util/CMakeFiles/bitcoin_util.dir/bip32.cpp.gcda
Processing src/util/CMakeFiles/bitcoin_util.dir/serfloat.cpp.gcda
Processing src/util/CMakeFiles/bitcoin_util.dir/time.cpp.gcda
Processing src/util/CMakeFiles/bitcoin_util.dir/feefrac.cpp.gcda
Processing src/util/CMakeFiles/bitcoin_util.dir/check.cpp.gcda
geninfo: ERROR: unable to open /b-c/bld/src/src/httpserver.cpp: No such file or directory
	(use "geninfo --ignore-errors source ..." to bypass this error)
Deleting all .da files in src and subdirectories
Done.
lcov: ERROR: trace file 'fuzz_filtered.info' is empty
	(use "lcov --ignore-errors empty ..." to bypass this error)
Combining tracefiles.
.. found 1 files to aggregate.
Merging fuzz_filtered.info..0 remaining
lcov: ERROR: trace file 'baseline_filtered.info' is empty
	(use "lcov --ignore-errors empty ..." to bypass this error)
genhtml: ERROR: cannot read file fuzz_coverage.info!

[maflcko]

Also, the exit code may not be passed up, which could be improved as well?

# echo $?
0

[maflcko]

Normal coverage also fails.

What is the commit and the exact commands where it last worked?

[hebasto]

Normal coverage also fails.

What is the commit and the exact commands where it last worked?

Working on it.

[hebasto]

Normal coverage also fails.

What is the commit and the exact commands where it last worked?

Tested bitcoin#30454 @ 4105129 on Ubuntu 24.04 with the default gcc and lcov packages:

$ cmake -B build -DCMAKE_BUILD_TYPE=Coverage -DCMAKE_C_FLAGS="-fprofile-update=atomic" -DCMAKE_CXX_FLAGS="-fprofile-update=atomic"
$ cmake --build build
$ cmake -DLCOV_OPTS="--ignore-errors mismatch" -P build/Coverage.cmake
$ firefox ./build/test_bitcoin.coverage/index.html
$ firefox ./build/total.coverage/index.html

However, on my machine, I had to add --timeout-factor=4.

Additional -DCMAKE_C_FLAGS="-fprofile-update=atomic" became required after #192.

[maflcko]

Ah, so normal coverage may work with gcc. However, it would be nice if clang was supported as well (like it does with autotools).

[hebasto]

cc @vasild

[hebasto]

Ah, so normal coverage may work with gcc. However, it would be nice if clang was supported as well (like it does with autotools).

Tested bitcoin#30454 @ 4105129 on Ubuntu 24.10 with the default clang and lcov packages:

$ cmake -B build -DCMAKE_BUILD_TYPE=Coverage -DCMAKE_C_COMPILER=clang -DCMAKE_CXX_COMPILER=clang++
$ cmake --build build
$ cmake -DLCOV_OPTS="--rc branch_coverage=1 --ignore-errors mismatch,inconsistent" -P build/Coverage.cmake

On Ubuntu 24.04 lcov still randomly fails.

[maflcko]

Did you pass -DJOBS=$( nproc )?

[hebasto]

Did you pass -DJOBS=$( nproc )?

I did. Why?

[maflcko]

Because I couldn't find it in your commands to reproduce the failure, and it seems to be required locally to reproduce it.

I'll try to create a full testing matrix to see what works and what needs fixing.

[vasild]

Clang has its own coverage workflow, which produces superior results. It also supports gcc/gcov/lcov like workflow but I think it is a waste of time^* to try to fix that (if it is broken) to force Clang through the gcc way, provided it has a better native way.

#233 implements the Clang native workflow.

^* my time, of course; ymmv