Tool verible-verilog-lint: fatal flex scanner internal error #2438
Description
Describe the bug
I have found suspicious behavior of verible-verilog-lint involving LexerError and ASAN heap-overflow.
Without ASAN:
Fatal LexerError: fatal flex scanner internal error--end of buffer missed*** SIGABRT received at time=1752593638 on cpu 8 ***
PC: @ 0x60ad2c (unknown) pthread_kill
@ 0x5ebb40 64906160 (unknown)
@ 0x5ebabe 32 gsignal
@ 0x408777 192 abort
@ 0x424894 32 verible::FlexLexerAdapter<>::LexerError()
@ 0x4c6d4e 64 verilogFlexLexer::yy_get_next_buffer()
@ 0x4cb82c 96 verilog::VerilogLexer::yylex()
@ 0x4d1d28 1280 verilog::RecursiveLexText()
@ 0x4b8450 304 verilog::analysis::MacroNameStyleRule::HandleToken()
@ 0x46dadd 96 verible::TokenStreamLinter::Lint()
@ 0x41f4c1 48 verilog::VerilogLinter::Lint()
@ 0x422628 576 verilog::VerilogLintTextStructure()
@ 0x42290a 272 verilog::LintOneFile()
@ 0x40958a 480 main
@ 0x5d2d58 160 __libc_start_call_main
@ 0x5d4f20 80 __libc_start_main_impl
@ 0x40d695 (unknown) _start
With ASAN:
=379==ERROR: AddressSanitizer: heap-buffer-overflow on address 0x7c26447e0e62 at pc 0x5602e0f22ab4 bp 0x7ffcbc230d80 sp 0x7ffcbc230538
READ of size 20 at 0x7c26447e0e62 thread T0
#0 0x5602e0f22ab3 in memcpy (/root/.cache/bazel/_bazel_root/9b7f7d7d4c72fd368a9ff96e17d6e76d/execroot/_main/bazel-out/k8-opt/bin/verible/verilog/tools/lint/verible-verilog-lint+0x182ab3) (BuildId: 6b1f2d7da8c416b8742c5689c07dc0ec6708af83)
#1 0x7f76457b3729 in std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char>>::basic_string(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char>>::__sv_wrapper, std::allocator<char> const&) (/lib/x86_64-linux-gnu/libstdc++.so.6+0x193729) (BuildId: 289ee39f8c07bd4fa48102dfeeb7e6f9c76158b4)
#2 0x5602e0f90342 in std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char>>::basic_string<std::basic_string_view<char, std::char_traits<char>>, void>(std::basic_string_view<char, std::char_traits<char>> const&, std::allocator<char> const&) /usr/lib/gcc/x86_64-linux-gnu/12/../../../../include/c++/12/bits/basic_string.h:787:4
#3 0x5602e1215497 in verible::FlexLexerAdapter<verilogFlexLexer>::FlexLexerAdapter(std::basic_string_view<char, std::char_traits<char>>) /proc/self/cwd/./verible/common/lexer/flex-lexer-adapter.h:72:22
#4 0x5602e121502e in verilog::VerilogLexer::VerilogLexer(std::basic_string_view<char, std::char_traits<char>>) /proc/self/cwd/verible/verilog/parser/verilog-lexer.cc:27:53
#5 0x5602e1215eae in verilog::RecursiveLexText(std::basic_string_view<char, std::char_traits<char>>, std::function<void (verible::TokenInfo const&)> const&) /proc/self/cwd/verible/verilog/parser/verilog-lexer.cc:59:16
#6 0x5602e11dfb4b in verilog::analysis::MacroNameStyleRule::HandleToken(verible::TokenInfo const&) /proc/self/cwd/verible/verilog/analysis/checkers/macro-name-style-rule.cc:89:5
#7 0x5602e11e13ad in verilog::analysis::MacroNameStyleRule::HandleToken(verible::TokenInfo const&)::$_0::operator()(verible::TokenInfo const&) const /proc/self/cwd/verible/verilog/analysis/checkers/macro-name-style-rule.cc:90:51
#8 0x5602e11e135c in void std::__invoke_impl<void, verilog::analysis::MacroNameStyleRule::HandleToken(verible::TokenInfo const&)::$_0&, verible::TokenInfo const&>(std::__invoke_other, verilog::analysis::MacroNameStyleRule::HandleToken(verible::TokenInfo const&)::$_0&, verible::TokenInfo const&) /usr/lib/gcc/x86_64-linux-gnu/12/../../../../include/c++/12/bits
/invoke.h:61:14
#9 0x5602e11e12fc in std::enable_if<is_invocable_r_v<void, verilog::analysis::MacroNameStyleRule::HandleToken(verible::TokenInfo const&)::$_0&, verible::TokenInfo const&>, void>::type std::__invoke_r<void, verilog::analysis::MacroNameStyleRule::HandleToken(verible::TokenInfo const&)::$_0&, verible::TokenInfo const&>(verilog::analysis::MacroNameStyleRule::Han
dleToken(verible::TokenInfo const&)::$_0&, verible::TokenInfo const&) /usr/lib/gcc/x86_64-linux-gnu/12/../../../../include/c++/12/bits/invoke.h:111:2
#10 0x5602e11e11e4 in std::_Function_handler<void (verible::TokenInfo const&), verilog::analysis::MacroNameStyleRule::HandleToken(verible::TokenInfo const&)::$_0>::_M_invoke(std::_Any_data const&, verible::TokenInfo const&) /usr/lib/gcc/x86_64-linux-gnu/12/../../../../include/c++/12/bits/std_function.h:290:9
#11 0x5602e111f93f in std::function<void (verible::TokenInfo const&)>::operator()(verible::TokenInfo const&) const /usr/lib/gcc/x86_64-linux-gnu/12/../../../../include/c++/12/bits/std_function.h:591:9
#12 0x5602e1215f33 in verilog::RecursiveLexText(std::basic_string_view<char, std::char_traits<char>>, std::function<void (verible::TokenInfo const&)> const&) /proc/self/cwd/verible/verilog/parser/verilog-lexer.cc:63:5
#13 0x5602e11dfb4b in verilog::analysis::MacroNameStyleRule::HandleToken(verible::TokenInfo const&) /proc/self/cwd/verible/verilog/analysis/checkers/macro-name-style-rule.cc:89:5
#14 0x5602e1122e41 in verible::TokenStreamLinter::Lint(std::vector<verible::TokenInfo, std::allocator<verible::TokenInfo>> const&) /proc/self/cwd/verible/common/analysis/token-stream-linter.cc:31:31
#15 0x5602e0fc5ff0 in verilog::VerilogLinter::Lint(verible::TextStructureView const&, std::basic_string_view<char, std::char_traits<char>>) /proc/self/cwd/verible/verilog/analysis/verilog-linter.cc:248:24
#16 0x5602e0fc10a1 in verilog::VerilogLintTextStructure(std::basic_string_view<char, std::char_traits<char>>, verilog::LinterConfiguration const&, verible::TextStructureView const&) /proc/self/cwd/verible/verilog/analysis/verilog-linter.cc:326:10
#17 0x5602e0fbfad4 in verilog::LintOneFile(std::ostream*, std::basic_string_view<char, std::char_traits<char>>, verilog::LinterConfiguration const&, verible::ViolationHandler*, bool, bool, bool, bool) /proc/self/cwd/verible/verilog/analysis/verilog-linter.cc:149:7
#18 0x5602e0f606f6 in main /proc/self/cwd/verible/verilog/tools/lint/verilog-lint.cc:231:29
#19 0x7f7645355249 (/lib/x86_64-linux-gnu/libc.so.6+0x27249) (BuildId: 79005c16293efa45b441fed45f4f29b138557e9e)
#20 0x7f7645355304 in __libc_start_main (/lib/x86_64-linux-gnu/libc.so.6+0x27304) (BuildId: 79005c16293efa45b441fed45f4f29b138557e9e)
#21 0x5602e0e82a90 in _start (/root/.cache/bazel/_bazel_root/9b7f7d7d4c72fd368a9ff96e17d6e76d/execroot/_main/bazel-out/k8-opt/bin/verible/verilog/tools/lint/verible-verilog-lint+0xe2a90) (BuildId: 6b1f2d7da8c416b8742c5689c07dc0ec6708af83)
Version: v0.0-4007-g98bdb38a
Platform: Linux x86_64
To Reproduce
- Compile the linter with ASAN enabled. I used the command:
CC=clang-20 \
CXX=clang++-20 \
exec bazel build -c opt \
--copt="-g" \
--copt="-O0" \
--copt="-fsanitize=address" \
--cxxopt="-g" \
--cxxopt="-O0" \
--cxxopt="-fsanitize=address" \
--linkopt="-L/usr/lib/llvm-20/lib/" \
--linkopt="-fsanitize=address" \
--jobs=$(nproc) \
--//bazel:use_local_flex_bison \
--sandbox_debug \
--verbose_failures \
//verible/verilog/tools/lint:verible-verilog-lint
Save the resulting binary, for example, as /tmp/verible-verilog-lint-asan
-
Download the
v0.0-4007-g98bdb38abinary for x86_46 Linux. Save it, for example, as/tmp/verible-verilog-lint -
Prepare the input file:
00000000: 6d6f 6475 6c65 206c 6564 5f73 7728 7377 module led_sw(sw
00000010: 293b 0a46 6046 2846 6046 2846 0a29 3b0a );.F`F(F`F(F.);.
00000020: 4660 c346 2846 4660 4646 5c46 4646 2246 F`.F(FF`FF\FFF"F
00000030: 4669 2046 4660 4646 4646 2846 6d6f 7374 Fi FF`FFFF(Fmost
00000040: 6922 c665 7520 2020 202f 2020 2020 200a i".eu / .
00000050: 293b 0a29 3b0a 656e 646d 6f64 756c 650a );.);.endmodule.
00000060: 0a .
-
Save it as
/tmp/data.xdd, then run:xdd -r /tmp/data.xdd > /tmp/data-xdd.v -
Run the binaries prepared in steps 1 and 2:
/tmp/verible-verilog-lint-asan /tmp/data-xdd.v
/tmp/verible-verilog-lint /tmp/data-xdd.v
Actual behavior:
The verible-verilog-lint tool crashes when given the provided input. A Heap-overflow occurs.
Expected behavior
An error in Verilog should be reported without causing a crash or heap overflow.