[BUG]: Many pybind11 tests fail under ASan/UBSan

[NAThompson]

Required prerequisites

• Make sure you've read the documentation. Your issue may be addressed there.
• Search the issue tracker and Discussions to verify that this hasn't already been reported. +1 or comment there if it has.
• Consider asking first in the Gitter chat room or in a Discussion.

Problem description

In user code I have been observing lots of segfaults and UB; naturally I tried to discover its cause via ASan and UBSan. After discovering issue #3514 it was unclear to me where the fixes should go: In user code or in pybind11. Hence I ran the pybind11 unit tests after a compilation with:

cmake -DDOWNLOAD_CATCH=ON  -DCMAKE_CXX_FLAGS="-fsanitize=address -fsanitize=undefined -g" ../ -G Ninja

The following tests failed:

~/pybind11/build/tests ❯❯❯ python3 -m pytest ../../tests/ -v
../../tests/test_builtin_casters.py::test_single_char_arguments 
~/pybind11/include/pybind11/detail/internals.h:391:46: runtime error: member call on address 0x000130eb9540 which does not point to an object of type 'pybind11::builtin_exception'
0x000130eb9540: note: object is of type 'pybind11::value_error'
 01 00 00 00  88 75 9e 17 01 00 00 00  28 78 16 3a 01 00 00 00  dd dd dd dd dd dd dd dd  00 00 00 00
              ^~~~~~~~~~~~~~~~~~~~~~~
              vptr for 'pybind11::value_error'
SUMMARY: UndefinedBehaviorSanitizer: undefined-behavior 
~/pybind11/include/pybind11/detail/internals.h:391:46 in
PASSED                                                                                 [  2%]
../../tests/test_local_bindings.py::test_stl_bind_local 
~/pybind11/include/pybind11/detail/internals.h:391:46: runtime error: member call on address 0x0001134b83c0 which does not point to an object of type 'pybind11::builtin_exception'
0x0001134b83c0: note: object is of type 'pybind11::stop_iteration'
 01 00 00 00  c8 ca 9f 17 01 00 00 00  18 e4 37 0a 01 00 00 00  dd dd dd dd dd dd dd dd  00 00 00 00
              ^~~~~~~~~~~~~~~~~~~~~~~
              vptr for 'pybind11::stop_iteration'
SUMMARY: UndefinedBehaviorSanitizer: undefined-behavior 
~/pybind11/include/pybind11/detail/internals.h:391:46 in
PASSED                                                                                         [ 51%]
../../tests/test_numpy_dtypes.py::test_recarray ~/pybind11/tests/test_numpy_dtypes.cpp:56:30: runtime error: reference binding to misaligned address 0x000147c4fc29 for type 'const SimpleStruct', which requires 8 byte alignment
0x000147c4fc29: note: pointer points here
 00 00 04  c0 01 be be be 01 00 00  00 00 00 c0 3f be be be  be 00 00 00 00 00 00 04  c0 00 02 00 00
              ^
SUMMARY: UndefinedBehaviorSanitizer: undefined-behavior 
~/pybind11/tests/test_numpy_dtypes.cpp:56:30 in
FAILED                                                                                                 [ 71%]

Note that the python interpreter must be source compiled with ./configure --with-address-sanitizer --with-pydebug.

Reproducible example code

In the repo.

[Skylion007]

@rwgk You have PyBind11 setup with ASAN right?

[rwgk]

@rwgk You have PyBind11 setup with ASAN right?

Yes, also UBSAN, MSAN, TSAN, but only Google-internally. Our toolchain makes the sanitizers accessible via simple flags amounting to "use asan" or "use ubsan", what it does underneath could well be (very) different from what @NAThompson is trying.

@NAThompson without having looked at details, the errors you're getting probably fall mostly into two classes:

• False positives.
• Specific to your environment.

It would be great to have someone working on a CI job that runs ASAN (to start with), with related cleanup. My gut feeling is that there will not be a lot of cleanup, because we already have valgrind GHA and the Google-internal sanitizers, but most of the work will be in setting up the CI job to have all the right prerequisites, flags, and suppressions.