Crash in test_import: Assertion error about monitoring version.

[markshannon]

This

./python -m test -j1 -R 3:3 test_import -v -m test_concurrency -m test___cached___legacy_pyc -m test_package___cached___from_pyc

Crashes with Assertion code->_co_instrumentation_version == tstate->interp->monitoring_version' failed.`

Linked PRs

• GH-106012: Fix monitoring with static code objects #106017

[vstinner]

Issue first reported as: #91095 (comment)

[sunmy2019]

There are other tests facing the same errors.

./python -m test -R: test_bdb
0:00:00 load avg: 3.54 Run tests sequentially
0:00:00 load avg: 3.54 [1/1] test_bdb
beginning 9 repetitions
123456789
..python: Python/instrumentation.c:940: call_instrumentation_vector: Assertion `code->_co_instrumentation_version == tstate->interp->monitoring_version' failed.
Fatal Python error: Aborted

Current thread 0x00007f6f27a06740 (most recent call first):
  File "__init__", line ??? in __init__
  File "<frozen importlib._bootstrap_external>", line 1465 in _path_hooks
  File "<frozen importlib._bootstrap_external>", line 1489 in _path_importer_cache
  File "<frozen importlib._bootstrap_external>", line 1502 in _get_spec
  File "<frozen importlib._bootstrap_external>", line 1530 in find_spec
  File "<frozen importlib._bootstrap>", line 1168 in _find_spec
  File "<frozen importlib._bootstrap>", line 1228 in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 1266 in _find_and_load
  File "/home/xxx/cpython/Lib/test/test_bdb.py", line 557 in tfunc_import
  File "/home/xxx/cpython/Lib/bdb.py", line 643 in runcall
  File "/home/xxx/cpython/Lib/test/test_bdb.py", line 953 in test_clear_two_bp_on_same_line
  File "/home/xxx/cpython/Lib/unittest/case.py", line 589 in _callTestMethod
  File "/home/xxx/cpython/Lib/unittest/case.py", line 634 in run
  File "/home/xxx/cpython/Lib/unittest/case.py", line 690 in __call__
  File "/home/xxx/cpython/Lib/unittest/suite.py", line 122 in run
  File "/home/xxx/cpython/Lib/unittest/suite.py", line 84 in __call__
  File "/home/xxx/cpython/Lib/unittest/suite.py", line 122 in run
  File "/home/xxx/cpython/Lib/unittest/suite.py", line 84 in __call__
  File "/home/xxx/cpython/Lib/unittest/suite.py", line 122 in run
  File "/home/xxx/cpython/Lib/unittest/suite.py", line 84 in __call__
  File "/home/xxx/cpython/Lib/test/support/testresult.py", line 143 in run
  File "/home/xxx/cpython/Lib/test/support/__init__.py", line 1115 in _run_suite
  File "/home/xxx/cpython/Lib/test/support/__init__.py", line 1241 in run_unittest
  File "/home/xxx/cpython/Lib/test/libregrtest/runtest.py", line 294 in _test_module
  File "/home/xxx/cpython/Lib/test/libregrtest/refleak.py", line 90 in dash_R
  File "/home/xxx/cpython/Lib/test/libregrtest/runtest.py", line 328 in _runtest_inner2
  File "/home/xxx/cpython/Lib/test/libregrtest/runtest.py", line 373 in _runtest_inner
  File "/home/xxx/cpython/Lib/test/libregrtest/runtest.py", line 248 in _runtest
  File "/home/xxx/cpython/Lib/test/libregrtest/runtest.py", line 278 in runtest
  File "/home/xxx/cpython/Lib/test/libregrtest/main.py", line 483 in run_tests_sequential
  File "/home/xxx/cpython/Lib/test/libregrtest/main.py", line 621 in run_tests
  File "/home/xxx/cpython/Lib/test/libregrtest/main.py", line 799 in _main
  File "/home/xxx/cpython/Lib/test/libregrtest/main.py", line 758 in main
  File "/home/xxx/cpython/Lib/test/libregrtest/main.py", line 822 in main
  File "/home/xxx/cpython/Lib/test/__main__.py", line 2 in <module>
  File "/home/xxx/cpython/Lib/runpy.py", line 88 in _run_code
  File "/home/xxx/cpython/Lib/runpy.py", line 198 in _run_module_as_main

Extension modules: _testcapi (total: 1)
Aborted

[sunmy2019]

Also bisected to 04492cb

[alex]

regexp fuzzer is crashing with the following stack, potentially related?

fuzz_sre_compile: Python/frame.c:138: void _PyFrame_ClearExceptCode(_PyInterpreterFrame *): Assertion `frame->stacktop >= 0' failed.
--
  | ==139203== ERROR: libFuzzer: deadly signal
  | #0 0x4dee74 in __sanitizer_print_stack_trace /src/llvm-project/compiler-rt/lib/ubsan/ubsan_diag_standalone.cpp:31:3
  | #1 0x460018 in fuzzer::PrintStackTrace() /src/llvm-project/compiler-rt/lib/fuzzer/FuzzerUtil.cpp:210:5
  | #2 0x444cf3 in fuzzer::Fuzzer::CrashCallback() /src/llvm-project/compiler-rt/lib/fuzzer/FuzzerLoop.cpp:233:3
  | #3 0x7c3535d4841f in libpthread.so.0
  | #4 0x7c3535b5e00a in __libc_signal_restore_set /build/glibc-SzIz7B/glibc-2.31/sysdeps/unix/sysv/linux/internal-signals.h:86:3
  | #5 0x7c3535b5e00a in raise /build/glibc-SzIz7B/glibc-2.31/sysdeps/unix/sysv/linux/raise.c:48:3
  | #6 0x7c3535b3d858 in abort /build/glibc-SzIz7B/glibc-2.31/stdlib/abort.c:79:7
  | #7 0x7c3535b3d728 in __assert_fail_base /build/glibc-SzIz7B/glibc-2.31/assert/assert.c:92:3
  | #8 0x7c3535b4efd5 in __assert_fail /build/glibc-SzIz7B/glibc-2.31/assert/assert.c:101:3
  | #9 0xa45132 in _PyFrame_ClearExceptCode cpython3/Python/frame.c:138:5
  | #10 0x92b0c0 in clear_thread_frame cpython3/Python/ceval.c:1548:5
  | #11 0x91f110 in _PyEvalFrameClearAndPop cpython3/Python/ceval.c:1574:9
  | #12 0x897259 in _PyEval_EvalFrameDefault cpython3/Python/ceval.c:1001:5
  | #13 0x892f8a in _PyEval_EvalFrame cpython3/Include/internal/pycore_ceval.h:88:16
  | #14 0x892a29 in _PyEval_Vector cpython3/Python/ceval.c:1683:12
  | #15 0x51391b in _PyFunction_Vectorcall cpython3/Objects/call.c:0
  | #16 0x10758b5 in _PyObject_VectorcallTstate cpython3/Include/internal/pycore_call.h:92:11
  | #17 0x10717e6 in method_vectorcall cpython3/Objects/classobject.c:67:20
  | #18 0x9238c3 in _PyObject_VectorcallTstate cpython3/Include/internal/pycore_call.h:92:11
  | #19 0x9238c3 in _PyObject_CallNoArgs cpython3/Include/internal/pycore_call.h:108:12
  | #20 0x8f8f97 in _PyEval_EvalFrameDefault cpython3/Python/bytecodes.c:2540:19
  | #21 0x892f8a in _PyEval_EvalFrame cpython3/Include/internal/pycore_ceval.h:88:16
  | #22 0x892a29 in _PyEval_Vector cpython3/Python/ceval.c:1683:12
  | #23 0x51391b in _PyFunction_Vectorcall cpython3/Objects/call.c:0
  | #24 0x516994 in _PyObject_VectorcallTstate cpython3/Include/internal/pycore_call.h:92:11
  | #25 0x516994 in object_vacall cpython3/Objects/call.c:824:14
  | #26 0x51649a in PyObject_CallMethodObjArgs cpython3/Objects/call.c:885:24
  | #27 0xa8a733 in import_find_and_load cpython3/Python/import.c:2767:11
  | #28 0xa8a733 in PyImport_ImportModuleLevelObject cpython3/Python/import.c:2847:15
  | #29 0x8e76f5 in import_name cpython3/Python/ceval.c:2458:15
  | #30 0x8e76f5 in _PyEval_EvalFrameDefault cpython3/Python/bytecodes.c:2132:19
  | #31 0x892f8a in _PyEval_EvalFrame cpython3/Include/internal/pycore_ceval.h:88:16
  | #32 0x892a29 in _PyEval_Vector cpython3/Python/ceval.c:1683:12
  | #33 0x51391b in _PyFunction_Vectorcall cpython3/Objects/call.c:0
  | #34 0x5136c6 in _PyObject_VectorcallTstate cpython3/Include/internal/pycore_call.h:92:11
  | #35 0x5136c6 in PyObject_CallOneArg cpython3/Objects/call.c:401:12
  | #36 0x86f57e in call_show_warning cpython3/Python/_warnings.c:604:11
  | #37 0x8654fb in warn_explicit cpython3/Python/_warnings.c:741:13
  | #38 0x868f6c in do_warn cpython3/Python/_warnings.c:997:11
  | #39 0x8719b9 in warnings_warn_impl cpython3/Python/_warnings.c:1044:12
  | #40 0x8719b9 in warnings_warn cpython3/Python/clinic/_warnings.c.h:123:20
  | #41 0x908834 in _PyEval_EvalFrameDefault cpython3/Python/bytecodes.c:3074:19
  | #42 0x892f8a in _PyEval_EvalFrame cpython3/Include/internal/pycore_ceval.h:88:16
  | #43 0x892a29 in _PyEval_Vector cpython3/Python/ceval.c:1683:12
  | #44 0x51391b in _PyFunction_Vectorcall cpython3/Objects/call.c:0
  | #45 0x516994 in _PyObject_VectorcallTstate cpython3/Include/internal/pycore_call.h:92:11
  | #46 0x516994 in object_vacall cpython3/Objects/call.c:824:14
  | #47 0x516f08 in PyObject_CallFunctionObjArgs cpython3/Objects/call.c:931:14
  | #48 0x4e0730 in fuzz_sre_compile cpython3/Modules/_xxtestfuzz/fuzzer.c:248:26
  | #49 0x4e0730 in _run_fuzz cpython3/Modules/_xxtestfuzz/fuzzer.c:443:14
  | #50 0x4e0730 in LLVMFuzzerTestOneInput cpython3/Modules/_xxtestfuzz/fuzzer.c:541:15
  | #51 0x446293 in fuzzer::Fuzzer::ExecuteCallback(unsigned char const*, unsigned long) /src/llvm-project/compiler-rt/lib/fuzzer/FuzzerLoop.cpp:611:15
  | #52 0x4319f2 in fuzzer::RunOneTest(fuzzer::Fuzzer*, char const*, unsigned long) /src/llvm-project/compiler-rt/lib/fuzzer/FuzzerDriver.cpp:324:6
  | #53 0x43729c in fuzzer::FuzzerDriver(int*, char***, int (*)(unsigned char const*, unsigned long)) /src/llvm-project/compiler-rt/lib/fuzzer/FuzzerDriver.cpp:860:9
  | #54 0x4607d2 in main /src/llvm-project/compiler-rt/lib/fuzzer/FuzzerMain.cpp:20:10
  | #55 0x7c3535b3f082 in __libc_start_main /build/glibc-SzIz7B/glibc-2.31/csu/libc-start.c:308:16
  | #56 0x427bbd in _start

[markshannon]

That doesn't look related

[markshannon]

Seems to be fixed. Both ./python -m test -j1 -R 3:3 test_import -v -m test_concurrency -m test___cached___legacy_pyc -m test_package___cached___from_pyc and ./python -m test -R: test_bdb pass.