gh-127085: Fix `memoryview->exports` race condition. #127412

LindaSummer · 2024-11-29T13:40:29Z

Issue

Close #127085 .

Proposed Changes

Make memoryview->exports protected by critical section.
Add unit test from the issue.

Cause of the problem

With TSAN, I find that the root cause should be racing between memory_getbuf and memory_releasebuf.

After adding the critical section, this racing has been fixed.

Here are the TSAN outputs.

Python 3.14.0a2+ experimental free-threading build (heads/main:db5c5763f3, Nov 28 2024, 21:41:41) [GCC 11.4.1 20231218 (Red Hat 11.4.1-3)] on linux
Type "help", "copyright", "credits" or "license" for more information.
warning: can't use pyrepl: No module named 'msvcrt'
>>> import multiprocessing.shared_memory
from threading import Thread

obj = multiprocessing.shared_memory.ShareableList("Uq..SeDAmB+EBrkLl.SG.Z+Z.ZdsV..wT+zLxKwdN\b")

for x in range(10):
    Thread(target=obj.count, args=(1,)).start()

del obj

>>> >>> >>> >>> >>> ... ... ==================
WARNING: ThreadSanitizer: data race (pid=3465000)
  Read of size 8 at 0x7f056b7e1010 by thread T2:
    #0 memory_getbuf Objects/memoryobject.c:1592 (python+0x5c0b9c)
    #1 PyObject_GetBuffer Objects/abstract.c:442 (python+0x4d8844)
    #2 unpack_from Modules/clinic/_struct.c.h:370 (_struct.cpython-314td-x86_64-linux-gnu.so+0xc66a)
    #3 cfunction_vectorcall_FASTCALL_KEYWORDS Objects/methodobject.c:452 (python+0x5c9ec7)
    #4 _PyObject_VectorcallTstate Include/internal/pycore_call.h:167 (python+0x510279)
    #5 PyObject_Vectorcall Objects/call.c:327 (python+0x5103b4)
    #6 _PyEval_EvalFrameDefault Python/generated_cases.c.h:953 (python+0x77d0fa)
    #7 _PyEval_EvalFrame Include/internal/pycore_ceval.h:116 (python+0x7bb5f9)
    #8 _PyEval_Vector Python/ceval.c:1871 (python+0x7bb5f9)
    #9 _PyFunction_Vectorcall Objects/call.c:413 (python+0x50fa9b)
    #10 _PyObject_VectorcallTstate Include/internal/pycore_call.h:167 (python+0x649082)
    #11 vectorcall_unbound Objects/typeobject.c:2776 (python+0x649082)
    #12 vectorcall_method Objects/typeobject.c:2807 (python+0x649082)
    #13 slot_sq_item Objects/typeobject.c:9617 (python+0x649446)
    #14 PySequence_GetItem Objects/abstract.c:1847 (python+0x4dd14f)
    #15 iter_iternext Objects/iterobject.c:66 (python+0x572a7d)
    #16 _PyEval_EvalFrameDefault Python/generated_cases.c.h:3927 (python+0x794437)
    #17 _PyEval_EvalFrame Include/internal/pycore_ceval.h:116 (python+0x550b43)
    #18 gen_send_ex2 Objects/genobject.c:245 (python+0x550b43)
    #19 gen_iternext Objects/genobject.c:618 (python+0x5514e3)
    #20 iternext Objects/abstract.c:2874 (python+0x4d60e7)
    #21 PyIter_Next Objects/abstract.c:2924 (python+0x4dffb4)
    #22 builtin_sum_impl Python/bltinmodule.c:2716 (python+0x764943)
    #23 builtin_sum Python/clinic/bltinmodule.c.h:1167 (python+0x766254)
    #24 cfunction_vectorcall_FASTCALL_KEYWORDS Objects/methodobject.c:452 (python+0x5c9ec7)
    #25 _PyObject_VectorcallTstate Include/internal/pycore_call.h:167 (python+0x510279)
    #26 PyObject_Vectorcall Objects/call.c:327 (python+0x5103b4)
    #27 _PyEval_EvalFrameDefault Python/generated_cases.c.h:953 (python+0x77d0fa)
    #28 _PyEval_EvalFrame Include/internal/pycore_ceval.h:116 (python+0x7bb5f9)
    #29 _PyEval_Vector Python/ceval.c:1871 (python+0x7bb5f9)
    #30 _PyFunction_Vectorcall Objects/call.c:413 (python+0x50fa9b)
    #31 _PyObject_VectorcallTstate Include/internal/pycore_call.h:167 (python+0x515230)
    #32 method_vectorcall Objects/classobject.c:93 (python+0x51603f)
    #33 _PyVectorcall_Call Objects/call.c:273 (python+0x51303f)
    #34 _PyObject_Call Objects/call.c:348 (python+0x5134f9)
    #35 PyObject_Call Objects/call.c:373 (python+0x513578)
    #36 _PyEval_EvalFrameDefault Python/generated_cases.c.h:1745 (python+0x782ad6)
    #37 _PyEval_EvalFrame Include/internal/pycore_ceval.h:116 (python+0x7bb5f9)
    #38 _PyEval_Vector Python/ceval.c:1871 (python+0x7bb5f9)
    #39 _PyFunction_Vectorcall Objects/call.c:413 (python+0x50fa9b)
    #40 _PyObject_VectorcallTstate Include/internal/pycore_call.h:167 (python+0x51623d)
    #41 method_vectorcall Objects/classobject.c:71 (python+0x51623d)
    #42 _PyVectorcall_Call Objects/call.c:273 (python+0x51303f)
    #43 _PyObject_Call Objects/call.c:348 (python+0x5134f9)
    #44 PyObject_Call Objects/call.c:373 (python+0x513578)
    #45 thread_run Modules/_threadmodule.c:337 (python+0x985d20)
    #46 pythread_wrapper Python/thread_pthread.h:242 (python+0x8aaff3)

  Previous write of size 8 at 0x7f056b7e1010 by thread T1:
    #0 memory_getbuf Objects/memoryobject.c:1592 (python+0x5c0ba8)
    #1 PyObject_GetBuffer Objects/abstract.c:442 (python+0x4d8844)
    #2 unpack_from Modules/clinic/_struct.c.h:370 (_struct.cpython-314td-x86_64-linux-gnu.so+0xc66a)
    #3 cfunction_vectorcall_FASTCALL_KEYWORDS Objects/methodobject.c:452 (python+0x5c9ec7)
    #4 _PyObject_VectorcallTstate Include/internal/pycore_call.h:167 (python+0x510279)
    #5 PyObject_Vectorcall Objects/call.c:327 (python+0x5103b4)
    #6 _PyEval_EvalFrameDefault Python/generated_cases.c.h:953 (python+0x77d0fa)
    #7 _PyEval_EvalFrame Include/internal/pycore_ceval.h:116 (python+0x7bb5f9)
    #8 _PyEval_Vector Python/ceval.c:1871 (python+0x7bb5f9)
    #9 _PyFunction_Vectorcall Objects/call.c:413 (python+0x50fa9b)
    #10 _PyObject_VectorcallTstate Include/internal/pycore_call.h:167 (python+0x649082)
    #11 vectorcall_unbound Objects/typeobject.c:2776 (python+0x649082)
    #12 vectorcall_method Objects/typeobject.c:2807 (python+0x649082)
    #13 slot_sq_item Objects/typeobject.c:9617 (python+0x649446)
    #14 PySequence_GetItem Objects/abstract.c:1847 (python+0x4dd14f)
    #15 iter_iternext Objects/iterobject.c:66 (python+0x572a7d)
    #16 _PyEval_EvalFrameDefault Python/generated_cases.c.h:3927 (python+0x794437)
    #17 _PyEval_EvalFrame Include/internal/pycore_ceval.h:116 (python+0x550b43)
    #18 gen_send_ex2 Objects/genobject.c:245 (python+0x550b43)
    #19 gen_iternext Objects/genobject.c:618 (python+0x5514e3)
    #20 iternext Objects/abstract.c:2874 (python+0x4d60e7)
    #21 PyIter_Next Objects/abstract.c:2924 (python+0x4dffb4)
    #22 builtin_sum_impl Python/bltinmodule.c:2716 (python+0x764943)
    #23 builtin_sum Python/clinic/bltinmodule.c.h:1167 (python+0x766254)
    #24 cfunction_vectorcall_FASTCALL_KEYWORDS Objects/methodobject.c:452 (python+0x5c9ec7)
    #25 _PyObject_VectorcallTstate Include/internal/pycore_call.h:167 (python+0x510279)
    #26 PyObject_Vectorcall Objects/call.c:327 (python+0x5103b4)
    #27 _PyEval_EvalFrameDefault Python/generated_cases.c.h:953 (python+0x77d0fa)
    #28 _PyEval_EvalFrame Include/internal/pycore_ceval.h:116 (python+0x7bb5f9)
    #29 _PyEval_Vector Python/ceval.c:1871 (python+0x7bb5f9)
    #30 _PyFunction_Vectorcall Objects/call.c:413 (python+0x50fa9b)
    #31 _PyObject_VectorcallTstate Include/internal/pycore_call.h:167 (python+0x515230)
    #32 method_vectorcall Objects/classobject.c:93 (python+0x51603f)
    #33 _PyVectorcall_Call Objects/call.c:273 (python+0x51303f)
    #34 _PyObject_Call Objects/call.c:348 (python+0x5134f9)
    #35 PyObject_Call Objects/call.c:373 (python+0x513578)
    #36 _PyEval_EvalFrameDefault Python/generated_cases.c.h:1745 (python+0x782ad6)
    #37 _PyEval_EvalFrame Include/internal/pycore_ceval.h:116 (python+0x7bb5f9)
    #38 _PyEval_Vector Python/ceval.c:1871 (python+0x7bb5f9)
    #39 _PyFunction_Vectorcall Objects/call.c:413 (python+0x50fa9b)
    #40 _PyObject_VectorcallTstate Include/internal/pycore_call.h:167 (python+0x51623d)
    #41 method_vectorcall Objects/classobject.c:71 (python+0x51623d)
    #42 _PyVectorcall_Call Objects/call.c:273 (python+0x51303f)
    #43 _PyObject_Call Objects/call.c:348 (python+0x5134f9)
    #44 PyObject_Call Objects/call.c:373 (python+0x513578)
    #45 thread_run Modules/_threadmodule.c:337 (python+0x985d20)
    #46 pythread_wrapper Python/thread_pthread.h:242 (python+0x8aaff3)

SUMMARY: ThreadSanitizer: data race Objects/memoryobject.c:1592 in memory_getbuf
==================
>>> >>> >>> 
Exception ignored in: <function SharedMemory.__del__ at 0x7f056b26f350>
Traceback (most recent call last):
  File "/home/mizuki/project/cpython/Lib/multiprocessing/shared_memory.py", line 189, in __del__
    self.close()
  File "/home/mizuki/project/cpython/Lib/multiprocessing/shared_memory.py", line 229, in close
    self._buf.release()
BufferError: memoryview has 2 exported buffers
python: Objects/memoryobject.c:1143: memory_dealloc: Assertion `self->exports == 0' failed.
/home/mizuki/project/cpython/Lib/multiprocessing/resource_tracker.py:276: UserWarning: resource_tracker: There appear to be 1 leaked shared_memory objects to clean up at shutdown: {'/psm_3a39690b'}
  warnings.warn(

Python 3.14.0a2+ experimental free-threading build (heads/main-dirty:db5c5763f3, Nov 28 2024, 21:44:57) [GCC 11.4.1 20231218 (Red Hat 11.4.1-3)] on linux
Type "help", "copyright", "credits" or "license" for more information.
warning: can't use pyrepl: No module named 'msvcrt'
>>> import multiprocessing.shared_memory
from threading import Thread

obj = multiprocessing.shared_memory.ShareableList("Uq..SeDAmB+EBrkLl.SG.Z+Z.ZdsV..wT+zLxKwdN\b")

for x in range(10):
    Thread(target=obj.count, args=(1,)).start()

del obj
>>> >>> >>> >>> >>> ... ... ==================
WARNING: ThreadSanitizer: data race (pid=3470244)
  Read of size 8 at 0x7f221cf8fb70 by thread T1:
    #0 memory_getbuf Objects/memoryobject.c:1593 (python+0x5c0ba8)
    #1 PyObject_GetBuffer Objects/abstract.c:442 (python+0x4d8844)
    #2 unpack_from Modules/clinic/_struct.c.h:370 (_struct.cpython-314td-x86_64-linux-gnu.so+0xc66a)
    #3 cfunction_vectorcall_FASTCALL_KEYWORDS Objects/methodobject.c:452 (python+0x5c9edc)
    #4 _PyObject_VectorcallTstate Include/internal/pycore_call.h:167 (python+0x510279)
    #5 PyObject_Vectorcall Objects/call.c:327 (python+0x5103b4)
    #6 _PyEval_EvalFrameDefault Python/generated_cases.c.h:953 (python+0x77d10f)
    #7 _PyEval_EvalFrame Include/internal/pycore_ceval.h:116 (python+0x7bb60e)
    #8 _PyEval_Vector Python/ceval.c:1871 (python+0x7bb60e)
    #9 _PyFunction_Vectorcall Objects/call.c:413 (python+0x50fa9b)
    #10 _PyObject_VectorcallTstate Include/internal/pycore_call.h:167 (python+0x649097)
    #11 vectorcall_unbound Objects/typeobject.c:2776 (python+0x649097)
    #12 vectorcall_method Objects/typeobject.c:2807 (python+0x649097)
    #13 slot_sq_item Objects/typeobject.c:9617 (python+0x64945b)
    #14 PySequence_GetItem Objects/abstract.c:1847 (python+0x4dd14f)
    #15 iter_iternext Objects/iterobject.c:66 (python+0x572a7d)
    #16 _PyEval_EvalFrameDefault Python/generated_cases.c.h:3927 (python+0x79444c)
    #17 _PyEval_EvalFrame Include/internal/pycore_ceval.h:116 (python+0x550b43)
    #18 gen_send_ex2 Objects/genobject.c:245 (python+0x550b43)
    #19 gen_iternext Objects/genobject.c:618 (python+0x5514e3)
    #20 iternext Objects/abstract.c:2874 (python+0x4d60e7)
    #21 PyIter_Next Objects/abstract.c:2924 (python+0x4dffb4)
    #22 builtin_sum_impl Python/bltinmodule.c:2716 (python+0x764958)
    #23 builtin_sum Python/clinic/bltinmodule.c.h:1167 (python+0x766269)
    #24 cfunction_vectorcall_FASTCALL_KEYWORDS Objects/methodobject.c:452 (python+0x5c9edc)
    #25 _PyObject_VectorcallTstate Include/internal/pycore_call.h:167 (python+0x510279)
    #26 PyObject_Vectorcall Objects/call.c:327 (python+0x5103b4)
    #27 _PyEval_EvalFrameDefault Python/generated_cases.c.h:953 (python+0x77d10f)
    #28 _PyEval_EvalFrame Include/internal/pycore_ceval.h:116 (python+0x7bb60e)
    #29 _PyEval_Vector Python/ceval.c:1871 (python+0x7bb60e)
    #30 _PyFunction_Vectorcall Objects/call.c:413 (python+0x50fa9b)
    #31 _PyObject_VectorcallTstate Include/internal/pycore_call.h:167 (python+0x515230)
    #32 method_vectorcall Objects/classobject.c:93 (python+0x51603f)
    #33 _PyVectorcall_Call Objects/call.c:273 (python+0x51303f)
    #34 _PyObject_Call Objects/call.c:348 (python+0x5134f9)
    #35 PyObject_Call Objects/call.c:373 (python+0x513578)
    #36 _PyEval_EvalFrameDefault Python/generated_cases.c.h:1745 (python+0x782aeb)
    #37 _PyEval_EvalFrame Include/internal/pycore_ceval.h:116 (python+0x7bb60e)
    #38 _PyEval_Vector Python/ceval.c:1871 (python+0x7bb60e)
    #39 _PyFunction_Vectorcall Objects/call.c:413 (python+0x50fa9b)
    #40 _PyObject_VectorcallTstate Include/internal/pycore_call.h:167 (python+0x51623d)
    #41 method_vectorcall Objects/classobject.c:71 (python+0x51623d)
    #42 _PyVectorcall_Call Objects/call.c:273 (python+0x51303f)
    #43 _PyObject_Call Objects/call.c:348 (python+0x5134f9)
    #44 PyObject_Call Objects/call.c:373 (python+0x513578)
    #45 thread_run Modules/_threadmodule.c:337 (python+0x985d35)
    #46 pythread_wrapper Python/thread_pthread.h:242 (python+0x8ab008)

  Previous write of size 8 at 0x7f221cf8fb70 by thread T2:
    #0 memory_releasebuf Objects/memoryobject.c:1603 (python+0x5bd1d3)
    #1 PyBuffer_Release Objects/abstract.c:812 (python+0x4d91bd)
    #2 unpack_from Modules/clinic/_struct.c.h:396 (_struct.cpython-314td-x86_64-linux-gnu.so+0xc8d4)
    #3 cfunction_vectorcall_FASTCALL_KEYWORDS Objects/methodobject.c:452 (python+0x5c9edc)
    #4 _PyObject_VectorcallTstate Include/internal/pycore_call.h:167 (python+0x510279)
    #5 PyObject_Vectorcall Objects/call.c:327 (python+0x5103b4)
    #6 _PyEval_EvalFrameDefault Python/generated_cases.c.h:953 (python+0x77d10f)
    #7 _PyEval_EvalFrame Include/internal/pycore_ceval.h:116 (python+0x7bb60e)
    #8 _PyEval_Vector Python/ceval.c:1871 (python+0x7bb60e)
    #9 _PyFunction_Vectorcall Objects/call.c:413 (python+0x50fa9b)
    #10 _PyObject_VectorcallTstate Include/internal/pycore_call.h:167 (python+0x649097)
    #11 vectorcall_unbound Objects/typeobject.c:2776 (python+0x649097)
    #12 vectorcall_method Objects/typeobject.c:2807 (python+0x649097)
    #13 slot_sq_item Objects/typeobject.c:9617 (python+0x64945b)
    #14 PySequence_GetItem Objects/abstract.c:1847 (python+0x4dd14f)
    #15 iter_iternext Objects/iterobject.c:66 (python+0x572a7d)
    #16 _PyEval_EvalFrameDefault Python/generated_cases.c.h:3927 (python+0x79444c)
    #17 _PyEval_EvalFrame Include/internal/pycore_ceval.h:116 (python+0x550b43)
    #18 gen_send_ex2 Objects/genobject.c:245 (python+0x550b43)
    #19 gen_iternext Objects/genobject.c:618 (python+0x5514e3)
    #20 iternext Objects/abstract.c:2874 (python+0x4d60e7)
    #21 PyIter_Next Objects/abstract.c:2924 (python+0x4dffb4)
    #22 builtin_sum_impl Python/bltinmodule.c:2716 (python+0x764958)
    #23 builtin_sum Python/clinic/bltinmodule.c.h:1167 (python+0x766269)
    #24 cfunction_vectorcall_FASTCALL_KEYWORDS Objects/methodobject.c:452 (python+0x5c9edc)
    #25 _PyObject_VectorcallTstate Include/internal/pycore_call.h:167 (python+0x510279)
    #26 PyObject_Vectorcall Objects/call.c:327 (python+0x5103b4)
    #27 _PyEval_EvalFrameDefault Python/generated_cases.c.h:953 (python+0x77d10f)
    #28 _PyEval_EvalFrame Include/internal/pycore_ceval.h:116 (python+0x7bb60e)
    #29 _PyEval_Vector Python/ceval.c:1871 (python+0x7bb60e)
    #30 _PyFunction_Vectorcall Objects/call.c:413 (python+0x50fa9b)
    #31 _PyObject_VectorcallTstate Include/internal/pycore_call.h:167 (python+0x515230)
    #32 method_vectorcall Objects/classobject.c:93 (python+0x51603f)
    #33 _PyVectorcall_Call Objects/call.c:273 (python+0x51303f)
    #34 _PyObject_Call Objects/call.c:348 (python+0x5134f9)
    #35 PyObject_Call Objects/call.c:373 (python+0x513578)
    #36 _PyEval_EvalFrameDefault Python/generated_cases.c.h:1745 (python+0x782aeb)
    #37 _PyEval_EvalFrame Include/internal/pycore_ceval.h:116 (python+0x7bb60e)
    #38 _PyEval_Vector Python/ceval.c:1871 (python+0x7bb60e)
    #39 _PyFunction_Vectorcall Objects/call.c:413 (python+0x50fa9b)
    #40 _PyObject_VectorcallTstate Include/internal/pycore_call.h:167 (python+0x51623d)
    #41 method_vectorcall Objects/classobject.c:71 (python+0x51623d)
    #42 _PyVectorcall_Call Objects/call.c:273 (python+0x51303f)
    #43 _PyObject_Call Objects/call.c:348 (python+0x5134f9)
    #44 PyObject_Call Objects/call.c:373 (python+0x513578)
    #45 thread_run Modules/_threadmodule.c:337 (python+0x985d35)
    #46 pythread_wrapper Python/thread_pthread.h:242 (python+0x8ab008)

SUMMARY: ThreadSanitizer: data race Objects/memoryobject.c:1593 in memory_getbuf
==================
>>> >>> 
Exception ignored in: <function SharedMemory.__del__ at 0x7f221cf81eb0>
Traceback (most recent call last):
  File "/home/mizuki/project/cpython/Lib/multiprocessing/shared_memory.py", line 189, in __del__
    self.close()
  File "/home/mizuki/project/cpython/Lib/multiprocessing/shared_memory.py", line 229, in close
    self._buf.release()
BufferError: memoryview has 7 exported buffers
python: Objects/memoryobject.c:1143: memory_dealloc: Assertion `self->exports == 0' failed.
/home/mizuki/project/cpython/Lib/multiprocessing/resource_tracker.py:276: UserWarning: resource_tracker: There appear to be 1 leaked shared_memory objects to clean up at shutdown: {'/psm_616253a2'}
  warnings.warn(

Issue: Calling ShareableList.count in threads aborts: Assertion 'self->exports == 0' failed #127085

ghost · 2024-11-29T13:40:32Z

All commit authors signed the Contributor License Agreement.

bedevere-app · 2024-11-29T13:40:36Z

Most changes to Python require a NEWS entry. Add one using the blurb_it web app or the blurb command-line tool.

If this change has little impact on Python users, wait for a maintainer to apply the skip news label instead.

bedevere-app · 2024-11-29T22:01:25Z

Most changes to Python require a NEWS entry. Add one using the blurb_it web app or the blurb command-line tool.

If this change has little impact on Python users, wait for a maintainer to apply the skip news label instead.

bedevere-app · 2024-11-30T00:41:02Z

Most changes to Python require a NEWS entry. Add one using the blurb_it web app or the blurb command-line tool.

If this change has little impact on Python users, wait for a maintainer to apply the skip news label instead.

ZeroIntensity

Thanks for contributing! I'm always happy to see new contributors 😄. Please address my comments and let me know if you have any questions.

Lib/test/test_free_threading/test_memoryview.py

Objects/memoryobject.c

… for reading.

bedevere-app · 2024-11-30T15:01:22Z

Most changes to Python require a NEWS entry. Add one using the blurb_it web app or the blurb command-line tool.

If this change has little impact on Python users, wait for a maintainer to apply the skip news label instead.

bedevere-app · 2024-11-30T15:01:34Z

Most changes to Python require a NEWS entry. Add one using the blurb_it web app or the blurb command-line tool.

If this change has little impact on Python users, wait for a maintainer to apply the skip news label instead.

ZeroIntensity · 2024-11-30T15:02:40Z

Please add a news entry so the bot stops spamming.

Objects/memoryobject.c

Lib/test/test_memoryview.py

Objects/memoryobject.c

…actions.

ZeroIntensity

Just some formatting nitpicks. I'll do a more thorough audit of the thread safety later today.

Objects/memoryobject.c

LindaSummer · 2024-12-01T14:34:54Z

Just some formatting nitpicks. I'll do a more thorough audit of the thread safety later today.

Hi @ZeroIntensity ,

Thanks very much for your patience and warm help! 😊

Wish a nice day!

Best Regards,
Edward

picnixz · 2024-12-02T13:06:55Z

@LindaSummer Please do not update the branch with the latest main unless you have conflicts to resolve. This wastes resources and notifies everyone subscribed to the PR.

ZeroIntensity

OK, on the basis that there will be a follow-up PR, this LGTM. Thanks, @LindaSummer!

LindaSummer · 2024-12-02T15:00:52Z

OK, on the basis that there will be a follow-up PR, this LGTM. Thanks, @LindaSummer!

Hi @ZeroIntensity , @picnixz ,

Thanks again for your guidance and patience! ❤

Best Regards,
Edward

LindaSummer · 2024-12-05T05:32:17Z

Hi @ZeroIntensity ,

Sorry to bother you again. 😊

This PR has been pending for three days in awaiting core review.

Should I mention someone in this thread for next steps? 😊

Best Regards,
Edward

ZeroIntensity · 2024-12-05T12:58:30Z

No worries. Typically the topic-free-threading label is enough to get one of the free threading experts to look at it, but not always. Tag me next week if nobody has taken a look.

ZeroIntensity

Sorry, I didn't notice the blurb before. That needs to get changed.

Misc/NEWS.d/next/Core_and_Builtins/2024-11-30-23-35-45.gh-issue-127085.KLKylb.rst

…hongh-127085

kumaraditya303 · 2024-12-09T14:09:54Z

I'll be reviewing this by next week

…-127085

kumaraditya303

LGTM, this doesn't fix all of the thread safety issues but is a step in right direction so I'm merging it

miss-islington-app · 2024-12-16T19:12:25Z

Thanks @LindaSummer for the PR, and @kumaraditya303 for merging it 🌮🎉.. I'm working now to backport this PR to: 3.13.
🐍🍒⛏🤖

miss-islington-app · 2024-12-16T19:12:38Z

Sorry, @LindaSummer and @kumaraditya303, I could not cleanly backport this to 3.13 due to a conflict.
Please backport using cherry_picker on command line.

cherry_picker 4937ba54c0ff7cc4a83d7345d398b804365af2d6 3.13

kumaraditya303 · 2024-12-16T19:14:06Z

I'll take of backporting this if needed as part of #127716

bedevere-bot · 2024-12-16T19:29:04Z

⚠️⚠️⚠️ Buildbot failure ⚠️⚠️⚠️

Hi! The buildbot aarch64 Android 3.x has failed when building commit 4937ba5.

What do you need to do:

Don't panic.
Check the buildbot page in the devguide if you don't know what the buildbots are or how they work.
Go to the page of the buildbot that failed (https://buildbot.python.org/#/builders/1594/builds/825) and take a look at the build logs.
Check if the failure is related to this commit (4937ba5) or if it is a false positive.
If the failure is related to this commit, please, reflect that on the issue and make a new Pull Request with a fix.

You can take a look at the buildbot page here:

https://buildbot.python.org/#/builders/1594/builds/825

Failed tests:

test_memoryview

Summary of the results of the build (if available):

==

Click to see traceback logs

Traceback (most recent call last):
  File "/data/user/0/org.python.testbed/files/python/lib/python3.14/test/test_memoryview.py", line 742, in test_racing_getbuf_and_releasebuf
    from multiprocessing.managers import SharedMemoryManager
ImportError: cannot import name 'SharedMemoryManager' from 'multiprocessing.managers' (/data/user/0/org.python.testbed/files/python/lib/python3.14/multiprocessing/managers.py)

bedevere-bot · 2024-12-16T19:33:01Z

⚠️⚠️⚠️ Buildbot failure ⚠️⚠️⚠️

Hi! The buildbot iOS ARM64 Simulator 3.x has failed when building commit 4937ba5.

What do you need to do:

Don't panic.
Check the buildbot page in the devguide if you don't know what the buildbots are or how they work.
Go to the page of the buildbot that failed (https://buildbot.python.org/#/builders/1380/builds/2128) and take a look at the build logs.
Check if the failure is related to this commit (4937ba5) or if it is a false positive.
If the failure is related to this commit, please, reflect that on the issue and make a new Pull Request with a fix.

You can take a look at the buildbot page here:

https://buildbot.python.org/#/builders/1380/builds/2128

Summary of the results of the build (if available):

==

Click to see traceback logs

Traceback (most recent call last):

bedevere-bot · 2024-12-16T19:44:22Z

⚠️⚠️⚠️ Buildbot failure ⚠️⚠️⚠️

Hi! The buildbot AMD64 Android 3.x has failed when building commit 4937ba5.

What do you need to do:

Don't panic.
Check the buildbot page in the devguide if you don't know what the buildbots are or how they work.
Go to the page of the buildbot that failed (https://buildbot.python.org/#/builders/1591/builds/728) and take a look at the build logs.
Check if the failure is related to this commit (4937ba5) or if it is a false positive.
If the failure is related to this commit, please, reflect that on the issue and make a new Pull Request with a fix.

You can take a look at the buildbot page here:

https://buildbot.python.org/#/builders/1591/builds/728

Failed tests:

test_memoryview

Summary of the results of the build (if available):

==

Click to see traceback logs

Traceback (most recent call last):
  File "/data/user/0/org.python.testbed/files/python/lib/python3.14/test/test_memoryview.py", line 742, in test_racing_getbuf_and_releasebuf
    from multiprocessing.managers import SharedMemoryManager
ImportError: cannot import name 'SharedMemoryManager' from 'multiprocessing.managers' (/data/user/0/org.python.testbed/files/python/lib/python3.14/multiprocessing/managers.py)

mhsmith · 2024-12-16T21:08:10Z

iOS and Android don't support subprocesses, and therefore don't support the multiprocessing. It looks like this test doesn't actually use multiple processes, so maybe it can be fixed to avoid the use of multiprocessing. Otherwise, it'll have to be given a decorator to skip it when multiprocessing is unavailable.

LindaSummer · 2024-12-17T00:38:03Z

iOS and Android don't support subprocesses, and therefore don't support the multiprocessing. It looks like this test doesn't actually use multiple processes, so maybe it can be fixed to avoid the use of multiprocessing. Otherwise, it'll have to be given a decorator to skip it when multiprocessing is unavailable.

Hi @mhsmith ,

Thanks for your suggestion! 😊

I will try to fix the test case issue.

Best Regards,
Edward

…ython#127412)

hugovk · 2025-02-26T13:38:02Z

I'll take of backporting this if needed as part of #127716

@kumaraditya303 Triage: Please could you make the backport if needed? Otherwise let's remove the label. Thanks!

pythongh-127085: fix memoryview->exports race condition.

c7872cd

bedevere-app bot mentioned this pull request Nov 29, 2024

Calling ShareableList.count in threads aborts: Assertion 'self->exports == 0' failed #127085

Closed

bedevere-app bot added the awaiting review label Nov 29, 2024

Merge branch 'main' into bugfix/pythongh-127085

06f462f

Merge branch 'main' into bugfix/pythongh-127085

9e6402c

ZeroIntensity added the needs backport to 3.13 bugs and security fixes label Nov 30, 2024

ZeroIntensity reviewed Nov 30, 2024

View reviewed changes

pythongh-127085: use atomic updating for exports and add protection…

bb9d67e

… for reading.

Merge branch 'main' into bugfix/pythongh-127085

376f382

ZeroIntensity reviewed Nov 30, 2024

View reviewed changes

Objects/memoryobject.c Show resolved Hide resolved

ZeroIntensity reviewed Nov 30, 2024

View reviewed changes

Lib/test/test_memoryview.py Show resolved Hide resolved

ZeroIntensity added the topic-free-threading label Nov 30, 2024

picnixz reviewed Nov 30, 2024

View reviewed changes

Lib/test/test_memoryview.py Outdated Show resolved Hide resolved

Lib/test/test_memoryview.py Outdated Show resolved Hide resolved

Objects/memoryobject.c Show resolved Hide resolved

LindaSummer added 4 commits December 1, 2024 00:01

pythongh-127085: refactor test case and add macro for exports atomic …

49957d1

…actions.

Merge branch 'main' into bugfix/pythongh-127085

3991201

Merge branch 'main' into bugfix/pythongh-127085

d4dc897

Merge branch 'main' into bugfix/pythongh-127085

e287b88

ZeroIntensity reviewed Dec 1, 2024

View reviewed changes

Objects/memoryobject.c Outdated Show resolved Hide resolved

Objects/memoryobject.c Outdated Show resolved Hide resolved

pythongh-127085: fix formatting issue.

666fde3

LindaSummer added 2 commits December 2, 2024 12:49

Merge branch 'main' into bugfix/pythongh-127085

a3d58f1

Merge branch 'main' into bugfix/pythongh-127085

d348113

ZeroIntensity approved these changes Dec 2, 2024

View reviewed changes

bedevere-app bot added awaiting core review and removed awaiting review labels Dec 2, 2024

picnixz mentioned this pull request Dec 7, 2024

make memoryview thread safe in free-threading #127716

Open

ZeroIntensity reviewed Dec 7, 2024

View reviewed changes

Misc/NEWS.d/next/Core_and_Builtins/2024-11-30-23-35-45.gh-issue-127085.KLKylb.rst Outdated Show resolved Hide resolved

LindaSummer added 2 commits December 8, 2024 23:10

Merge branch 'main' of github.com:LindaSummer/cpython into bugfix/pyt…

9594f38

…hongh-127085

rewrite the blurb changelog.

102d30a

kumaraditya303 self-assigned this Dec 9, 2024

Merge branch 'main' of github.com:python/cpython into bugfix/pythongh…

0ca811e

…-127085

kumaraditya303 approved these changes Dec 16, 2024

View reviewed changes

bedevere-app bot added awaiting merge and removed awaiting core review labels Dec 16, 2024

kumaraditya303 merged commit 4937ba5 into python:main Dec 16, 2024
45 checks passed

bedevere-app bot removed the awaiting merge label Dec 16, 2024

freakboy3742 mentioned this pull request Dec 17, 2024

gh-127085: Add a test skip if multiprocessing isn't available #128019

Merged

srinivasreddy pushed a commit to srinivasreddy/cpython that referenced this pull request Jan 8, 2025

pythongh-127085: fix some data races in memoryview in free-threading (p…

0be115d

…ython#127412)

mhsmith mentioned this pull request Jan 23, 2025

iOS buildbot failure comments don't contain failure details anymore #129248

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

gh-127085: Fix `memoryview->exports` race condition. #127412

gh-127085: Fix `memoryview->exports` race condition. #127412

LindaSummer commented Nov 29, 2024 •

edited by bedevere-app bot

Loading

ghost commented Nov 29, 2024 •

edited by ghost

Loading

bedevere-app bot commented Nov 29, 2024

bedevere-app bot commented Nov 29, 2024

bedevere-app bot commented Nov 30, 2024

ZeroIntensity left a comment

bedevere-app bot commented Nov 30, 2024

bedevere-app bot commented Nov 30, 2024

ZeroIntensity commented Nov 30, 2024

ZeroIntensity left a comment

LindaSummer commented Dec 1, 2024

picnixz commented Dec 2, 2024

ZeroIntensity left a comment

LindaSummer commented Dec 2, 2024

LindaSummer commented Dec 5, 2024

ZeroIntensity commented Dec 5, 2024

ZeroIntensity left a comment

kumaraditya303 commented Dec 9, 2024 •

edited

Loading

kumaraditya303 left a comment

miss-islington-app bot commented Dec 16, 2024

miss-islington-app bot commented Dec 16, 2024

kumaraditya303 commented Dec 16, 2024

bedevere-bot commented Dec 16, 2024

bedevere-bot commented Dec 16, 2024

bedevere-bot commented Dec 16, 2024

mhsmith commented Dec 16, 2024

LindaSummer commented Dec 17, 2024

hugovk commented Feb 26, 2025

gh-127085: Fix memoryview->exports race condition. #127412

gh-127085: Fix memoryview->exports race condition. #127412

Conversation

LindaSummer commented Nov 29, 2024 • edited by bedevere-app bot Loading

Issue

Proposed Changes

Cause of the problem

ghost commented Nov 29, 2024 • edited by ghost Loading

bedevere-app bot commented Nov 29, 2024

bedevere-app bot commented Nov 29, 2024

bedevere-app bot commented Nov 30, 2024

ZeroIntensity left a comment

Choose a reason for hiding this comment

bedevere-app bot commented Nov 30, 2024

bedevere-app bot commented Nov 30, 2024

ZeroIntensity commented Nov 30, 2024

ZeroIntensity left a comment

Choose a reason for hiding this comment

LindaSummer commented Dec 1, 2024

picnixz commented Dec 2, 2024

ZeroIntensity left a comment

Choose a reason for hiding this comment

LindaSummer commented Dec 2, 2024

LindaSummer commented Dec 5, 2024

ZeroIntensity commented Dec 5, 2024

ZeroIntensity left a comment

Choose a reason for hiding this comment

kumaraditya303 commented Dec 9, 2024 • edited Loading

kumaraditya303 left a comment

Choose a reason for hiding this comment

miss-islington-app bot commented Dec 16, 2024

miss-islington-app bot commented Dec 16, 2024

kumaraditya303 commented Dec 16, 2024

bedevere-bot commented Dec 16, 2024

⚠️⚠️⚠️ Buildbot failure ⚠️⚠️⚠️

bedevere-bot commented Dec 16, 2024

⚠️⚠️⚠️ Buildbot failure ⚠️⚠️⚠️

bedevere-bot commented Dec 16, 2024

⚠️⚠️⚠️ Buildbot failure ⚠️⚠️⚠️

mhsmith commented Dec 16, 2024

LindaSummer commented Dec 17, 2024

hugovk commented Feb 26, 2025

gh-127085: Fix `memoryview->exports` race condition. #127412

gh-127085: Fix `memoryview->exports` race condition. #127412

LindaSummer commented Nov 29, 2024 •

edited by bedevere-app bot

Loading

ghost commented Nov 29, 2024 •

edited by ghost

Loading

kumaraditya303 commented Dec 9, 2024 •

edited

Loading