Skip to content

Race between _PyObject_GenericGetAttrWithDict and ensure_nonmanaged_dict under free-threading #128100

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Closed
hawkinsp opened this issue Dec 19, 2024 · 3 comments
Closed

Race between _PyObject_GenericGetAttrWithDict and ensure_nonmanaged_dict under free-threading #128100

hawkinsp opened this issue Dec 19, 2024 · 3 comments
Labels
3.13 bugs and security fixes 3.14 bugs and security fixes topic-free-threading type-bug An unexpected behavior, bug, or error

Comments

@hawkinsp
Copy link
Contributor

hawkinsp commented Dec 19, 2024 •
edited by bedevere-app bot
Loading

Bug report

Bug description:

Under Python 3.13.1t build with thread-sanitizer:

import concurrent.futures
import functools
import threading

num_threads = 100


def closure(b, o):
  b.wait()
  getattr(o, "foo", None)
  o.foo = 42

with concurrent.futures.ThreadPoolExecutor(max_workers=num_threads) as executor:
  for _ in range(100):
    b = threading.Barrier(num_threads)
    o = functools.partial(lambda x: x, 42)
    for _ in range(num_threads):
      executor.submit(functools.partial(closure, b, o))

Get (it may take a few runs):

WARNING: ThreadSanitizer: data race (pid=239136)
  Read of size 8 at 0x7fa5e69e6a28 by thread T91:
    #0 _PyObject_GenericGetAttrWithDict /usr/local/google/home/phawkins/p/cpython/Objects/object.c:1665:24 (python3.13+0x2960a5) (BuildId: 9c1c16fb1bb8a435fa6fa4c6944da5d41f654e96)
    #1 PyObject_GetOptionalAttr /usr/local/google/home/phawkins/p/cpython/Objects/object.c:1266:19 (python3.13+0x2930c4) (BuildId: 9c1c16fb1bb8a435fa6fa4c6944da5d41f654e96)
    #2 builtin_getattr /usr/local/google/home/phawkins/p/cpython/Python/bltinmodule.c:1200:13 (python3.13+0x3d8bf9) (BuildId: 9c1c16fb1bb8a435fa6fa4c6944da5d41f654e96)
    #3 cfunction_vectorcall_FASTCALL /usr/local/google/home/phawkins/p/cpython/Objects/methodobject.c:425:24 (python3.13+0x289efb) (BuildId: 9c1c16fb1bb8a435fa6fa4c6944da5d41f654e96)
    #4 _PyObject_VectorcallTstate /usr/local/google/home/phawkins/p/cpython/./Include/internal/pycore_call.h:168:11 (python3.13+0x1eafaa) (BuildId: 9c1c16fb1bb8a435fa6fa4c6944da5d41f654e96)
    #5 PyObject_Vectorcall /usr/local/google/home/phawkins/p/cpython/Objects/call.c:327:12 (python3.13+0x1eafaa)
    #6 _PyEval_EvalFrameDefault /usr/local/google/home/phawkins/p/cpython/Python/generated_cases.c.h:813:23 (python3.13+0x3e24fb) (BuildId: 9c1c16fb1bb8a435fa6fa4c6944da5d41f654e96)
    #7 _PyEval_EvalFrame /usr/local/google/home/phawkins/p/cpython/./Include/internal/pycore_ceval.h:119:16 (python3.13+0x3de62a) (BuildId: 9c1c16fb1bb8a435fa6fa4c6944da5d41f654e96)
    #8 _PyEval_Vector /usr/local/google/home/phawkins/p/cpython/Python/ceval.c:1811:12 (python3.13+0x3de62a)
    #9 _PyFunction_Vectorcall /usr/local/google/home/phawkins/p/cpython/Objects/call.c (python3.13+0x1eb61f) (BuildId: 9c1c16fb1bb8a435fa6fa4c6944da5d41f654e96)
    #10 _PyObject_VectorcallTstate /usr/local/google/home/phawkins/p/cpython/./Include/internal/pycore_call.h:168:11 (python3.13+0x571bb2) (BuildId: 9c1c16fb1bb8a435fa6fa4c6944da5d41f654e96)
    #11 partial_vectorcall /usr/local/google/home/phawkins/p/cpython/./Modules/_functoolsmodule.c:252:16 (python3.13+0x571bb2)
    #12 _PyVectorcall_Call /usr/local/google/home/phawkins/p/cpython/Objects/call.c:273:16 (python3.13+0x1eb293) (BuildId: 9c1c16fb1bb8a435fa6fa4c6944da5d41f654e96)
    #13 _PyObject_Call /usr/local/google/home/phawkins/p/cpython/Objects/call.c:348:16 (python3.13+0x1eb293)
    #14 PyObject_Call /usr/local/google/home/phawkins/p/cpython/Objects/call.c:373:12 (python3.13+0x1eb315) (BuildId: 9c1c16fb1bb8a435fa6fa4c6944da5d41f654e96)
    #15 _PyEval_EvalFrameDefault /usr/local/google/home/phawkins/p/cpython/Python/generated_cases.c.h:1355:26 (python3.13+0x3e46e2) (BuildId: 9c1c16fb1bb8a435fa6fa4c6944da5d41f654e96)
    #16 _PyEval_EvalFrame /usr/local/google/home/phawkins/p/cpython/./Include/internal/pycore_ceval.h:119:16 (python3.13+0x3de62a) (BuildId: 9c1c16fb1bb8a435fa6fa4c6944da5d41f654e96)
    #17 _PyEval_Vector /usr/local/google/home/phawkins/p/cpython/Python/ceval.c:1811:12 (python3.13+0x3de62a)
    #18 _PyFunction_Vectorcall /usr/local/google/home/phawkins/p/cpython/Objects/call.c (python3.13+0x1eb61f) (BuildId: 9c1c16fb1bb8a435fa6fa4c6944da5d41f654e96)
    #19 _PyObject_VectorcallTstate /usr/local/google/home/phawkins/p/cpython/./Include/internal/pycore_call.h:168:11 (python3.13+0x1ef5ef) (BuildId: 9c1c16fb1bb8a435fa6fa4c6944da5d41f654e96)
    #20 method_vectorcall /usr/local/google/home/phawkins/p/cpython/Objects/classobject.c:70:20 (python3.13+0x1ef5ef)
    #21 _PyVectorcall_Call /usr/local/google/home/phawkins/p/cpython/Objects/call.c:273:16 (python3.13+0x1eb293) (BuildId: 9c1c16fb1bb8a435fa6fa4c6944da5d41f654e96)
    #22 _PyObject_Call /usr/local/google/home/phawkins/p/cpython/Objects/call.c:348:16 (python3.13+0x1eb293)
    #23 PyObject_Call /usr/local/google/home/phawkins/p/cpython/Objects/call.c:373:12 (python3.13+0x1eb315) (BuildId: 9c1c16fb1bb8a435fa6fa4c6944da5d41f654e96)
    #24 thread_run /usr/local/google/home/phawkins/p/cpython/./Modules/_threadmodule.c:337:21 (python3.13+0x564292) (BuildId: 9c1c16fb1bb8a435fa6fa4c6944da5d41f654e96)
    #25 pythread_wrapper /usr/local/google/home/phawkins/p/cpython/Python/thread_pthread.h:243:5 (python3.13+0x4bd637) (BuildId: 9c1c16fb1bb8a435fa6fa4c6944da5d41f654e96)

  Previous atomic write of size 8 at 0x7fa5e69e6a28 by thread T18:
    #0 _Py_atomic_store_ptr_release /usr/local/google/home/phawkins/p/cpython/./Include/cpython/pyatomic_gcc.h:501:3 (python3.13+0x272a17) (BuildId: 9c1c16fb1bb8a435fa6fa4c6944da5d41f654e96)
    #1 ensure_nonmanaged_dict /usr/local/google/home/phawkins/p/cpython/Objects/dictobject.c:7275:9 (python3.13+0x272a17)
    #2 _PyObjectDict_SetItem /usr/local/google/home/phawkins/p/cpython/Objects/dictobject.c:7311:12 (python3.13+0x272aee) (BuildId: 9c1c16fb1bb8a435fa6fa4c6944da5d41f654e96)
    #3 _PyObject_GenericSetAttrWithDict /usr/local/google/home/phawkins/p/cpython/Objects/object.c:1791:19 (python3.13+0x297c8e) (BuildId: 9c1c16fb1bb8a435fa6fa4c6944da5d41f654e96)
    #4 PyObject_GenericSetAttr /usr/local/google/home/phawkins/p/cpython/Objects/object.c:1819:12 (python3.13+0x298667) (BuildId: 9c1c16fb1bb8a435fa6fa4c6944da5d41f654e96)
    #5 PyObject_SetAttr /usr/local/google/home/phawkins/p/cpython/Objects/object.c:1385:15 (python3.13+0x294f8a) (BuildId: 9c1c16fb1bb8a435fa6fa4c6944da5d41f654e96)
    #6 _PyEval_EvalFrameDefault /usr/local/google/home/phawkins/p/cpython/Python/generated_cases.c.h:5488:27 (python3.13+0x3f3f6d) (BuildId: 9c1c16fb1bb8a435fa6fa4c6944da5d41f654e96)
    #7 _PyEval_EvalFrame /usr/local/google/home/phawkins/p/cpython/./Include/internal/pycore_ceval.h:119:16 (python3.13+0x3de62a) (BuildId: 9c1c16fb1bb8a435fa6fa4c6944da5d41f654e96)
    #8 _PyEval_Vector /usr/local/google/home/phawkins/p/cpython/Python/ceval.c:1811:12 (python3.13+0x3de62a)
    #9 _PyFunction_Vectorcall /usr/local/google/home/phawkins/p/cpython/Objects/call.c (python3.13+0x1eb61f) (BuildId: 9c1c16fb1bb8a435fa6fa4c6944da5d41f654e96)
    #10 _PyObject_VectorcallTstate /usr/local/google/home/phawkins/p/cpython/./Include/internal/pycore_call.h:168:11 (python3.13+0x571bb2) (BuildId: 9c1c16fb1bb8a435fa6fa4c6944da5d41f654e96)
    #11 partial_vectorcall /usr/local/google/home/phawkins/p/cpython/./Modules/_functoolsmodule.c:252:16 (python3.13+0x571bb2)
    #12 _PyVectorcall_Call /usr/local/google/home/phawkins/p/cpython/Objects/call.c:273:16 (python3.13+0x1eb293) (BuildId: 9c1c16fb1bb8a435fa6fa4c6944da5d41f654e96)
    #13 _PyObject_Call /usr/local/google/home/phawkins/p/cpython/Objects/call.c:348:16 (python3.13+0x1eb293)
    #14 PyObject_Call /usr/local/google/home/phawkins/p/cpython/Objects/call.c:373:12 (python3.13+0x1eb315) (BuildId: 9c1c16fb1bb8a435fa6fa4c6944da5d41f654e96)
    #15 _PyEval_EvalFrameDefault /usr/local/google/home/phawkins/p/cpython/Python/generated_cases.c.h:1355:26 (python3.13+0x3e46e2) (BuildId: 9c1c16fb1bb8a435fa6fa4c6944da5d41f654e96)
    #16 _PyEval_EvalFrame /usr/local/google/home/phawkins/p/cpython/./Include/internal/pycore_ceval.h:119:16 (python3.13+0x3de62a) (BuildId: 9c1c16fb1bb8a435fa6fa4c6944da5d41f654e96)
    #17 _PyEval_Vector /usr/local/google/home/phawkins/p/cpython/Python/ceval.c:1811:12 (python3.13+0x3de62a)
    #18 _PyFunction_Vectorcall /usr/local/google/home/phawkins/p/cpython/Objects/call.c (python3.13+0x1eb61f) (BuildId: 9c1c16fb1bb8a435fa6fa4c6944da5d41f654e96)
    #19 _PyObject_VectorcallTstate /usr/local/google/home/phawkins/p/cpython/./Include/internal/pycore_call.h:168:11 (python3.13+0x1ef5ef) (BuildId: 9c1c16fb1bb8a435fa6fa4c6944da5d41f654e96)
    #20 method_vectorcall /usr/local/google/home/phawkins/p/cpython/Objects/classobject.c:70:20 (python3.13+0x1ef5ef)
    #21 _PyVectorcall_Call /usr/local/google/home/phawkins/p/cpython/Objects/call.c:273:16 (python3.13+0x1eb293) (BuildId: 9c1c16fb1bb8a435fa6fa4c6944da5d41f654e96)
    #22 _PyObject_Call /usr/local/google/home/phawkins/p/cpython/Objects/call.c:348:16 (python3.13+0x1eb293)
    #23 PyObject_Call /usr/local/google/home/phawkins/p/cpython/Objects/call.c:373:12 (python3.13+0x1eb315) (BuildId: 9c1c16fb1bb8a435fa6fa4c6944da5d41f654e96)
    #24 thread_run /usr/local/google/home/phawkins/p/cpython/./Modules/_threadmodule.c:337:21 (python3.13+0x564292) (BuildId: 9c1c16fb1bb8a435fa6fa4c6944da5d41f654e96)
    #25 pythread_wrapper /usr/local/google/home/phawkins/p/cpython/Python/thread_pthread.h:243:5 (python3.13+0x4bd637) (BuildId: 9c1c16fb1bb8a435fa6fa4c6944da5d41f654e96)

CPython versions tested on:

3.13

Operating systems tested on:

Linux

Linked PRs
@hawkinsp hawkinsp added the type-bug An unexpected behavior, bug, or error label Dec 19, 2024
@ZeroIntensity ZeroIntensity added 3.13 bugs and security fixes topic-free-threading 3.14 bugs and security fixes labels Dec 19, 2024
@colesbury
Copy link
Contributor

I think there are two data races related to this:

  1. The dictionary load in _PyObject_GenericGetAttrWithDict should be atomic (with acquire ordering since we're not using consume)
  2. We need to handle the case where you change the dictionary (via obj.__dict__ = newdict). That doesn't show up in this example, but it's relevant since it may also race with the load. That was partly addressed in the following PR, but I don't think PyObject_GenericSetDict got updated.

@wrongnull
Copy link
Contributor

I think it is worth adding _PyObject_GenericGetAttrWithDict here #117657 since it is reproducible in the manner described in this issue

@bedevere-app bedevere-app bot mentioned this issue Dec 27, 2024
Merged
colesbury pushed a commit that referenced this issue Dec 30, 2024
@wrongnull
gh-128100: Use atomic dictionary load in `_PyObject_GenericGetAttrWit…
47d2cb8 
…hDict` (GH-128297)
srinivasreddy pushed a commit to srinivasreddy/cpython that referenced this issue Jan 8, 2025
@wrongnull @srinivasreddy
pythongh-128100: Use atomic dictionary load in `_PyObject_GenericGetA…
b5cc4cf 
…ttrWithDict` (pythonGH-128297)
miss-islington pushed a commit to miss-islington/cpython that referenced this issue Feb 10, 2025
@wrongnull @miss-islington
pythongh-128100: Use atomic dictionary load in `_PyObject_GenericGetA…
0cf631d 
…ttrWithDict` (pythonGH-128297)

(cherry picked from commit 47d2cb8)

Co-authored-by: Bogdan Romanyuk <[email protected]>
@bedevere-app bedevere-app bot mentioned this issue Feb 10, 2025
Merged
colesbury pushed a commit that referenced this issue Feb 10, 2025
@miss-islington @wrongnull
[3.13] gh-128100: Use atomic dictionary load in `_PyObject_GenericGet…
0526322 
…AttrWithDict` (GH-128297) (GH-129979)

(cherry picked from commit 47d2cb8)

Co-authored-by: Bogdan Romanyuk <[email protected]>
@ZeroIntensity
Copy link
Member

Thanks @hawkinsp for the report and @wrongnull for the fix!

@hawkinsp hawkinsp mentioned this issue Feb 26, 2025
Closed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
3.13 bugs and security fixes 3.14 bugs and security fixes topic-free-threading type-bug An unexpected behavior, bug, or error
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
4 participants
@hawkinsp @colesbury @ZeroIntensity @wrongnull