-
-
Notifications
You must be signed in to change notification settings - Fork 31.9k
gh-112075: _Py_dict_lookup needs to lock shared keys #117528
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Conversation
c51437d
to
f3e3db3
Compare
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This looks right, but I'm still seeing issues with the test that uncovered the problem. I'll investigate more today.
8942e4d
to
2f5f5a1
Compare
35cf56e
to
d580aa5
Compare
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I'm still looking at _Py_dict_lookup
, but I think insertdict
and dict_setdefault_ref_lock_held
can be simpler if they handle split dictionaries early on and separate from the main code path.
cfd659c
to
19d1446
Compare
…sertdict and setdefault
19d1446
to
a74c0ee
Compare
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
LGTM other than the unused code warning in the default build
@@ -1171,6 +1188,10 @@ _PyDictKeys_StringLookup(PyDictKeysObject* dk, PyObject *key) | |||
return unicodekeys_lookup_unicode(dk, key, hash); | |||
} | |||
|
|||
static Py_ssize_t | |||
unicodekeys_lookup_unicode_threadsafe(PyDictKeysObject* dk, PyObject *key, |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Unused code warning
_Py_dict_lookup
needs to lock the shared keys if we have a split dictionary. If we're looking up with a non-exact unicode we need to also incref the keys as the lookup could mutate the keys and we could lose the last reference.insertdict
is updated to avoid contention on the shared dict lookup by calling the threadsafe unicode lookup directly and only falling back to_Py_dict_lookup
if the thread safe lookup can't succeed.dict
objects thread-safe in--disable-gil
builds #112075