Skip to content

Incorrect MRO cache entry is populated for types that do not have a version assigned #128108

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
mpage opened this issue Dec 20, 2024 · 2 comments
Labels
interpreter-core (Objects, Python, Grammar, and Parser dirs) type-bug An unexpected behavior, bug, or error

Comments

@mpage
Copy link
Contributor

mpage commented Dec 20, 2024

Bug report

Bug description:

_PyType_LookupRefAndVersion will populate the incorrect cache entry if it is called with a type that does not have a version tag (tp_version_tag is 0). I think this only impacts performance, not correctness. The next call to _PyType_LookupRefAndVersion for the same (type, name) pair will miss the cache, but will populate the correct entry, so any subsequent calls will hit the cache. Additionally, this may create some amount of unnecessary thrash on the cache.

The root of the issue is that the version tag is an input to the hash that is used to determine the cache entry and we do not recompute the hash, and the associated cache entry, after assigning a version.

_PyType_LookupRefAndVersion first computes the cache entry:

cpython/Objects/typeobject.c

Lines 5542 to 5544 in 39e69a7

unsigned int h = MCACHE_HASH_METHOD(type, name);
struct type_cache *cache = get_type_cache();
struct type_cache_entry *entry = &cache->hashtable[h];

If the cache entry doesn't match, and a type version was assigned successfully, it populates the same entry:

cpython/Objects/typeobject.c

Lines 5629 to 5634 in 39e69a7

#if Py_GIL_DISABLED
update_cache_gil_disabled(entry, name, assigned_version, res);
#else
PyObject *old_value = update_cache(entry, name, assigned_version, res);
Py_DECREF(old_value);
#endif

The next call for the same (type, name) pair, will find a different entry, because the type version is used to compute the hash that indexes into the cache:

#define MCACHE_HASH_METHOD(type, name) \
MCACHE_HASH(FT_ATOMIC_LOAD_UINT32_RELAXED((type)->tp_version_tag), \
((Py_ssize_t)(name)) >> 3)

It looks like the change to use the same entry was introduced in gh-113930. cc @DinoV

CPython versions tested on:

3.13, 3.14

Operating systems tested on:

Linux

@mpage mpage added type-bug An unexpected behavior, bug, or error interpreter-core (Objects, Python, Grammar, and Parser dirs) labels Dec 20, 2024
@DinoV
Copy link
Contributor

DinoV commented Jan 29, 2025

Maybe the fix here is to just use the type's address as the entry into hash table.

@mpage
Copy link
Contributor Author

mpage commented Jan 30, 2025

Maybe the fix here is to just use the type's address as the entry into hash table.

That sounds reasonable to me, assuming you mean using the address of the type in place of the version. Though I assume there was some reason why they used the version in the first place...

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
interpreter-core (Objects, Python, Grammar, and Parser dirs) type-bug An unexpected behavior, bug, or error
Projects
None yet
Development

No branches or pull requests

2 participants