gh-116738: Make _codecs module thread-safe
#117530
Merged
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
swtaarrs
added
skip news
interpreter-core
colesbury
reviewed
Apr 10, 2024
colesbury
reviewed
Apr 15, 2024
…istry's return type
|
This should be ready to go as of my last commit - let me know if any other changes are needed. |
colesbury
approved these changes
May 2, 2024
SonicField
pushed a commit
to SonicField/cpython
that referenced
this pull request
May 8, 2024
The module itself is a thin wrapper around calls to functions in `Python/codecs.c`, so that's where the meaningful changes happened: - Move codecs-related state that lives on `PyInterpreterState` to a struct declared in `pycore_codecs.h`. - In free-threaded builds, add a mutex to `codecs_state` to synchronize operations on `search_path`. Because `search_path_mutex` is used as a normal mutex and not a critical section, we must be extremely careful with operations called while holding it. - The codec registry is explicitly initialized as part of `_PyUnicode_InitEncodings` to simplify thread-safety.
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
The module itself is a thin wrapper around calls to functions in
Python/codecs.c, so that's where the meaningful changes happened:Move codecs-related state that lives on
PyInterpreterStateto a struct declared inpycore_codecs.h.In free-threaded builds, add two mutexes to
codecs_state, one to synchronize operations onsearch_path, and one to synchronize initialization._PyCodecRegistry_Init()is now_PyCodecRegistry_EnsureInit(), and is called unconditionally to ensure initialization, rather than only when initialization is needed. When it returns 0 (as opposed to -1 to indicate an error), thePyObject *members ofcodecs_statecan be read without further synchronization.Operations that mutate
codecs.search_pathmust be performed while holdingcodecs.search_path_mutex. This allowsPyCodec_Unregister()to search for and delete a specific item from the list. Becausesearch_path_mutexis used as a normal mutex and not a critical section, we must be extremely careful with operations called while holding it.Issue: Audit all built-in modules for thread safety #116738