-
-
Notifications
You must be signed in to change notification settings - Fork 31.9k
[subinterpreters] Improve Interpreter Isolation for Uses of PyArg_Parser #95909
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Comments
We don't really need to make kwtuple per interpreter as it is both not needed and will make kwnames parsing slower as #95910 need a dict lookup to find the kwtuple. Here's my plan: To avoid a performance regression we want to avoid locking for the general case when the parser is already initialized and no dict lookups. Acquiring and releasing a lock is very slow compared to an atomic read as the latter it is implemented in the hardware.
With this we can avoid locking and releasing the global mutex for the general case and retain performance. |
Thanks for the PR, @kumaraditya303. |
(This is a follow-up to gh-90928.)
Argument clinic adds a static local
PyArg_Parser
for each function it generates. Each of those variables is initialized at runtime the first time it is used (even though, technically, it could be done statically). That means there's a (very unlikely) race on initializing each of those variables, at least once interpreters stop sharing the GIL. Furthermore, thePyArg_Parser.kwtuple
field holds aPyObject *
, which means we're leaking objects between interpreters (albeit completely immutable ones), and there still can be a race on refcounts if we don't do something about it. (For core cases, incl. builtin modules, we statically initialize kwtuple, but not for extension modules.)For the sake of isolation (and of a per-interpreter GIL) we should store
PyArg_Parser.kwtuple
onPyInterpreterState
in the cases where we aren't statically initializing it. For per-interpreter GIL we should also add a global lock around initializing each staticPyArg_Parser
.The text was updated successfully, but these errors were encountered: