You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Previously, I proposed to directly expose PyInterpreterState_IsMain() to the limited C API, but the idea got rejected.
The syslog module behaves differently if it's used in the main interpreter:
staticinlineintis_main_interpreter(void)
{
return (PyInterpreterState_Get() ==PyInterpreterState_Main());
}
...
if (!is_main_interpreter()) {
PyErr_SetString(PyExc_RuntimeError, "subinterpreter can't use syslog.openlog()");
returnNULL;
}
The syslog module stores a Python object in static PyObject *S_ident_o = NULL;. It's a way to keep a string passed to C openlog() function alive until closelog() is called. The problem is to decide which extension instance and/or which Python interpreter is responsible for that memory: the memory must be kept alive until closelog() is called. But what if each Python interpreter call openlog()? Calling openlog() more than once overrides the previous call. But if closelog() is called, if multiple interpreters called openlog(), you might think that syslog is still usable, whereas closelog() has been called.
See issue gh-99127 and PR gh-99128 which added is_main_interpreter() function to syslog.
Cython already implements a similar, but different check: it raises "this module can only be loaded into one interpreter per process" error message.
Cython uses a different logic to handle PEP 489 "Multi-phase extension module initialization". It uses a static PY_INT64_T main_interpreter_id = -1; variable which is initialized at the first call of __Pyx_check_single_interpreter().
//#if CYTHON_PEP489_MULTI_PHASE_INITstaticCYTHON_SMALL_CODEint__Pyx_check_single_interpreter(void) {
#ifPY_VERSION_HEX >= 0x030700A1staticPY_INT64_Tmain_interpreter_id=-1;
PY_INT64_Tcurrent_id=PyInterpreterState_GetID(PyThreadState_Get()->interp);
if (main_interpreter_id==-1) {
main_interpreter_id=current_id;
return (unlikely(current_id==-1)) ? -1 : 0;
} elseif (unlikely(main_interpreter_id!=current_id))
#elsestaticPyInterpreterState*main_interpreter=NULL;
PyInterpreterState*current_interpreter=PyThreadState_Get()->interp;
if (!main_interpreter) {
main_interpreter=current_interpreter;
} elseif (unlikely(main_interpreter!=current_interpreter))
#endif
{
PyErr_SetString(
PyExc_ImportError,
"Interpreter change detected - this module can only be loaded into one interpreter per process.");
return-1;
}
return0;
}
The text was updated successfully, but these errors were encountered:
Calling some C functions have a process-wide effect (ex: not limited to the current thread), and some also expect that the caller keeps the memory alive.
In the past, os.putenv() was implemented with C putenv() which expects the caller to keep the memory alive. Python had to store the string in an internal list (posix_putenv_garbage) until Python exit. Python started to crash when the os module was converted to multi-phase init API (PEP 489).
Or maybe the restriction should be that an extension cannot be loaded in two different interpreters. Honestly, I'm not sure.
IMO, this depends on the extension.
The recipe is relatively easy to extend in this way -- reset the flag when freeing the module. (That's the power of recipes -- they spell out what's happening, and you can adjust them.)
The downside of this recipe is that it will need locking under nogil. (But it seems to me that it's not the only thing to need that, and so nogil should add a general mechanism for process-global locking.)
Somewhere in
PyModuleDef
, we should add a way to disallow globally the creation of more than one extension instance.Or maybe the restriction should be that an extension cannot be loaded in two different interpreters. Honestly, I'm not sure.
@ericsnowcurrently no longer wants to treat the main interpreter differently: see issue gh-109857.
Previously, I proposed to directly expose PyInterpreterState_IsMain() to the limited C API, but the idea got rejected.
The syslog module behaves differently if it's used in the main interpreter:
The syslog module stores a Python object in
static PyObject *S_ident_o = NULL;
. It's a way to keep a string passed to C openlog() function alive until closelog() is called. The problem is to decide which extension instance and/or which Python interpreter is responsible for that memory: the memory must be kept alive until closelog() is called. But what if each Python interpreter call openlog()? Calling openlog() more than once overrides the previous call. But if closelog() is called, if multiple interpreters called openlog(), you might think that syslog is still usable, whereas closelog() has been called.See issue gh-99127 and PR gh-99128 which added
is_main_interpreter()
function to syslog.Cython already implements a similar, but different check: it raises "this module can only be loaded into one interpreter per process" error message.
Cython uses a different logic to handle PEP 489 "Multi-phase extension module initialization". It uses a
static PY_INT64_T main_interpreter_id = -1;
variable which is initialized at the first call of__Pyx_check_single_interpreter()
.The text was updated successfully, but these errors were encountered: