Repository navigation
Isolate the Default Object Allocator between Interpreters #101659
Description
(see gh-100227)
By default, the allocator for the "object" and "mem" domains is historically known as "obmalloc". The implementation of the allocator is _PyObject_Malloc(), etc., for which the runtime state is found in _PyRuntimeState.obmalloc. Thus all interpreters currently share the runtime state for the default allocator.
Isolating that state to each interpreter is important for a per-interpreter GIL (my short-term motivation here), but there are other benefits. For example, it helps us manage resources (e.g. objects) relative to the lifecycle of each interpreter, and to do so more efficiently. Furthermore, any situation where we share memory between interpreters is an invitation for memory leaks and other bugs.
The solution here should be as simple as moving the "obmalloc" state to PyInterpreterState.
Linked PRs
- gh-101659: Isolate "obmalloc" State to Each Interpreter #101660
- gh-101659: Avoid Allocation for Shared Exceptions in the _xxsubinterpreters Module #102659
- gh-101659: initialize stack variable _sharedexception #103048
- gh-101659: Clean Up the General Import Tests for Subinterpreters #103151
- gh-101659: Use the Raw Allocator in the _xxinterpchannels Module #103287
- gh-101659: Add _Py_AtExit() #103298
Metadata
Metadata
Assignees
Labels
Projects
- Status
Activity
ericsnowcurrently commentedon Feb 9, 2023
FYI, one consequence of debugging CI failures in gh-101660 is that I was reminded that we may be able to take advantage of an object's corresponding obmalloc pool/arena to determine to which interpreter that object belongs. I'll open an issue about that once this one is settled.
gh-101659: Avoid Allocation for Shared Exceptions in the _xxsubinterp…
Merge branch 'main' into inlinecomp2
pythongh-101659: Avoid Allocation for Shared Exceptions in the _xxsub…
gh-101659: Clean Up the General Import Tests for Subinterpreters (gh-…
gh-101659: Use the Raw Allocator in the _xxinterpchannels Module (gh-…
gh-101659: Add _Py_AtExit() (gh-103298)
pythongh-101659: Use the Raw Allocator in the _xxinterpchannels Module (
pythongh-101659: Add _Py_AtExit() (pythongh-103298)
pythongh-101659: Avoid Allocation for Shared Exceptions in the _xxsub…
pythongh-101659: Clean Up the General Import Tests for Subinterpreters (
pythongh-101659: Use the Raw Allocator in the _xxinterpchannels Module (
pythongh-101659: Add _Py_AtExit() (pythongh-103298)
gh-101659: Isolate "obmalloc" State to Each Interpreter (gh-101660)