Skip to content

gh-113055: Fix memory leak of obmalloc state #113218

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
wants to merge 2 commits into from
Closed

gh-113055: Fix memory leak of obmalloc state #113218

Changes from all commits
Commits
Show all changes
2 commits
Select commit Hold shift + click to select a range
0d0c5a9
Fix memory leak of obmalloc state.
nascheme Dec 16, 2023
4471ac7
Fix for various config options.
nascheme Dec 17, 2023
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
38 changes: 37 additions & 1 deletion Objects/obmalloc.c
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -1052,6 +1052,8 @@ _PyInterpreterState_GetAllocatedBlocks(PyInterpreterState *interp)
return n;
}

static void free_arenas(PyInterpreterState *interp);

void
_PyInterpreterState_FinalizeAllocatedBlocks(PyInterpreterState *interp)
{
Expand All @@ -1065,6 +1067,7 @@ _PyInterpreterState_FinalizeAllocatedBlocks(PyInterpreterState *interp)
assert(has_own_state(interp) || leaked == 0);
interp->runtime->obmalloc.interpreter_leaks += leaked;
}
free_arenas(interp);
}

static Py_ssize_t get_num_global_allocated_blocks(_PyRuntimeState *);
Expand Down Expand Up @@ -2612,7 +2615,6 @@ _PyObject_DebugDumpAddress(const void *p)
_PyMem_DumpTraceback(fileno(stderr), p);
}


static size_t
printone(FILE *out, const char* msg, size_t value)
{
Expand Down Expand Up @@ -2666,6 +2668,40 @@ _PyDebugAllocatorStats(FILE *out,

#ifdef WITH_PYMALLOC

static void
free_arenas(PyInterpreterState *interp)
{
OMState *state = &interp->obmalloc;
for (uint i = 0; i < maxarenas; ++i) {
// free each obmalloc memory arena
struct arena_object *ao = &allarenas[i];
_PyObject_Arena.free(_PyObject_Arena.ctx,
(void *)ao->address, ARENA_SIZE);
}
// free the array containing pointers to all arenas
PyMem_RawFree(allarenas);
Comment on lines +2675 to +2682
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I've double-checked that this is correct. I didn't see any other allocations for arenas (outside the radix tree stuff).

#if WITH_PYMALLOC_RADIX_TREE
#ifdef USE_INTERIOR_NODES
// Free the middle and bottom nodes of the radix tree. These are allocated
// by arena_map_mark_used() but not freed when arenas are freed.
Comment on lines +2685 to +2686
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why aren't they freed when arenas are freed?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Better performance and simpler code. It might not be too expensive to free them. We would have to track how many nodes are children and then remove when count reaches zero.

for (int i1 = 0; i1 < MAP_TOP_LENGTH; i1++) {
arena_map_mid_t *mid = arena_map_root.ptrs[i1];
if (mid == NULL) {
continue;
}
for (int i2 = 0; i2 < MAP_MID_LENGTH; i2++) {
arena_map_bot_t *bot = arena_map_root.ptrs[i1]->ptrs[i2];
if (bot == NULL) {
continue;
}
PyMem_RawFree(bot);
}
PyMem_RawFree(mid);
}
#endif
#endif
Comment on lines +2683 to +2702
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm not as familiar with the radix tree stuff. I do see the corresponding allocations in arena_map_get() (via arena_map_mark_used()) though. The logic you have here make sense relative to that.

}

#ifdef Py_DEBUG
/* Is target in the list? The list is traversed via the nextpool pointers.
* The list may be NULL-terminated, or circular. Return 1 if target is in
Expand Down