-
Notifications
You must be signed in to change notification settings - Fork 168
Open
Labels
enhancementNew feature or requestNew feature or request
Milestone
Description
At the moment, we allocate memory as we need it, making a single malloc
call for every node in the tree, as well as whichever fields on those nodes need memory. When we're done with the tree, we recursively visit the tree and deallocate all at once. This is a really good candidate for arena allocation. But more than that, it would be good enough just to centralize our memory management functions, which we need to do anyway. Below are a couple of tasks that are related to memory that would improve our general memory story.
- Centralize our memory management functions. Currently we call the stdlib
malloc
/calloc
/realloc
/free
functions whenever we need memory. (We purposefully avoidalloca
because it's not C standard.) Those functions calls are scattered throughout the codebase. For all subsequent bullets, we need them to go through centralized functions. These functions should all accept ayp_parser_t *
, but it can be marked as unused for now (YP_ATTRIBUTE_UNUSED
). For this first task, they should all simply call the stdlib function. For example, ayp_malloc
function would callmalloc
with the same arguments. - Provide the ability to pass memory function pointers to a
yp_parser_t
such that theyp_
version of the memory functions call the function pointers if they are provided. We need this for CRuby, which providesxmalloc
which will attempt a GC if a call tomalloc
fails. This should follow the same pattern we have for theencoding_changed
callback, which is that it provides a function that accepts a function pointer. - Request a certain number of pages at a time, estimated by the size of the input string. This would be our own arena allocation.
yp_malloc
would then change to callmalloc
/xmalloc
only when the current arena has run out of allocatable memory. We can be very naive with this and make it simply a bump allocator since we so rarely deallocate nodes while parsing.
The first two of these tasks are necessary for our integration into CRuby. The last one is a nice-to-have enhancement, and is not necessary for v1.0.
Metadata
Metadata
Assignees
Labels
enhancementNew feature or requestNew feature or request