Skip to content

slice_count overflow for very large allocations #876

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Closed
colesbury opened this issue Apr 12, 2024 · 2 comments
Closed

slice_count overflow for very large allocations #876

colesbury opened this issue Apr 12, 2024 · 2 comments

Comments

@colesbury
Copy link
Contributor

This was seen in python/cpython#117755 on s390x (Linux on IBMz) for an allocation of 0x7fffffffffffffff bytes:

mimalloc/src/segment.c

Lines 617 to 618 in f199b88

slice->slice_count = (uint32_t)slice_count;
mi_assert_internal(slice->slice_count == slice_count); // no overflow?

The mi_segment_os_alloc is successful (possibly due to overcommit), but then the slice_count is truncated, which leads to either an assertion error (in debug builds) or floating point error due to a divide by zero.
@colesbury colesbury mentioned this issue Apr 12, 2024
Merged
vstinner added a commit to vstinner/mimalloc that referenced this issue Apr 15, 2024
@vstinner
Issue microsoft#876: Fix integer overflow on slice_count
8b8057e 
If the slice count doesn't fit into uint32_t, consider that the
memory allocation failed.

On s390x, allocating around 8,589,934,592 GiB with mmap() works
thanks to overcommit on a machine with 8 GiB of memory:

    mmap(NULL, 0x8000000000400000,
         PROT_READ|PROT_WRITE,
         MAP_PRIVATE|MAP_ANONYMOUS|MAP_NORESERVE,
         -1, 0)
vstinner added a commit to vstinner/mimalloc that referenced this issue Apr 15, 2024
@vstinner
Issue microsoft#876: Fix integer overflow on slice_count
84ceba4 
If the slice count doesn't fit into uint32_t, consider that the
memory allocation failed.

On Linux s390x, allocating around 8,589,934,592 GiB with mmap() works
thanks to overcommit on a machine with 8 GiB of memory:

    mmap(NULL,
         0x8000000000400000,
         PROT_READ | PROT_WRITE,
         MAP_PRIVATE | MAP_ANONYMOUS | MAP_NORESERVE,
         -1, 0)
@vstinner
Copy link

I proposed a fix: #877

@daanx
Copy link
Collaborator

daanx commented Apr 19, 2024

Ah, that is a terrible bug but I just merged a fix. I am going to do a fresh release tag soon.
Thanks so much!

@daanx daanx closed this as completed Apr 19, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
3 participants
@vstinner @colesbury @daanx