Overflow in `Bigarray.reshape`

[last-genius]

Bigarray.reshape does not seem to handle overflow well, allowing to read into uninitialized memory.

Below example shows how data pointer isn't reallocated, caml_ba_num_elts still returns 0, but .get() into uninitialized memory is allowed:

Array1.create 0
arr->data: 0xebb0480
elements: 0
arr.get(5) exception: Invalid_argument("index out of bounds")

Bigarray.reshape [|8; 2305843009213693952|]
Size in bytes: 0 (dim1: 8, dim2: 2305843009213693952)

arr->data: 0xebb0480
elements: 0
arr[5][5]: -2134031706542294431 <--- this value is random on every program invocation

It seems like caml_umul_overflow in caml_ba_alloc should catch this, raising Out_of_memory, but this doesn't seem to be the case.

---

Source code of the example above.

bin/dune:

(executable
 (foreign_stubs
  (language c)
  (names reshape_stubs)
 )
 (public_name reshape)
 (name main)
 (libraries reshape))

bin/main.ml:

external elements : ('a, 'b, 'c) Bigarray.Genarray.t -> int = "stub_arr_size"

let _ =
  let open Bigarray in
  print_endline "Array1.create 0" ;
  let arr = Array1.create int c_layout 0 in
  Printf.printf "elements: %d\n" (elements (genarray_of_array1 arr)) ;

  let _ =
    try Array1.get arr 5
    with e ->
      Printf.printf "arr.get(5) exception: %s\n" (Printexc.to_string e) ;
      1
  in

  print_endline "\nBigarray.reshape [|8; 2305843009213693952|]" ;
  let x =
    array2_of_genarray
      (reshape (genarray_of_array1 arr) [|8; 2305843009213693952|])
  in

  print_endline
    (Printf.sprintf "Size in bytes: %d (dim1: %d, dim2: %d)\n"
       (Array2.size_in_bytes x) (Array2.dim1 x) (Array2.dim2 x)
    ) ;

  Printf.printf "elements: %d\n" (elements (genarray_of_array2 x)) ;
  Printf.printf "arr[5][5]: %d\n" (Array2.get x 5 5)

bin/reshape_stubs.c:

#include <caml/bigarray.h>
#include <caml/mlvalues.h>
#include <caml/memory.h>


CAMLprim value
stub_arr_size (value arr)
{
    CAMLparam1(arr);
    printf("arr->data: %p\n", Caml_ba_array_val(arr)->data);
    CAMLreturn(Val_int(caml_ba_num_elts(Caml_ba_array_val(arr))));
}

[last-genius]

The above is the case on a 4.14.2 switch, but on 5.3.0 the program throws Out_of_memory indeed:

Array1.create 0
arr->data: 0x34357ea0
elements: 0
arr.get(5) exception: Invalid_argument("index out of bounds")

Bigarray.reshape [|8; 2305843009213693952|]
Fatal error: exception Out of memory

Still, the error should be Bigarray.reshape: size mismatch instead here

[nojb]

I believe this was fixed as a side effect of #11022, which is included in 5.0 and later. cc @stedolan

[edwintorok]

It seems like caml_umul_overflow in caml_ba_alloc should catch this, raising Out_of_memory, but this doesn't seem to be the case.

I think this is because data != NULL when called from caml_ba_reshape, so the if condition that would call caml_umul_overflow is never run. Instead the multiplication is done in a different way by caml_ba_reshape itself, without any overflow checking.

Although the final number of elements has to match, that could still be the case if there was an overflow. But on an overflow individual dimensions may exceed the size of the array.

The linked patch changes the order, and always executes the umul_overflow checks.

[hannesm]

From my point of view: yes, this should be fixed in 4.14. Yes, it deserves a security advisory when fixed. On OCaml 5, is there something that needs to be done (Out of memory vs Bigarray.reshape: size mismatch)?

[hannesm]

We have a CVE number assigned for this: CVE-2026-34353