jit: avoid wraparound in stack size

pcre2_jit_stack_create() allows the user to indicate how big of a
stack size, JIT should be able to allocate and use, using a size_t
variable that should be able to hold bigger values than reasonable.

Internally, the value is rounded to the next 8K, but if the value
is unreasonable large, would overflow.

Avoid the overflowing by checking and failing the value early, and
while at it make the check clearer and document the failure mode.

Signed-off-by: Carlo Marcelo Arenas Belón <[email protected]>

Author: carenas

doc/pcre2_jit_stack_create.3

@@ -22,7 +22,8 @@ allocation. The result can be passed to the JIT run-time code by calling
 \fBpcre2_jit_stack_assign()\fP to associate the stack with a compiled pattern,
 which can then be processed by \fBpcre2_match()\fP or \fBpcre2_jit_match()\fP.
 A maximum stack size of 512KiB to 1MiB should be more than enough for any
-pattern. For more details, see the
+pattern. If the stack couldn't be allocated or the values passed were not
+reasonable, NULL will be returned. For more details, see the
 .\" HREF
 \fBpcre2jit\fP
 .\"

src/pcre2_jit_misc.c

@@ -135,7 +135,7 @@ return NULL;
 
 pcre2_jit_stack *jit_stack;
 
-if (startsize < 1 || maxsize < 1)
+if (startsize == 0 || maxsize == 0 || maxsize > PCRE2_SIZE_MAX - STACK_GROWTH_RATE)
   return NULL;
 if (startsize > maxsize)
   startsize = maxsize;