Skip to content

Commit dba1487

Browse files
committed
[opt] Fix oss-fuzz bug in optimal parser
oss-fuzz uncovered a scenario where we're evaluating the cost of litLength = 131072, which can't be represented in the zstd format, so we accessed 1 beyond LL_bits. Fix the issue by making it cost 1 bit more than litLength = 131071. There are still follow ups: 1. This happened because literals_cost[0] = 0, so the optimal parser chose 36 literals over a match. Should we bound literals_cost[literal] > 0, unless the block truly only has one literal value? 2. When no matches are found, the cost model isn't updated. In this case no matches were found for an entire block. So the literals cost model wasn't updated at all. That made the optimal parser think literals_cost[0] = 0, where it is actually quite high, since the block was entirely random noise. Credit to OSS-Fuzz.
1 parent 26d88c0 commit dba1487

File tree

1 file changed

+5
-1
lines changed

1 file changed

+5
-1
lines changed

lib/compress/zstd_opt.c

Lines changed: 5 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -269,7 +269,11 @@ static U32 ZSTD_rawLiteralsCost(const BYTE* const literals, U32 const litLength,
269269
* cost of literalLength symbol */
270270
static U32 ZSTD_litLengthPrice(U32 const litLength, const optState_t* const optPtr, int optLevel)
271271
{
272-
if (optPtr->priceType == zop_predef) return WEIGHT(litLength, optLevel);
272+
assert(litLength <= ZSTD_BLOCKSIZE_MAX);
273+
if (optPtr->priceType == zop_predef)
274+
return WEIGHT(litLength, optLevel);
275+
if (litLength == ZSTD_BLOCKSIZE_MAX)
276+
return BITCOST_MULTIPLIER + ZSTD_litLengthPrice(ZSTD_BLOCKSIZE_MAX - 1, optPtr, optLevel);
273277

274278
/* dynamic statistics */
275279
{ U32 const llCode = ZSTD_LLcode(litLength);

0 commit comments

Comments
 (0)