We are more or less limited to testing around 2^36 inputs in a reasonable amount of time, which can exhaustively cover unary f32 ops and unary/binary f16 ops, but not much else. I have been thinking with how to better improve what we cover and came up with the following:
- Divide 36 by the number of inputs (36 unary, 18 binary, 12 ternary)
- Divide that number by 4 (9/4/3). Allocate these four bits as follows:
- One group covers for the sign bit and upper exponent bits
- One group covers the lower exponent bits
- One group covers the upper exponent bits
- One group covers the lower exponent bits
- If the exponent is smaller than 2 times the above number, allocate additional bits to the lower significand bits.
- Exhaustively sweep the assigned bits in these ranges, setting remaining bits to zero.
- Repeat but set the remaining bits to one.
- Repeat but use random values for the remaining bits.
That will cover both negative and positive for zero, max normal, min normal, max subnormal, min subnormal, infinity, qNaN, sNaN, and a number of cases around their transition points, which seems pretty nicely comprehensive.
I think I'd design this as an extensive_edges generator that takes a callback for how to fill the middle portion. This could replace the spaced generator we currently use for extensive tests.
We are more or less limited to testing around 2^36 inputs in a reasonable amount of time, which can exhaustively cover unary f32 ops and unary/binary f16 ops, but not much else. I have been thinking with how to better improve what we cover and came up with the following:
That will cover both negative and positive for zero, max normal, min normal, max subnormal, min subnormal, infinity, qNaN, sNaN, and a number of cases around their transition points, which seems pretty nicely comprehensive.
I think I'd design this as an
extensive_edgesgenerator that takes a callback for how to fill the middle portion. This could replace thespacedgenerator we currently use for extensive tests.