Support `Float16` data type

[wlongxiang]

Problem description

Hi there,

Is there any plan to add support for float16, I recently run into a situation where I have to optimize the memory footprint of my dataframe to some extreme due to the memory limit of our k8s clusters. The biggest area for improvement in my case would be to downcast many of the float32 columns to float16.

I can imagine this can be useful for many such resource constrained scenarios too.

Thanks.

[alexander-beedie]

Float16 is a very limited type, and for many (most?) use-cases it's quite inaccurate; do you know that your data would actually work with it? (Genuinely curious here, as there aren't that many times where you'd really want to do this ;)

[kocas]

I would also be happy to see f16 support.

We use f16 for storage and transfer of large feature data. Conversion is trivial. For majority of the tasks, memory saving is a massive plus. Loss of precision (especially during arithmetics) is something to think about, but I guess that goes for f32 as well if you push enough :)

[ghuls]

There is also no native (IEEE half precision) Float16 calculation support for CPUs, so conversions to float32 have to be done.
https://scicomp.stackexchange.com/questions/35187/is-half-precision-supported-by-modern-architecture

If you don't need most of the float16 column for calculations (with Polars), you can encode those float16 columns as uint16 or int16 and after filtering, groupby, ... operations on other columns of the dataframe, convert them back to float16:

df = pl.DataFrame(
    {
        "a": ["some", "text", "!"],
        "b": np.array([5.0, 7.0, 3.0], dtype=np.float16).view(np.uint16)
    }
)


df.filter(pl.col("a").str.contains("e"))

Out[218]:
shape: (2, 2)
┌──────┬───────┐
│ a    ┆ b     │
│ ---  ┆ ---   │
│ str  ┆ u16   │
╞══════╪═══════╡
│ some ┆ 17664 │
│ text ┆ 18176 │
└──────┴───────┘



df.filter(
    # Filter some columns, grouby, ...
    pl.col("a").str.contains("e")
)
.with_columns(
    # Convert all encoded Float16 (UInt16) columns to Float32 Polars Series.
    pl.col(pl.UInt16).map(
        lambda x: pl.Series(x.to_numpy().view(np.float16).astype(np.float32))
    )
)

Out[219]:
shape: (2, 2)
┌──────┬─────┐
│ a    ┆ b   │
│ ---  ┆ --- │
│ str  ┆ f32 │
╞══════╪═════╡
│ some ┆ 5.0 │
│ text ┆ 7.0 │
└──────┴─────┘

[slonik-az]

you can encode those float16 columns as uint16 or int16 and after filtering, groupby, ... operations on other columns of the dataframe, convert them back to float16

Does this f16 -> u16 conversion preserve ordering? In other words can one filter u16 using < or > and get the same results as filtering f16 numbers?

[mcrumiller]

Does this f16 -> u16 conversion preserve ordering

They should for all positive f16 but with negative I'm not so sure, based on the way negative integers are stored via 2's complement.

To expand on that, the first bit (sign bit) is 1 if the floating point value is negative, and 0 if not. For unsigned integers, this bit would be the most significant, so this would definitely disrupt ordering in that if you have a = -1.0 and b = 1.0, when viewed as floats you have a < b but when viewed as unsigned ints you'll have a < b.

[ghuls]

If you want to keep the ordering, you need some additional manipulation as is done when ordering Float32 values in Rust by their integer representation:

https://doc.rust-lang.org/src/core/num/f32.rs.html#1338

In [86]: x = np.array([-200.0, -180.00008, -8.0, -5.0, -0.000002, 0.0, 0.0000002, 3.0, 5.0, 7.0, 200.0], dtype=np.float16).view(np.int16)

In [87]: x
Out[87]: 
array([ -9664,  -9824, -14336, -15104, -32734,      0,      3,  16896,
        17664,  18176,  23104], dtype=int16)

In [88]: x ^ (((x >> 15).view(np.uint16) >> 1)).view(np.int16)
Out[88]: 
array([-23105, -22945, -18433, -17665,    -35,      0,      3,  16896,
        17664,  18176,  23104], dtype=int16)

[ghuls]

Here are some encoding and decoding functions that will keep the ordering of float16 when converted to int16:

In [106]: def encode_float16_array_as_ordered_int16_series(array_f16):
     ...:     x = array_f16.view(np.int16)
     ...:     total_order_float16_as_int16 = x ^ (((x >> 15).view(np.uint16) >> 1)).view(np.int16)
     ...:     return pl.Series("", total_order_float16_as_int16)
     ...: 

In [107]: def decode_ordered_int16_series_to_float16_array(series_int16):
     ...:     x = series_int16.to_numpy().view(np.int16)
     ...:     array_float16 = (x ^ (((x >> 15).view(np.uint16) >> 1)).view(np.int16)).view(np.float16)
     ...:     return array_float16
     ...: 

In [108]: x = np.array([-200.0, -180.00008, -8.0, -5.0, -0.000002, 0.0, 0.0000002, 3.0, 5.0, 7.0, 200.0], dtype=np.float16)

In [109]: x
Out[109]: 
array([-2.0e+02, -1.8e+02, -8.0e+00, -5.0e+00, -2.0e-06,  0.0e+00,
        1.8e-07,  3.0e+00,  5.0e+00,  7.0e+00,  2.0e+02], dtype=float16)

In [110]: s = encode_float16_array_as_ordered_int16_series(x)

In [111]: s
Out[111]: 
shape: (11,)
Series: '' [i16]
[
        -23105
        -22945
        -18433
        -17665
        -35
        0
        3
        16896
        17664
        18176
        23104
]

In [112]: decode_ordered_int16_series_to_float16_array(s)
Out[112]: 
array([-2.0e+02, -1.8e+02, -8.0e+00, -5.0e+00, -2.0e-06,  0.0e+00,
        1.8e-07,  3.0e+00,  5.0e+00,  7.0e+00,  2.0e+02], dtype=float16)

[wlongxiang]

Float16 is a very limited type, and for many (most?) use-cases it's quite inaccurate; do you know that your data would actually work with it? (Genuinely curious here, as there aren't that many times where you'd really want to do this ;)

We have a slew of 0 to 1 score columns generated by ML for ranking purposes. In our case, the numerical precision does not matter too much, as long as it doesn't change the ranking, but the reduction in memory footprint is significant.

[slonik-az]

We have a slew of 0 to 1 score columns generated by ML for ranking purposes. In our case, the numerical precision does not matter too much ...

Can use scale the score to [0, 255] range and cast to u8? You will save twice the memory compared to f16.

[itamarst]

Context: I have some paid time to work on Polars. My client would like float16 support.

As a general principle, it seems like "pass through data unchanged from inputs" is worth doing, especially given Polars now supports extension plugins.

In this specific case, I know @ritchie46 has been skeptical in the past, so I would like to suggest as a starting point a minimal implementation that would have low maintenance overhead, with the presumption that I do the work:

1. When loading data formats that are float16, they continue to be stored as float16.
2. When writing to data formats that support float16, it gets written as float16.
3. There is support for explicit casting between float16 to float32/float64 (and perhaps ints and strings).
4. Implicit conversion only happens when outputting to formats that don't support float16 (e.g. JSON), which I assume is pretty straightforward given that's already happening for e.g. float32.

That's it. So e.g. df.select((pl.col("thisisfloat16").cast(pl.Float32) * 2).cast(pl.Float16)) would work.

Benefits:

• Data can just be passed through unchanged and untouched, a likely use case in some cases, and plausibly something Polars should be able to do. I believe one format currently converts to float32 at load?
• Coupled with streaming, one can preserve the memory footprint benefits while still doing calculations.
• Limited scope of implementation footprint, since doing any calculations require casting.

[stinodego]

As a general principle, it seems like "pass through data unchanged from inputs" is worth doing, especially given Polars now supports extension plugins.

I don't think so. I don't know the ins and outs of half floats, but from a usability perspective, I am not happy with a data type that doesn't support the full Polars API.

We will get a host of issues of "why does sum work on Float32 but not Float16?". And if we were to implement it internally through a conversion to Float32, we will get a host of issues like "why is Float16 not as efficient as Float32"?

We can include a bunch of caveats in our documentation, but I'm not happy with caveats. If we don't want to fully support Float16, we should not have it as a data type.

I can see the potential benefits of allowing a Float16 data type as a 'passenger'. But I think the downsides are too significant. Just my 2 cents though.

[itamarst]

A comparison to another case may be useful in thinking this through.

Arrow supports extension types, which e.g. how the geoarrow work is being done: https://github.com/geoarrow/geoarrow/blob/main/extension-types.md. There are four things you'd want to do with extension typed data:

• Load it from a file. You might need do this with a third-party library, to some extent, but for something like Feather that is supposed to be Arrow-on-the-disk this would be annoying and/or wasteful, since you'd have to load the same file twice.
• Save it to a file. Similar to loading.
• Chop columns up due to filtering/grouping/etc based on the contents of other columns. The data is treated as opaque, but Polars must support this or it's impossible to do third-party GeoArrow support for Polars.
• Transform the data. Presumably done by via the new plugin API, so you'd do pl.col("geocol").geo.change_coordinate_system(new_coord) or something.

The above suggests that at minimum Polars must support having columns whose data types it can't handle internally, otherwise it would break an important third-party use case. Loading and saving these data types from e.g. Feather is probably worth doing too, and perhaps some sort of registry mechanism for mapping between Parquet and Arrow extensions for things like GeoParquet <-> GeoArrow.

Float16 is a slightly special case in that it's not a custom extension type, it's more built-in to Arrow, but similar concerns apply. And I can see plenty of good reasons for Polars to say it doesn't want to have any Float16 processing, or even casting code, as @stinodego suggests.

And that's fine, since casting could be done via a third-party extension namespace... but only if Polars is willing have column types it doesn't know about, and ideally to load and save the data. Otherwise you're vetoing everyone else from supporting the use case.