Compilation error with Categorical dtype

[niklasmueboe]

Using pyo3-polars 0.17 with the dtype-categorical feature for polars seems to lead to compilation errors. Using pyo3-polars 0.15 works fine.

error[E0004]: non-exhaustive patterns: &DataType::Categorical(_, _) and &DataType::Enum(_, _) not covered
--> /home/user/.cargo/registry/src/index.crates.io-6f17d22bba15001f/pyo3-polars-0.17.0/src/types.rs:359:15
|
359 | match &self.0 {
| ^^^^^^^ patterns &DataType::Categorical(_, _) and &DataType::Enum(_, _) not covered
|
note: DataType defined here
--> /home/user/.cargo/registry/src/index.crates.io-6f17d22bba15001f/polars-core-0.43.1/src/datatypes/dtype.rs:44:1
|
44 | pub enum DataType {
| ^^^^^^^^^^^^^^^^^
...
88 | Categorical(Option<Arc>, CategoricalOrdering),
| ----------- not covered
89 | #[cfg(feature = "dtype-categorical")]
90 | Enum(Option<Arc>, CategoricalOrdering),
| ---- not covered
= note: the matched value is of type &DataType
help: ensure that all possible cases are being handled by adding a match arm with a wildcard pattern, a match arm with multiple or-patterns as shown, or multiple match arms
|
494 ~ },
495 + &DataType::Categorical(_, ) | &DataType::Enum(, _) => todo!()

[Shoeboxam]

Have you tried adding the "dtype-categorical" feature? Things should ideally compile as-is, but it's possible to nudge it along.

[niklasmueboe]

Then I get

 error[E0432]: unresolved import `polars_core::datatypes::create_enum_data_type`
   --> /home/user/.cargo/registry/src/index.crates.io-6f17d22bba15001f/pyo3-polars-0.17.0/src/types.rs:6:5
    |
  6 | use polars_core::datatypes::create_enum_data_type;
    |     ^^^^^^^^^^^^^^^^^^^^^^^^---------------------
    |     |                       |
    |     |                       help: a similar name exists in the module: `create_enum_dtype`
    |     no `create_enum_data_type` in `datatypes`

[Shoeboxam]

Try pinning to Polars 0.43.0, and enabling the "futures" feature in polars?

• I think cargo is choosing 0.43.1 because it's a caret dep, but 0.43.1 had a data_type -> dtype refactor. Add polars to your cargo.toml as a pinned dep "=0.43.0"
• But 0.43.0 had an unguarded import from a "futures" feature, so "futures" has to be enabled in Polars 0.43.0 for Polars 0.43.0 to compile

They seem to have forgotten to update the pyo3 polars crate to support polars 0.43.1.

[Distortedlogic]

0.43.0 complains no feature futures for polars for me. I had to downgrade to 0.42.0 for poalrs and 0.16.1 for pyo3-polars :/ note this error occured for me from within maturin, not rust analyzer

[oscar6echo]

I had a compilation error with Struct type (feature dtype-struct), trying to reproduce tutorial https://marcogorelli.github.io/polars-plugins-tutorial/struct/.

Because it looks very similar to the problem @Distortedlogic met I followed the same steps, making the same observations.
But even downgrading to pyo3-polars=0.16.1 and polars=0.42.0 was not enough to fix the issue.

I still see the following error upon maturing develop but rust-analyzer does not see any problem:

Does anybody know a version combination that works with feature dtype-struct ?

[oscar6echo]

Here is a solution to my problem from @cmdlineluser in discord:

[dependencies]
pyo3 = { version = "0.22", features = ["extension-module", "abi3-py38"] }
pyo3-polars = { version = "0.17", features = ["derive", "dtype-struct"] }
polars = { version = "0.43", default-features = false, features = ["dtype-struct"] }
serde = { version = "1", features = ["derive"] }

The important point is to add feature=dtype-struct in pyo3-polars too.
Also this combination of versions works as of 4nov24 - the more recent ones do not.

[valkmit]

Here is a solution to my problem from @cmdlineluser in discord:

[dependencies]
pyo3 = { version = "0.22", features = ["extension-module", "abi3-py38"] }
pyo3-polars = { version = "0.17", features = ["derive", "dtype-struct"] }
polars = { version = "0.43", default-features = false, features = ["dtype-struct"] }
serde = { version = "1", features = ["derive"] }

The important point is to add feature=dtype-struct in pyo3-polars too. Also this combination of versions works as of 4nov24 - the more recent ones do not.

Thanks @oscar6echo , saved me a bit of digging!