Skip to content

BUG: Fix is_categorical_dtype for Sparse[category] #35797

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
wants to merge 12 commits into from
2 changes: 1 addition & 1 deletion doc/source/whatsnew/v1.2.0.rst
Original file line number Diff line number Diff line change
Expand Up @@ -236,7 +236,7 @@ Bug fixes
Categorical
^^^^^^^^^^^

-
- Bug in :func:`pandas.api.types.is_categorical_dtype` where sparse categorical dtypes would return ``False`` (:issue:`35793`)
-

Datetimelike
Expand Down
4 changes: 4 additions & 0 deletions pandas/core/dtypes/base.py
Original file line number Diff line number Diff line change
Expand Up @@ -287,6 +287,10 @@ def is_dtype(cls, dtype: object) -> bool:
return False
elif isinstance(dtype, cls):
return True
elif hasattr(dtype, "subtype") and isinstance(
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

do we really need to add this, this is a perf sensitive piece.

Copy link
Member Author

@dsaxton dsaxton Sep 16, 2020

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It's needed for test_is_categorical_sparse_categorical to pass; I'm not sure if there's another way to make this work. I ran the dtype asvs and didn't see a degradation.

dtype.subtype, cls # type: ignore[attr-defined]
):
return True
if isinstance(dtype, str):
try:
return cls.construct_from_string(dtype) is not None
Expand Down
6 changes: 4 additions & 2 deletions pandas/core/dtypes/common.py
Original file line number Diff line number Diff line change
Expand Up @@ -558,8 +558,10 @@ def is_categorical_dtype(arr_or_dtype) -> bool:
"""
if isinstance(arr_or_dtype, ExtensionDtype):
# GH#33400 fastpath for dtype object
return arr_or_dtype.name == "category"

return ("category" == arr_or_dtype.name) or (
is_sparse(arr_or_dtype)
and arr_or_dtype.subtype.name == "category" # type: ignore[attr-defined]
)
if arr_or_dtype is None:
return False
return CategoricalDtype.is_dtype(arr_or_dtype)
Expand Down
9 changes: 9 additions & 0 deletions pandas/tests/dtypes/test_common.py
Original file line number Diff line number Diff line change
Expand Up @@ -213,6 +213,15 @@ def test_is_categorical_deprecation():
com.is_categorical([1, 2, 3])


def test_is_categorical_sparse_categorical():
# https://github.com/pandas-dev/pandas/issues/35793
s = pd.Series(
["a", "b", "c"], dtype=pd.SparseDtype(CategoricalDtype(["a", "b", "c"]))
)
assert com.is_categorical_dtype(s)
assert com.is_categorical_dtype(s.dtype)


def test_is_datetime64_dtype():
assert not com.is_datetime64_dtype(object)
assert not com.is_datetime64_dtype([1, 2, 3])
Expand Down