Skip to content

ENH: Inference for ExtensionArray and ExtensionDtype.type #45399

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
Xyhlon opened this issue Jan 16, 2022 · 2 comments
Open

ENH: Inference for ExtensionArray and ExtensionDtype.type #45399

Xyhlon opened this issue Jan 16, 2022 · 2 comments
Labels
Enhancement ExtensionArray Extending pandas with custom dtypes or arrays.

Comments

@Xyhlon
Copy link

Xyhlon commented Jan 16, 2022

Is your feature request related to a problem?

I wish I could use pandas to do the conversion of ExtensionDtype and ExtensionArray automatically like it does with the internal Pandas Dtypes like eg. Period Intervall with their respective Dtype.

    test_uarray = [ufloat(10, 1), ufloat(3.8, 0.2)]
    test_periods = [Period("4Q2005")] * len(test_uarray)
    df_uarray = pd.DataFrame(
        {
            "ufloats": test_uarray,
            "periods": test_periods,
        }
    )
    print(df_uarray.dtypes)

output should be

ufloats           ufloat
ints               int64
periods    period[Q-DEC]
dtype: object

Where Ufloat is the registered corresponding ExtensionDtype and having a corresponding ExtensionArray UfloatArray.
[this should provide a description of what the problem is, e.g. "I wish I could use pandas to do [...]"]

Describe the solution you'd like

For this to work infer_dtype_from_scalar() needs to be patched.

A possible patch would be adding the following here

    from pandas.core.dtypes.base import _registry

    for extdtype in _registry.dtypes:
        if isinstance(extdtype.type, property):
            try:
                exttype = extdtype.type.fget(extdtype.type)
            except AttributeError:
                continue
        else:
            exttype = extdtype.type

        if isinstance(val, exttype):
            dtype = extdtype()

With this patch inference of registered ExtensionDtype is possible

Furthermore the Inference of ExtensionArray needs to be fixed.
This can be accomplished by patching sanitize_array

The following lines can be added to sanitize_array here in order to add the functionality, however I wouldn't recommend doing it that way.

                inferred, _ = infer_dtype_from_scalar(val=subarr[0], pandas_dtype=True)
                subarr = maybe_cast_to_extension_array(
                    inferred.construct_array_type(), subarr, inferred
                )

Furthermore these two functions need to be imported
infer_dtype_from_scalar,
maybe_cast_to_extension_array,

API breaking implications

This should affect the API it only uses the already requirements to the ExtensionDtype and ExtensionArray

@Xyhlon Xyhlon added Enhancement Needs Triage Issue that has not been reviewed by a pandas team member labels Jan 16, 2022
@Xyhlon Xyhlon changed the title ENH: ENH: Inference for ExtensionArray and ExtensionDtype.type Jan 16, 2022
@mroeschke mroeschke added ExtensionArray Extending pandas with custom dtypes or arrays. and removed Needs Triage Issue that has not been reviewed by a pandas team member labels Feb 6, 2022
@mocquin
Copy link

mocquin commented Apr 1, 2022

Duplicate of #41848 ?

@mocquin
Copy link

mocquin commented Apr 1, 2022

As well as #27995 ?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Enhancement ExtensionArray Extending pandas with custom dtypes or arrays.
Projects
None yet
Development

No branches or pull requests

3 participants