[Python] Casting round double scalars to temporal types raising ArrowNotImplementedError on pyarrow 13 dev

[mroeschke]

Describe the bug, including details regarding any error messages, version, and platform.

>>> import pyarrow as pa
>>> pa.__version__
'12.0.1'
>>> pa.scalar(43260.0).cast(pa.time32("s"))
<pyarrow.Time32Scalar: datetime.time(12, 1)>

>>> pa.__version__
'13.0.0.dev497'
>>> pa.scalar(43260.0).cast(pa.time32("s"))
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "pyarrow/scalar.pxi", line 91, in pyarrow.lib.Scalar.cast
  File "/opt/miniconda3/envs/pyarrow-pandas-dev/lib/python3.11/site-packages/pyarrow/compute.py", line 403, in cast
    return call_function("cast", [arr], options, memory_pool)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "pyarrow/_compute.pyx", line 572, in pyarrow._compute.call_function
  File "pyarrow/_compute.pyx", line 367, in pyarrow._compute.Function.call
  File "pyarrow/error.pxi", line 144, in pyarrow.lib.pyarrow_internal_check_status
  File "pyarrow/error.pxi", line 121, in pyarrow.lib.check_status
pyarrow.lib.ArrowNotImplementedError: Unsupported cast from double to time32 using function cast_time32

Not sure if this will be an intentional change, but it appears casting from double to temporal types has changed. In theory for the example above 43260.0 can be cast exactly back to datetime.time(12, 1). I could see how for a value like 43260.1 there would be implicit rounding in pyarrow 12

Component(s)

Python

[rohanjain101]

#35395

I think it's due to this PR where scalars were using a different cast method instead of the compute kernel. If you use an array instead of a scalar, it produces that same error:

>>> pa.array([43260.0]).cast(pa.time32("s"))
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "pyarrow\array.pxi", line 935, in pyarrow.lib.Array.cast
  File "\Python311\Lib\site-packages\pyarrow\compute.py", line 400, in cast
    return call_function("cast", [arr], options, memory_pool)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "pyarrow\_compute.pyx", line 572, in pyarrow._compute.call_function
  File "pyarrow\_compute.pyx", line 367, in pyarrow._compute.Function.call
  File "pyarrow\error.pxi", line 144, in pyarrow.lib.pyarrow_internal_check_status
  File "pyarrow\error.pxi", line 121, in pyarrow.lib.check_status
pyarrow.lib.ArrowNotImplementedError: Unsupported cast from double to time32 using function cast_time32
>>> pa.__version__
'12.0.1'
>>>

[mroeschke]

Thanks for pointing that out @rohanjain101! Moreover, it appears that cast gained a new safe keyword but it doesn't handle numeric to temporal casting #35395 (comment)

[westonpace]

@mroeschke does it work to go from double -> int64 -> temporal?

[westonpace]

(or I guess int32 in this case)

[mroeschke]

@mroeschke does it work to go from double -> int64 -> temporal?

Yeah this case seems to work

>>> import pyarrow as pa
>>> pa.scalar(1.1).cast(pa.int32(), safe=False).cast(pa.time32("s"), safe=False)
<pyarrow.Time32Scalar: datetime.time(0, 0, 1)>
>>> pa.__version__
'13.0.0.dev497'

[westonpace]

It it common to store temporal values as floating point values? My initial reaction is that this is unusual and requiring an intermediate cast is acceptable.

[mroeschke]

In pandas, the use case is to perform reduction functions (e.g. mean) on temporal values, and since the pyarrow reduction kernels don't support temporal values, we cast them numeric first (assuming they are some x value relative to y) and then back.

[jorisvandenbossche]

So this means that pyarrow 13.0.0 will break Series[timestamp].mean() (with the pyarrow backend)?

That's annoying .., but in general I also agree that the extra cast to int is fine (as we required for arrays already before as well). For example, in pandas we also don't allow casting timestamp to float (but do the other way around, a bit inconsistent).

[mroeschke]

Yeah pandas caught that breaking in the nightly arrow job, but I think I addressed Series[timestamp<pyarrow>].mean() by casting to int in pandas https://github.com/pandas-dev/pandas/pull/54110/files

[mroeschke]

With respect to the original issue, happy to close it if unsafe casting from temporal to numeric is intended in pyarrow 13