-
-
Notifications
You must be signed in to change notification settings - Fork 18.5k
BUG: to_datetime re-parsing Arrow-backed objects #53301
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Changes from 6 commits
234021a
e4b3d0f
76fd635
2fc915b
4ba7a65
ed80297
9b593ff
2630740
325e792
bccd932
f372d25
64478d7
a03c823
File filter
Filter by extension
Conversations
Jump to
Diff view
Diff view
There are no files selected for viewing
Original file line number | Diff line number | Diff line change |
---|---|---|
|
@@ -57,7 +57,10 @@ | |
is_list_like, | ||
is_numeric_dtype, | ||
) | ||
from pandas.core.dtypes.dtypes import DatetimeTZDtype | ||
from pandas.core.dtypes.dtypes import ( | ||
ArrowDtype, | ||
DatetimeTZDtype, | ||
) | ||
from pandas.core.dtypes.generic import ( | ||
ABCDataFrame, | ||
ABCSeries, | ||
|
@@ -400,6 +403,12 @@ def _convert_listlike_datetimes( | |
arg = arg.tz_convert(None).tz_localize("utc") | ||
return arg | ||
|
||
elif isinstance(arg_dtype, ArrowDtype) and arg_dtype.kind == "M": | ||
# TODO: Combine with above if DTI/DTA supports Arrow timestamps | ||
if utc: | ||
arg = arg.astype("timestamp[ns, UTC][pyarrow]") | ||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. I'd rather use There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. .astype shouldn't allow converting a tz-naive to tzaware (or vice-versa). i think this should use tz_convert/tz_localize instead There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. (btw if .astype works on ArrowEA to convert tznaive to tzaware we probably want to change that to match the non-arrow raising behavior) There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more.
Sure, I'll raise a PR after this one gets in. |
||
return arg | ||
|
||
elif lib.is_np_dtype(arg_dtype, "M"): | ||
arg_dtype = cast(np.dtype, arg_dtype) | ||
if not is_supported_unit(get_unit_from_dtype(arg_dtype)): | ||
|
Uh oh!
There was an error while loading. Please reload this page.