Skip to content

replace on Series with timezone-aware data breaks #11775

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
multiloc opened this issue Dec 6, 2015 · 1 comment
Closed

replace on Series with timezone-aware data breaks #11775

multiloc opened this issue Dec 6, 2015 · 1 comment

Comments

@multiloc
Copy link
Contributor

multiloc commented Dec 6, 2015

replace (and by extension other masking operations) fail on Series with tz-aware data. Observing the following behavior:

# non-tz aware
In [23]: ser = pd.Series([pd.NaT, pd.Timestamp('2015/01/01')])

In [24]: ser
Out[24]: 
0          NaT
1   2015-01-01
dtype: datetime64[ns]

In [25]: ser.replace(pd.NaT, pd.Timestamp.min)
Out[25]: 
0   1677-09-22 00:12:43.145225
1   2015-01-01 00:00:00.000000
dtype: datetime64[ns]

# timezone-aware
In [26]: ser = pd.Series([pd.NaT, pd.Timestamp('2015/01/01', tz='UTC')])

In [27]: ser.replace(pd.NaT, pd.Timestamp.min)

---------------------------------------------------------------------------
TypeError                                 Traceback (most recent call last)
<ipython-input-27-cfc55c339a93> in <module>()
----> 1 ser.replace(pd.NaT, pd.Timestamp.min)

/home/env/local/lib/python2.7/site-packages/pandas/core/generic.py in replace(self, to_replace, value, inplace, limit, regex, method, axis)
   2994                 elif not com.is_list_like(value):  # NA -> 0
   2995                     new_data = self._data.replace(to_replace=to_replace, value=value,
-> 2996                                                   inplace=inplace, regex=regex)
   2997                 else:
   2998                     msg = ('Invalid "to_replace" type: '

/home/env/local/lib/python2.7/site-packages/pandas/core/internals.py in replace(self, **kwargs)
   2760 
   2761     def replace(self, **kwargs):
-> 2762         return self.apply('replace', **kwargs)
   2763 
   2764     def replace_list(self, src_list, dest_list, inplace=False, regex=False):

/home/env/local/lib/python2.7/site-packages/pandas/core/internals.py in apply(self, f, axes, filter, do_integrity_check, consolidate, **kwargs)
   2709                                                  copy=align_copy)
   2710 
-> 2711             applied = getattr(b, f)(**kwargs)
   2712 
   2713             if isinstance(applied, list):

/home/env/local/lib/python2.7/site-packages/pandas/core/internals.py in replace(self, to_replace, value, inplace, filter, regex)
    567                 return [self]
    568             return [self.copy()]
--> 569         return self.putmask(mask, value, inplace=inplace)
    570 
    571     def setitem(self, indexer, value):

/home/env/local/lib/python2.7/site-packages/pandas/core/internals.py in putmask(self, mask, new, align, inplace, axis, transpose)
    723 
    724             print
--> 725             np.putmask(new_values, mask, new)
    726 
    727         # maybe upcast me

TypeError: putmask() argument 1 must be numpy.ndarray, not DatetimeIndex

Believe this was introduced in 0.17.0, and might be related to #11693 - however, the patch in that issue doesn't appear to fix this.

@multiloc
Copy link
Contributor Author

multiloc commented Dec 6, 2015

NVM, appears fixed in 0.17.1:

In [3]: ser = pd.Series([pd.NaT, pd.Timestamp('2015/01/01', tz='UTC')])

In [4]: ser.replace(pd.NaT, pd.Timestamp.min)
Out[4]: 
0    1677-09-22 00:12:43.145225
1     2015-01-01 00:00:00+00:00
dtype: object

In [5]: ser = pd.Series([pd.NaT, pd.Timestamp('2015/01/01')])

In [6]: ser.replace(pd.NaT, pd.Timestamp.min)
Out[6]: 
0   1677-09-22 00:12:43.145225
1   2015-01-01 00:00:00.000000
dtype: datetime64[ns]

@multiloc multiloc closed this as completed Dec 6, 2015
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant