Skip to content

Cannot index TimedeltaIndex with np.timedelta64 scalars #20393

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
shoyer opened this issue Mar 17, 2018 · 1 comment · Fixed by #20408
Closed

Cannot index TimedeltaIndex with np.timedelta64 scalars #20393

shoyer opened this issue Mar 17, 2018 · 1 comment · Fixed by #20408
Labels
Indexing Related to indexing on series/frames, not to indexes themselves Timedelta Timedelta data type
Milestone

Comments

@shoyer
Copy link
Member

shoyer commented Mar 17, 2018

Code Sample, a copy-pastable example if possible

In [1]: import pandas as pd

In [2]: import numpy as np

In [3]: s = pd.Series(range(11), pd.timedelta_range('0 days', '10 days'))

In [4]: s
Out[4]:
0 days      0
1 days      1
2 days      2
3 days      3
4 days      4
5 days      5
6 days      6
7 days      7
8 days      8
9 days      9
10 days    10
Freq: D, dtype: int64

In [5]: s.loc[np.timedelta64(0, 'ns'):]
---------------------------------------------------------------------------
TypeError                                 Traceback (most recent call last)
<ipython-input-5-44a523c6460e> in <module>()
----> 1 s.loc[np.timedelta64(0, 'ns'):]

~/miniconda3/envs/xarray-py36/lib/python3.6/site-packages/pandas/core/indexing.py in __getitem__(self, key)
   1371
   1372             maybe_callable = com._apply_if_callable(key, self.obj)
-> 1373             return self._getitem_axis(maybe_callable, axis=axis)
   1374
   1375     def _is_scalar_access(self, key):

~/miniconda3/envs/xarray-py36/lib/python3.6/site-packages/pandas/core/indexing.py in _getitem_axis(self, key, axis)
   1579         if isinstance(key, slice):
   1580             self._has_valid_type(key, axis)
-> 1581             return self._get_slice_axis(key, axis=axis)
   1582         elif is_bool_indexer(key):
   1583             return self._getbool_axis(key, axis=axis)

~/miniconda3/envs/xarray-py36/lib/python3.6/site-packages/pandas/core/indexing.py in _get_slice_axis(self, slice_obj, axis)
   1404         labels = obj._get_axis(axis)
   1405         indexer = labels.slice_indexer(slice_obj.start, slice_obj.stop,
-> 1406                                        slice_obj.step, kind=self.name)
   1407
   1408         if isinstance(indexer, slice):

~/miniconda3/envs/xarray-py36/lib/python3.6/site-packages/pandas/core/indexes/base.py in slice_indexer(self, start, end, step, kind)
   3455         """
   3456         start_slice, end_slice = self.slice_locs(start, end, step=step,
-> 3457                                                  kind=kind)
   3458
   3459         # return a slice

~/miniconda3/envs/xarray-py36/lib/python3.6/site-packages/pandas/core/indexes/base.py in slice_locs(self, start, end, step, kind)
   3656         start_slice = None
   3657         if start is not None:
-> 3658             start_slice = self.get_slice_bound(start, 'left', kind)
   3659         if start_slice is None:
   3660             start_slice = 0

~/miniconda3/envs/xarray-py36/lib/python3.6/site-packages/pandas/core/indexes/base.py in get_slice_bound(self, label, side, kind)
   3582         # For datetime indices label may be a string that has to be converted
   3583         # to datetime boundary according to its resolution.
-> 3584         label = self._maybe_cast_slice_bound(label, side, kind)
   3585
   3586         # we need to look up the label

~/miniconda3/envs/xarray-py36/lib/python3.6/site-packages/pandas/core/indexes/timedeltas.py in _maybe_cast_slice_bound(self, label, side, kind)
    739                         Timedelta(1, 'ns'))
    740         elif is_integer(label) or is_float(label):
--> 741             self._invalid_indexer('slice', label)
    742
    743         return label

~/miniconda3/envs/xarray-py36/lib/python3.6/site-packages/pandas/core/indexes/base.py in _invalid_indexer(self, form, key)
   1574                         "indexers [{key}] of {kind}".format(
   1575                             form=form, klass=type(self), key=key,
-> 1576                             kind=type(key)))
   1577
   1578     def get_duplicates(self):

TypeError: cannot do slice indexing on <class 'pandas.core.indexes.timedeltas.TimedeltaIndex'> with these indexers [0] of <class 'numpy.timedelta64'>

Problem description

I expect this to work as if I did indexing with a pd.Timedelta object:

In [6]: s.loc[pd.Timedelta('0 days'):]
Out[6]:
0 days      0
1 days      1
2 days      2
3 days      3
4 days      4
5 days      5
6 days      6
7 days      7
8 days      8
9 days      9
10 days    10
Freq: D, dtype: int64

This was surprising because every other pandas index can be indexed by the corresponding pandas scalar type. For example, this works with DatetimeIndex:

In [9]: s2 = pd.Series(range(10), pd.date_range('2001-01-01', periods=10))

In [10]: s2.loc[np.datetime64('2001-01-01'):]
Out[10]:
2001-01-01    0
2001-01-02    1
2001-01-03    2
2001-01-04    3
2001-01-05    4
2001-01-06    5
2001-01-07    6
2001-01-08    7
2001-01-09    8
2001-01-10    9
Freq: D, dtype: int64

Expected Output

Output of pd.show_versions()

INSTALLED VERSIONS ------------------ commit: None python: 3.6.4.final.0 python-bits: 64 OS: Darwin OS-release: 17.4.0 machine: x86_64 processor: i386 byteorder: little LC_ALL: None LANG: en_US.UTF-8 LOCALE: en_US.UTF-8

pandas: 0.22.0
pytest: 3.4.0
pip: 9.0.1
setuptools: 38.5.1
Cython: None
numpy: 1.14.0
scipy: 1.0.0
pyarrow: None
xarray: 0.10.2
IPython: 6.2.1
sphinx: 1.7.0
patsy: 0.5.0
dateutil: 2.6.1
pytz: 2018.3
blosc: None
bottleneck: 1.2.1
tables: None
numexpr: None
feather: None
matplotlib: 2.1.2
openpyxl: None
xlrd: None
xlwt: None
xlsxwriter: None
lxml: None
bs4: 4.6.0
html5lib: 1.0.1
sqlalchemy: None
pymysql: None
psycopg2: None
jinja2: 2.10
s3fs: None
fastparquet: None
pandas_gbq: None
pandas_datareader: None

@jreback
Copy link
Contributor

jreback commented Mar 18, 2018

its prob just not coercing. a PR would be helpful :>

@jreback jreback added Indexing Related to indexing on series/frames, not to indexes themselves Difficulty Intermediate labels Mar 18, 2018
@jreback jreback added this to the Next Major Release milestone Mar 18, 2018
@jreback jreback added the Timedelta Timedelta data type label Mar 18, 2018
@jreback jreback modified the milestones: Next Major Release, 0.23.0 Mar 19, 2018
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Indexing Related to indexing on series/frames, not to indexes themselves Timedelta Timedelta data type
Projects
None yet
Development

Successfully merging a pull request may close this issue.

2 participants