Skip to content

slicing dataframe with date index is not working properly #10406

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
kanatti opened this issue Jun 22, 2015 · 5 comments
Closed

slicing dataframe with date index is not working properly #10406

kanatti opened this issue Jun 22, 2015 · 5 comments
Labels
API Design Datetime Datetime data dtype Dtype Conversions Unexpected or buggy dtype conversions Indexing Related to indexing on series/frames, not to indexes themselves

Comments

@kanatti
Copy link

kanatti commented Jun 22, 2015

When slicing data time - indexed dataframe with date in string format, it works:

df.ix['2014-11-12'].head()
Out[55]: 
                     calories      gsr  heart-rate  skin-temp  steps
date                                                                
2014-11-12 00:00:00       2.1  7.04640         NaN    85.7750     30
2014-11-12 00:01:00       1.5  7.40759         NaN    85.7375      0
2014-11-12 00:02:00       1.4  7.46220          49    85.5500      0
2014-11-12 00:03:00       1.4  7.55800         NaN    85.5500      0
2014-11-12 00:04:00       1.5  7.52163         NaN    85.5125      0

But when i try it with a date object, of datetime class , it slices only for time 00:00:00.

d1=datetime.date(2014,11,12)
df.ix[d1]
Out[57]: 
calories       2.1000
gsr            7.0464
heart-rate        NaN
skin-temp     85.7750
steps         30.0000
Name: 2014-11-12 00:00:00, dtype: float64
@jreback
Copy link
Contributor

jreback commented Jun 22, 2015

conceptually similar to #10408. datetime.date objects are not well supported (and likely will not be). simply use datetime.datetime and better yet Timestamp/Period objects and all will be happy.

I suppose one could call this a bug, and if you'd like to dig-in would take a fix.

@jreback jreback added Bug Datetime Datetime data dtype Indexing Related to indexing on series/frames, not to indexes themselves Dtype Conversions Unexpected or buggy dtype conversions Difficulty Intermediate labels Jun 22, 2015
@jreback jreback added this to the Someday milestone Jun 22, 2015
@jorisvandenbossche
Copy link
Member

@jreback I don't know if we should call this a bug. Indexing with a datetime.date does work, but it is the question how it should be interpreted (if it should do partial string indexing or not):
As a timestamp (what you would get from pd.Timestamp(d1)), so then the above behaviour is correct. Or as a 'date' and so similar to providing a string with that date and do a partial string indexing-like thing.

I see the potential confusion in a different result of df.ix['2014-11-12'] and df.ix[date(2014,11,12)]. But personally I would keep the 'partial string indexing' feature restricted to strings.

@jreback
Copy link
Contributor

jreback commented Jun 23, 2015

@jorisvandenbossche hmm, good point. Yeh string indexing encompases internals, but Timestamp/datetime.datetime is an instant in time, its the slicing that actually introduces the interval

so

df.ix[date(2014,11,12):date(2014,11,13)] would be a day.

Introducing objects that expand into a slice then would be a fairly big API change.

@jorisvandenbossche
Copy link
Member

Yes, indeed. Therefore I would close this as not-a-bug.
It can be a bit questionable if a date should be regarded as a instant in time like datetime, but I wouldn't start doing slices with single date objects like with strings.

@mroeschke
Copy link
Member

Agreed that partial string indexing should be reserved for datetime strings only. Closing as this would provide too much API creep.

@mroeschke mroeschke modified the milestones: Someday, No action Mar 26, 2019
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
API Design Datetime Datetime data dtype Dtype Conversions Unexpected or buggy dtype conversions Indexing Related to indexing on series/frames, not to indexes themselves
Projects
None yet
Development

No branches or pull requests

4 participants