Skip to content

PERF: DatetimeIndex comparison with Timestamp mismatched resos #52111

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 1 commit into from
Mar 24, 2023

Conversation

jbrockmendel
Copy link
Member

import pandas as pd

rg = pd.date_range("2020-01-01", periods=100_000, freq="s")

ts_ns = pd.Timestamp("1996-01-01 00:00:00.00000000000")
ts_s = pd.Timestamp("1996-01-01")

%timeit rg < ts_s
3.68 ms ± 138 µs per loop (mean ± std. dev. of 7 runs, 100 loops each)   # <- main
139 µs ± 3.96 µs per loop (mean ± std. dev. of 7 runs, 10,000 loops each)  # <- PR

@MarcoGorelli can you confirm this closes #52080

@mroeschke mroeschke requested a review from MarcoGorelli March 22, 2023 17:42
@mroeschke mroeschke added Performance Memory or execution speed performance Non-Nano datetime64/timedelta64 with non-nanosecond resolution labels Mar 22, 2023
Copy link
Member

@MarcoGorelli MarcoGorelli left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good to me, can confirm this resolves the issue - both paths are covered, so should be fine

@phofl any objections?

@MarcoGorelli MarcoGorelli added this to the 2.0 milestone Mar 23, 2023
@MarcoGorelli
Copy link
Member

let's ship it

@MarcoGorelli
Copy link
Member

thanks @jbrockmendel

@MarcoGorelli MarcoGorelli merged commit 25c1942 into pandas-dev:main Mar 24, 2023
meeseeksmachine pushed a commit to meeseeksmachine/pandas that referenced this pull request Mar 24, 2023
@jbrockmendel jbrockmendel deleted the perf-nano-cmp branch March 24, 2023 15:18
Copy link
Member

@phofl phofl left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thx for taking care of this.

phofl pushed a commit that referenced this pull request Mar 24, 2023
…th Timestamp mismatched resos) (#52160)

Backport PR #52111: PERF: DatetimeIndex comparison with Timestamp mismatched resos

Co-authored-by: jbrockmendel <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Non-Nano datetime64/timedelta64 with non-nanosecond resolution Performance Memory or execution speed performance
Projects
None yet
Development

Successfully merging this pull request may close these issues.

PERF: performance problem when comparing timestamp to datetimindex
4 participants