Regression in pandas 1.1.0: no longer allows adding a timestamp to timedelta range to create a datetime range.

[analog-cbarber]

Under pandas 1.0.5:

>>> import pandas as pd
>>> pd.Timestamp.now() + pd.timedelta_range('0s','1s', periods=31)
DatetimeIndex([   '2020-08-25 12:27:57.690332',
               '2020-08-25 12:27:57.723665333',
               '2020-08-25 12:27:57.756998666',
                  '2020-08-25 12:27:57.790332',
               '2020-08-25 12:27:57.823665333',
               '2020-08-25 12:27:57.856998666',
                  '2020-08-25 12:27:57.890332',
               '2020-08-25 12:27:57.923665333',
               '2020-08-25 12:27:57.956998666',
                  '2020-08-25 12:27:57.990332',
               '2020-08-25 12:27:58.023665333',
               '2020-08-25 12:27:58.056998666',
                  '2020-08-25 12:27:58.090332',
               '2020-08-25 12:27:58.123665333',
               '2020-08-25 12:27:58.156998666',
                  '2020-08-25 12:27:58.190332',
               '2020-08-25 12:27:58.223665333',
               '2020-08-25 12:27:58.256998666',
                  '2020-08-25 12:27:58.290332',
               '2020-08-25 12:27:58.323665333',
               '2020-08-25 12:27:58.356998666',
                  '2020-08-25 12:27:58.390332',
               '2020-08-25 12:27:58.423665333',
               '2020-08-25 12:27:58.456998666',
                  '2020-08-25 12:27:58.490332',
               '2020-08-25 12:27:58.523665333',
               '2020-08-25 12:27:58.556998666',
                  '2020-08-25 12:27:58.590332',
               '2020-08-25 12:27:58.623665333',
               '2020-08-25 12:27:58.656998666',
                  '2020-08-25 12:27:58.690332'],
              dtype='datetime64[ns]', freq=None)

Under pandas 1.1.0:

>>> import pandas as pd
>>> pd.Timestamp.now() + pd.timedelta_range('0s','1s', periods=31)
Traceback (most recent call last):
  File "/Users/cbarber/miniconda3/envs/episodic-dev3/lib/python3.6/site-packages/pandas/core/arrays/datetimelike.py", line 1157, in _validate_frequency
    raise ValueError
ValueError
The above exception was the direct cause of the following exception:
Traceback (most recent call last):
  File "<input>", line 1, in <module>
  File "/Users/cbarber/miniconda3/envs/episodic-dev3/lib/python3.6/site-packages/pandas/core/indexes/extension.py", line 147, in method
    result = meth(_maybe_unwrap_index(other))
  File "/Users/cbarber/miniconda3/envs/episodic-dev3/lib/python3.6/site-packages/pandas/core/arrays/datetimelike.py", line 1446, in __radd__
    return self.__add__(other)
  File "/Users/cbarber/miniconda3/envs/episodic-dev3/lib/python3.6/site-packages/pandas/core/ops/common.py", line 65, in new_method
    return method(self, other)
  File "/Users/cbarber/miniconda3/envs/episodic-dev3/lib/python3.6/site-packages/pandas/core/arrays/datetimelike.py", line 1406, in __add__
    result = self._add_datetimelike_scalar(other)
  File "/Users/cbarber/miniconda3/envs/episodic-dev3/lib/python3.6/site-packages/pandas/core/arrays/timedeltas.py", line 441, in _add_datetimelike_scalar
    return DatetimeArray(result, dtype=dtype, freq=self.freq)
  File "/Users/cbarber/miniconda3/envs/episodic-dev3/lib/python3.6/site-packages/pandas/core/arrays/datetimes.py", line 284, in __init__
    type(self)._validate_frequency(self, freq)
  File "/Users/cbarber/miniconda3/envs/episodic-dev3/lib/python3.6/site-packages/pandas/core/arrays/datetimelike.py", line 1171, in _validate_frequency
    ) from e
ValueError: Inferred frequency None from passed values does not conform to passed frequency 33333333N

This error really makes no sense given the above code.

Obviously an unintended side-effect of some other fix.

[jreback]

try on 1.1.1 which patched this i think

[analog-cbarber]

Fails on 1.1.1 as well. That is actually when I first noticed the problem.

[jorisvandenbossche]

Yes, can confirm this fails on master as well.

[jorisvandenbossche]

Thanks for the report @analog-cbarber !

(cc @jbrockmendel)

[asishm]

#32377

d219c2c is the first bad commit
commit d219c2c
Author: jbrockmendel [email protected]
Date: Mon Mar 2 17:44:13 2020 -0800

ENH: infer freq in timedelta_range (#32377)

[jbrockmendel]

Two things at work here:

1. TimedeltaArray._add_datetimelike_scalar should use DatetimeArray._simple_new instead of DatetimeArray, which will skip freq validation
2. It looks like the freq we get from the timedelta_range here is incorrect due to rounding errors

[simonjayhawkins]

moved off 1.1.2 milestone (scheduled for this week) as no PRs to fix in the pipeline

[simonjayhawkins]

reopened as #36582 reverted