Skip to content

ERR: Better error msg when merging on tz-aware and tz-naive columns #19525

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 1 commit into from
Feb 3, 2018
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions doc/source/whatsnew/v0.23.0.txt
Original file line number Diff line number Diff line change
Expand Up @@ -342,6 +342,7 @@ Other API Changes
- Addition and subtraction of ``NaN`` from a :class:`Series` with ``dtype='timedelta64[ns]'`` will raise a ``TypeError` instead of treating the ``NaN`` as ``NaT`` (:issue:`19274`)
- Set operations (union, difference...) on :class:`IntervalIndex` with incompatible index types will now raise a ``TypeError`` rather than a ``ValueError`` (:issue:`19329`)
- :class:`DateOffset` objects render more simply, e.g. "<DateOffset: days=1>" instead of "<DateOffset: kwds={'days': 1}>" (:issue:`19403`)
- :func:`pandas.merge` provides a more informative error message when trying to merge on timezone-aware and timezone-naive columns (:issue:`15800`)

.. _whatsnew_0230.deprecations:

Expand Down
27 changes: 10 additions & 17 deletions pandas/core/reshape/merge.py
Original file line number Diff line number Diff line change
Expand Up @@ -940,6 +940,11 @@ def _maybe_coerce_merge_keys(self):
elif is_dtype_equal(lk.dtype, rk.dtype):
continue

msg = ("You are trying to merge on {lk_dtype} and "
"{rk_dtype} columns. If you wish to proceed "
"you should use pd.concat".format(lk_dtype=lk.dtype,
rk_dtype=rk.dtype))

# if we are numeric, then allow differing
# kinds to proceed, eg. int64 and int8, int and float
# further if we are object, but we infer to
Expand Down Expand Up @@ -968,30 +973,18 @@ def _maybe_coerce_merge_keys(self):
pass

# Check if we are trying to merge on obviously
# incompatible dtypes GH 9780
# incompatible dtypes GH 9780, GH 15800
elif is_numeric_dtype(lk) and not is_numeric_dtype(rk):
msg = ("You are trying to merge on {lk_dtype} and "
"{rk_dtype} columns. If you wish to proceed "
"you should use pd.concat".format(lk_dtype=lk.dtype,
rk_dtype=rk.dtype))
raise ValueError(msg)
elif not is_numeric_dtype(lk) and is_numeric_dtype(rk):
msg = ("You are trying to merge on {lk_dtype} and "
"{rk_dtype} columns. If you wish to proceed "
"you should use pd.concat".format(lk_dtype=lk.dtype,
rk_dtype=rk.dtype))
raise ValueError(msg)
elif is_datetimelike(lk) and not is_datetimelike(rk):
msg = ("You are trying to merge on {lk_dtype} and "
"{rk_dtype} columns. If you wish to proceed "
"you should use pd.concat".format(lk_dtype=lk.dtype,
rk_dtype=rk.dtype))
raise ValueError(msg)
elif not is_datetimelike(lk) and is_datetimelike(rk):
msg = ("You are trying to merge on {lk_dtype} and "
"{rk_dtype} columns. If you wish to proceed "
"you should use pd.concat".format(lk_dtype=lk.dtype,
rk_dtype=rk.dtype))
raise ValueError(msg)
elif is_datetime64tz_dtype(lk) and not is_datetime64tz_dtype(rk):
raise ValueError(msg)
elif not is_datetime64tz_dtype(lk) and is_datetime64tz_dtype(rk):
raise ValueError(msg)

# Houston, we have a problem!
Expand Down
4 changes: 3 additions & 1 deletion pandas/tests/reshape/merge/test_merge.py
Original file line number Diff line number Diff line change
Expand Up @@ -1512,11 +1512,13 @@ def test_merge_on_ints_floats_warning(self):
'2011-01-02']),
(pd.date_range('1/1/2011', periods=2, freq='D'), [0, 1]),
(pd.date_range('1/1/2011', periods=2, freq='D'), [0.0, 1.0]),
(pd.date_range('20130101', periods=3),
pd.date_range('20130101', periods=3, tz='US/Eastern')),
([0, 1, 2], Series(['a', 'b', 'a']).astype('category')),
([0.0, 1.0, 2.0], Series(['a', 'b', 'a']).astype('category')),
])
def test_merge_incompat_dtypes(self, df1_vals, df2_vals):
# GH 9780
# GH 9780, GH 15800
# Raise a ValueError when a user tries to merge on
# dtypes that are incompatible (e.g., obj and int/float)

Expand Down