Skip to content

Commit 5507452

Browse files
authored
Test for pd.to_sql column error if data contains -np.inf (#34493)
* TST: pd.to_sql for dataframes with -np.inf (#34431) * DOC: updated what's new (#34431) * DOC: improved entry (#34431) * TST: moved to _TestSQLAlchemy + added round trips * TST: rename + add comment with GH issue # * TST: rewrote using pytest.mark.parametrize for arg to DataFrame * TST: removed underscore from _input * DOC: added double backtick to np.inf & removed extraneous space * TST: pd.to_sql for dataframes with -np.inf (#34431) * DOC: updated what's new (#34431) * DOC: improved entry (#34431) * TST: moved to _TestSQLAlchemy + added round trips * TST: rename + add comment with GH issue # * TST: rewrote using pytest.mark.parametrize for arg to DataFrame * TST: removed underscore from _input * DOC: added double backtick to np.inf & removed extraneous space * BUG: add catch for MySQL error with np.inf * use regex for string match + add runtime import * clean up regex * TST: update to catch error for -np.inf with MySQL * DOC: resolved conflict in whatsnew * TST: update test_to_sql_with_neg_npinf * fixed error handler syntax in SQLDatabase.to_sql * fixed error handler syntax in SQLDatabase.to_sql * TST: added an xfail test for npinf entries with mysql * fixed imports * added reference to GH issue * fixed test_to_sql_with_npinf error catch * fixed spelling error in message (can not -> cannot) * DOC: added info re MySQL ValueError to whatsnew * fixed variable name in to_sql * replaced sqlalchemy's dialect.name with flavor * fixed typo in test_to_sql-with-npinf
1 parent c78c537 commit 5507452

File tree

3 files changed

+33
-1
lines changed

3 files changed

+33
-1
lines changed

doc/source/whatsnew/v1.1.0.rst

+1
Original file line numberDiff line numberDiff line change
@@ -1044,6 +1044,7 @@ I/O
10441044
- Bug in :meth:`HDFStore.append_to_multiple` was raising a ``ValueError`` when the min_itemsize parameter is set (:issue:`11238`)
10451045
- Bug in :meth:`~HDFStore.create_table` now raises an error when `column` argument was not specified in `data_columns` on input (:issue:`28156`)
10461046
- :meth:`read_json` now could read line-delimited json file from a file url while `lines` and `chunksize` are set.
1047+
- Bug in :meth:`DataFrame.to_sql` when reading DataFrames with ``-np.inf`` entries with MySQL now has a more explicit ``ValueError`` (:issue:`34431`)
10471048

10481049
Plotting
10491050
^^^^^^^^

pandas/io/sql.py

+14-1
Original file line numberDiff line numberDiff line change
@@ -1391,7 +1391,20 @@ def to_sql(
13911391
dtype=dtype,
13921392
)
13931393
table.create()
1394-
table.insert(chunksize, method=method)
1394+
1395+
from sqlalchemy import exc
1396+
1397+
try:
1398+
table.insert(chunksize, method=method)
1399+
except exc.SQLAlchemyError as err:
1400+
# GH34431
1401+
msg = "(1054, \"Unknown column 'inf' in 'field list'\")"
1402+
err_text = str(err.orig)
1403+
if re.search(msg, err_text):
1404+
raise ValueError("inf cannot be used with MySQL") from err
1405+
else:
1406+
raise err
1407+
13951408
if not name.isdigit() and not name.islower():
13961409
# check for potentially case sensitivity issues (GH7815)
13971410
# Only check when name is not a number and name is not lower case

pandas/tests/io/test_sql.py

+18
Original file line numberDiff line numberDiff line change
@@ -1813,6 +1813,24 @@ def main(connectable):
18131813
DataFrame({"test_foo_data": [0, 1, 2]}).to_sql("test_foo_data", self.conn)
18141814
main(self.conn)
18151815

1816+
@pytest.mark.parametrize(
1817+
"input",
1818+
[{"foo": [np.inf]}, {"foo": [-np.inf]}, {"foo": [-np.inf], "infe0": ["bar"]}],
1819+
)
1820+
def test_to_sql_with_negative_npinf(self, input):
1821+
# GH 34431
1822+
1823+
df = pd.DataFrame(input)
1824+
1825+
if self.flavor == "mysql":
1826+
msg = "inf cannot be used with MySQL"
1827+
with pytest.raises(ValueError, match=msg):
1828+
df.to_sql("foobar", self.conn, index=False)
1829+
else:
1830+
df.to_sql("foobar", self.conn, index=False)
1831+
res = sql.read_sql_table("foobar", self.conn)
1832+
tm.assert_equal(df, res)
1833+
18161834
def test_temporary_table(self):
18171835
test_data = "Hello, World!"
18181836
expected = DataFrame({"spam": [test_data]})

0 commit comments

Comments
 (0)