BUG: read_csv() silently ignores out-of-range integers

[davidchall]

Pandas version checks

• I have checked that this issue has not already been reported.

• I have confirmed this bug exists on the latest version of pandas.

• I have confirmed this bug exists on the main branch of pandas.

Reproducible Example

from io import StringIO
import pandas as pd

# raises exception: cannot safely cast non-equivalent int32 to uint8
pd.Series([-1, 257], dtype="UInt8")

# no exception raised
data = StringIO("x\n-1\n257")
df = pd.read_csv(data, dtype={"x": "UInt8"})

# unexpected wraparound behavior: -1 -> 255, 257 -> 1
df.x

Issue Description

The read_csv() function no longer raises an exception when it encounters an out-of-range integer. Instead, integer overflow silently exhibits a wraparound behavior.

Expected Behavior

On pandas 1.5.3, pd.read_csv() raises a "cannot cast" exception, which is similar to how this scenario is handled by the pd.Series() constructor. I expect pandas 2.1.1 to continue this behavior.

Installed Versions

python : 3.11.5.final.0
python-bits : 64
OS : Windows
OS-release : 10
Version : 10.0.22621

pandas : 2.1.1
numpy : 1.26.0

[paulreece]

I can confirm that the read_csv method no longer raises an exception on the main development branch. However, I received different output than noted for df.x:

>>> data = StringIO("x\n-1\n257")
>>> df = pd.read_csv(data, dtype={"x": "UInt8"})
>>> df.x
0    255
1      1
Name: x, dtype: UInt8

[davidchall]

Hi @paulreece - thanks for your quick reply. Your output is consistent with mine (sorry if I was unclear). The -1 input becomes 255 output and the 257 input becomes 1 output.

[rhshadrach]

I didn't see any notes on this in the whatsnew for 2.0.0 nor 2.1.0. A git bisect should be run to determine where this behavior changed.

[lithomas1]

So I think the issue is that _from_sequence_of_strings basically does an astype to the specified dtype, but without specifying safe casting, here.

pandas/pandas/core/arrays/numeric.py

Line 151 in 51f3d03

values = values.astype(dtype.numpy_dtype, copy=False)

The old code called _safe_cast on IntegerArray which raised a TypeError.

e.g. this

pd.UInt8Dtype().construct_array_type()._from_sequence_of_strings(['-1', '257'], dtype=pd.UInt8Dtype())

cc @jbrockmendel

[jbrockmendel]

i think we would need something like #45588 this but AFAIK there hasn't been any movement on that recently