Skip to content

dtype does not get set to object when adding column with df.at[...] = ['some list'] #30649

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
McToel opened this issue Jan 3, 2020 · 5 comments
Assignees
Labels
Indexing Related to indexing on series/frames, not to indexes themselves Needs Tests Unit test(s) needed to prevent regressions

Comments

@McToel
Copy link

McToel commented Jan 3, 2020

Code Sample, a copy-pastable example if possible

df = pd.DataFrame(data={'a':[0,0]})
# df['b'] = None
# df['b'] = df['b'].astype('object')
df.at[0, 'b'] = ['a']
df.at[1, 'b'] = ['a']

>>> df
   a    b
0  0    a
1  0  [a]

Problem description

df.at[0, 'b'] should be ['a'] instead of 'a'.
I understand that it does not work when while adding the column but it feels inconsistent anyways.
When adding line 2 and 3 it works as expected.

Expected Output

>>> df
   a    b
0  0  [a]
1  0  [a]

Output of pd.show_versions()

INSTALLED VERSIONS

commit : None
python : 3.7.6.final.0
python-bits : 64
OS : Windows
OS-release : 10
machine : AMD64
processor : Intel64 Family 6 Model 94 Stepping 3, GenuineIntel
byteorder : little
LC_ALL : None
LANG : None
LOCALE : None.None

pandas : 0.25.3
numpy : 1.18.0
pytz : 2019.3
dateutil : 2.8.1
pip : 19.3.1
setuptools : 41.2.0
Cython : None
pytest : None
hypothesis : None
sphinx : None
blosc : None
feather : None
xlsxwriter : None
lxml.etree : None
html5lib : None
pymysql : None
psycopg2 : None
jinja2 : 2.10.3
IPython : 7.10.2
pandas_datareader: None
bs4 : None
bottleneck : None
fastparquet : None
gcsfs : None
lxml.etree : None
matplotlib : None
numexpr : None
odfpy : None
openpyxl : None
pandas_gbq : None
pyarrow : None
pytables : None
s3fs : None
scipy : 1.4.1
sqlalchemy : None
tables : None
xarray : None
xlrd : None
xlwt : None
xlsxwriter : None

@ellequelle
Copy link
Contributor

DataFrame.at isn't intended for creating new columns. From the docs:

DataFrame.at
...
    Raises
    ------
    KeyError
        If 'label' does not exist in DataFrame.

It should work if you uncomment your second line:

df = pd.DataFrame(data={'a':[0,0]})
df['b'] = None
df.at[0, 'b'] = ['a']
df.at[1, 'b'] = ['a']

>>> df
   a    b
0  0  [a]
1  0  [a]

@TomAugspurger
Copy link
Contributor

That line of docs was added in #20290, and doesn't appear to be tested. It's not clear that that's the intentional behavior.

@TomAugspurger TomAugspurger added the Indexing Related to indexing on series/frames, not to indexes themselves label Jan 3, 2020
@mroeschke mroeschke added Bug Needs Discussion Requires discussion from core team before further action Nested Data Data where the values are collections (lists, sets, dicts, objects, etc.). labels Jul 25, 2021
@phofl
Copy link
Member

phofl commented Dec 30, 2021

This seems to work now, may need tests

@phofl phofl added good first issue Needs Tests Unit test(s) needed to prevent regressions and removed Bug Needs Discussion Requires discussion from core team before further action Nested Data Data where the values are collections (lists, sets, dicts, objects, etc.). labels Dec 30, 2021
@FactorizeD
Copy link
Contributor

take

@topper-123
Copy link
Contributor

This isn't actually fixed in v2.0.0rc1 AFAIKS, i.e. I get the same result as OP.

In general, the support in Pandas for non-scalar values is not all that good, so maybe not so surprising this is happening.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Indexing Related to indexing on series/frames, not to indexes themselves Needs Tests Unit test(s) needed to prevent regressions
Projects
None yet
8 participants