BUG: json invoke default handler for unsupported numpy dtypes #12878
Conversation
jreback
added
Bug
Enhancement
IO JSON
| @@ -203,6 +203,7 @@ Performance Improvements | |||
|
|
|||
|
|
|||
|
|
|||
| - Bug in ``DataFrame.to_json`` with unsupported `dtype` not passed to default handler (:issue:`12554`). | |||
There was a problem hiding this comment.
should go in bug fix section
Komnomnomnom
force-pushed
the
np-default-handler
branch
from
a8e1490 to
2ef21d6
Compare
|
ok so other PR merged, you can rebase |
Komnomnomnom
force-pushed
the
np-default-handler
branch
from
2ef21d6 to
c94abe7
Compare
|
FYI windows test https://ci.appveyor.com/project/Komnomnomnom/pandas/build/1.0.4 . Thanks for the tip! |
|
@Komnomnomnom gr8!. note that appveyor is REALLY SLOW! |
Komnomnomnom
force-pushed
the
np-default-handler
branch
2 times, most recently
from
5cf2147 to
ebf77e3
Compare
Komnomnomnom
force-pushed
the
np-default-handler
branch
from
ebf77e3 to
daa256c
Compare
|
|
||
| from datetime import timedelta | ||
| dftd = DataFrame([timedelta(23), timedelta(seconds=5), 42]) | ||
| * check if the object has defined a ``toDict`` method and call it. |
There was a problem hiding this comment.
where did toDict come from?
There was a problem hiding this comment.
ujson. Has been there since the beginning
There was a problem hiding this comment.
I am 👍 on deprecating / removing toDict and the object traversal behaviour.
There was a problem hiding this comment.
afaik ujson is the only library that supports toDict and fallback to object traversal (i.e. serialising obj.__dict__)
There was a problem hiding this comment.
sure, go ahead and deprecate and use obj.__dict__
There was a problem hiding this comment.
To be clear I'm suggesting deprecating toDict and the fallback to obj.__dict__
So default_handler would be the only fallback behaviour (and possibly a future default_enc)
There was a problem hiding this comment.
oh, I c.
for sure deprecate toDict().
is there something problematic with obj.__dict__? e.g. maybe as a fallback to default_handler (which you would use first, or if None then use obj.__dict__).
actually on second though. __dict__ is internal, however, you could do dict(obj) to coerce (and use its output).
There was a problem hiding this comment.
The current implementation uses dir(obj) and just encodes 'public' attrs. It works ok sometimes.
I don't have strong feelings on removing it but In my experience unless the objects are very simple it's generally not what you want and it's also quite brittle (OverflowErrors are common).
There was a problem hiding this comment.
ok, with deprecating non-supported things and just using default_handler that's what its there for. in theory using __json__ might be ok, but not really sure what is the 'standard'.
|
@jreback build clean (apart from msgpack & pickle again) also windows is ok: |
|
just remember how to fix the tag issue. on your local master tags come from your fork (and not from the clone, pandas) next time you branch you should be good |
|
thanks! make an issue about the deprecations for 0.19.0 pls. |
git diff upstream/master | flake8 --diffRecommend to merge after #12802 has been accepted. (valgrind should be clean then)