CLN: reorg pandas/io/json to sub-dirs

[jreback]

xref #14904

[jorisvandenbossche]

Super!

I would only call json.py -> _json.py and normalize.py -> _normalize.py so it is very explicit only the things in __init__.py are regarded as public

[jreback]

@jorisvandenbossche done, waiting for build to finish and will merge.

[jreback]

actually I changed it back; you can simply delete the module names and they don't appear in the namespace. I don't like having private _ leading modules in a sub-dir. I think its a bit messy.

[jorisvandenbossche]

you can simply delete the module names and they don't appear in the namespace

The disadvantage of this is that they are still in the namespace, you just can't tab complete them, but still import (I think). I agree that the leading _ is a bit ugly, but IMO that is a clean way to keep the internal organisation of the module internal (of course you can still import it, but at least it gives you a signal that you shouldn't do it), and have all public functions in __init__.py

[jreback]

The disadvantage of this is that they are still in the namespace, you just can't tab complete them, but

In [1]: from pandas.io import json

In [2]: dir(json)
Out[2]: 
['__builtins__',
 '__cached__',
 '__doc__',
 '__file__',
 '__loader__',
 '__name__',
 '__package__',
 '__path__',
 '__spec__',
 'dumps',
 'json_normalize',
 'loads',
 'read_json',
 'to_json']

In [3]: json.json
---------------------------------------------------------------------------
AttributeError                            Traceback (most recent call last)
<ipython-input-3-79fda60ed4a6> in <module>()
----> 1 json.json

AttributeError: module 'pandas.io.json' has no attribute 'json'

you can do from pandas.io.json import json but so what.

this removes most/all of the stuff from the namespace.

[jorisvandenbossche]

Hmm, I am getting some strange errors with ipython tab completion with this branch.

First, on from pandas.io.json import TAB, they still are visible, and on from pandas.io.json.normalize import TAB, I get an AttributeError, after which tab completion completely stops working, also for other things. So ipython does not seem to play very well with del (although I also though that should work)

[jreback]

I think you have to delete the .pyc! they are NOT automatically deleted, when you move things around the old ones are still there.

[jreback]

@jorisvandenbossche py2 is crazy! (nvm was a flake issue)

https://travis-ci.org/jreback/pandas/jobs/199007258

[jreback]

http://stackoverflow.com/questions/17556488/hiding-names-from-export-list-in-python-2-7

[jorisvandenbossche]

Still giving me the same troubles with ipython completion after deleting the pyc files (and even after a full setup.py clean and rebuild)

So I would either just remove the del, or use the _ in the file names.

you can do from pandas.io.json import json but so what.

It's not that important, but in my experience, people tend to see that pd.io.json.json_normalize returns pandas.io.json.normalize.json_normalize in the terminal, and then use that path to use it later.

[jreback]

@jorisvandenbossche just merged. try it again.

[jreback]

we don't use _ leading anywhere else, except to mean compiled code. So pretty loath to resort to that.

[jorisvandenbossche]

You cannot replicate the problem I see? (eg on from pandas.io.json.json.TAB) I am still having the same problem with master.

we don't use _ leading anywhere

Yes, but changing habits always starts somewhere :-) And you typically do it there were you reorganize files and have the chance to do it. I also wanted to propose it for the new plotting module. But anyway, we don't have to decide it here and now.
But, if the ipython completion problems are not due to something on my computer, I would at least remove the del

[TomAugspurger]

@jorisvandenbossche what version of IPython are you using? I don't see any issues with IPython master.

Ahh, but installing IPython 5.2.2, I do see the tab completer crashing :/