patsy is a hard dependency?

setup.py checks for the presence of numpy,scipy and pandas, but not patsy.
This is with current git master.

$ python -c "import statsmodels.api as sm"
Traceback (most recent call last):
  File "<string>", line 1, in <module>
  File "/home/travis/virtualenv/python2.7_with_system_site_packages/local/lib/python2.7/site-packages/statsmodels/api.py", line 3, in <module>
    import regression
  File "/home/travis/virtualenv/python2.7_with_system_site_packages/local/lib/python2.7/site-packages/statsmodels/regression/__init__.py", line 1, in <module>
    from linear_model import yule_walker
  File "/home/travis/virtualenv/python2.7_with_system_site_packages/local/lib/python2.7/site-packages/statsmodels/regression/linear_model.py", line 41, in <module>
    import statsmodels.base.model as base
  File "/home/travis/virtualenv/python2.7_with_system_site_packages/local/lib/python2.7/site-packages/statsmodels/base/model.py", line 10, in <module>
    from statsmodels.formula import handle_formula_data
  File "/home/travis/virtualenv/python2.7_with_system_site_packages/local/lib/python2.7/site-packages/statsmodels/formula/__init__.py", line 4, in <module>
    from formulatools import handle_formula_data
  File "/home/travis/virtualenv/python2.7_with_system_site_packages/local/lib/python2.7/site-packages/statsmodels/formula/formulatools.py", line 2, in <module>
    from patsy import dmatrices
ImportError: No module named patsy

[josef-pkt]

I also ran into this with a virtualenv last week.

I would add it as a requirement that can be automatically installed, however, we will most likely include patsy directly in a source or binary distribution, and link it in the repository.

[jseabold]

Related to #549.

[gliptak]

I installed statsmodels using pip:

$ pip install statsmodels
Downloading/unpacking statsmodels
  Downloading statsmodels-0.5.0-tutorial-beta.tar.gz (4.4Mb): 4.4Mb downloaded

when running it fails with ``patsy` not installed.

  File "/.virtualenvs/.env/options/local/lib/python2.7/site-packages/statsmodels/formula/__init__.py", line 4, in <module>
    from formulatools import handle_formula_data
  File "/.virtualenvs/.env/options/local/lib/python2.7/site-packages/statsmodels/formula/formulatools.py", line 2, in <module>
    from patsy import dmatrices
ImportError: No module named patsy

It seems that patsy is indeed a required dependency.

Thanks

[jseabold]

Hmm, I'm considering adding this to install_requires. It's not quite so bad if something goes wrong with a patsy install as it is with numpy/scipy. Thoughts?

[rgommers]

Patsy install_requires numpy though (and Nathaniel doesn't want to change that). So pip/easy_install -U will still upgrade numpy.

[josef-pkt]

Patsy doesn't require a specific version of numpy.

easy_install doesn't upgrade dependencies, at least when I looked at it a long time ago

What does pip install -U do? (I'm not sure and I thought it behaves the same as easy_install)
From a quick search it really sucks. (Uninstalling and reinstalling everything ?)
http://stackoverflow.com/questions/2861183/pip-upgrade-package-without-upgrading-dependencies
https://github.com/pypa/pip/issues/search?q=upgrade

[josef-pkt]

Last time I worked with the virtualenv and pip install, I thought it just didn't recognize my manually installed numpy, and it reinstalled numpy for that reason.

the patsy setup.py can detect an installed numpy version and specify it as
a ver requirement in install_requires. That would hemstring pip -U.

[jseabold]

I'm not sure what the suggestion is for us in these comments / if there is one.

I looked into git submodules some more last night, and that's not the way to go here (I don't think).

So, AFAIK, we have two choices.

1. just specify patsy as a dependencies in the places it's missing and check for it just like we're checking for numpy/scipy/etc.
2. we put it in install_requires and don't worry about whatever it may do / assume users know what they're doing

My suggestion was to go ahead and use install_requires, with the suggested
workaround for the "pip -U" problem raised by @rgommers.

[josef-pkt]

The way I see it now, if pip install -U reinstalls all dependencies, the 2) is out, unless patsy changes.

2. would mean if my tox testing uses pip install -U statsmodels, then it would wipe my virtualenv (as it had before) but it's too stupid to rebuild it again from the binaries. -U statsmodels triggers -U patsy triggers -U numpy which is definitely useless for version testing.
   So it essentially kills my tox version testing

2/2 means my explanation wasn't clear. Let me try again.
If patsy's setup.py contained something along the lines of:

np_ver = ""
try:
   import numpy as np
   np_ver = "==" + str(np.__version__)
except:
   pass

...
setup(
install_requires = ['numpy'+np_ver ]
}

then when you did

pip install -U statsmodels

numpy would not be upgraded when patsy was.

[jseabold]

Yes, but this is a patsy issue not something we can fix on our end right? I can make a PR there.

That's right. It was my understanding that the two projects are siblings.
If that's not the case, my suggestion might not be a good way to go.

[jseabold]

And this would have to be contained in the patsy version that is currently released right? I have to relearn everything about setuptools/distribute/distutils every time I want to get any of this right, and I just don't have time.