Can't run tests with py.test

[techtonik]

Documented way of running tests with just py.test doesn't work.

# py.test
Traceback (most recent call last):
  File "/usr/local/lib/python2.7/dist-packages/_pytest/config.py", line 379, in _importconftest
    mod = conftestpath.pyimport()
  File "/usr/local/lib/python2.7/dist-packages/py/_path/local.py", line 668, in pyimport
    __import__(modname)
  File "/usr/local/lib/python2.7/dist-packages/_pytest/assertion/rewrite.py", line 212, in load_module
    py.builtin.exec_(co, mod.__dict__)
  File "/usr/local/lib/python2.7/dist-packages/py/_builtin.py", line 221, in exec_
    exec2(obj, globals, locals)
  File "<string>", line 7, in exec2
  File "/root/pip/tests/conftest.py", line 10, in <module>
    import pip._internal
ImportError: No module named _internal
ERROR: could not load /root/pip/tests/conftest.py

tox runs fine though.

[benoit-pierre]

You need to add src to the Python path: PYTHONPATH=$PWD/src py.test

[benoit-pierre]

And yes the documentation should be updated :P.

[techtonik]

Command helps, but tests fail - https://pastebin.mozilla.org/9073260 - which is strange, considering that builds are green.

[pradyunsg]

I think the tests ran invoked the system pip. (/usr/bin/pip on line 58 makes me think so)

I, for one, use the tox environment itself -- with ./.tox/py36/bin/py.test aliased to tox-pytest (and a tox-pip as well).

[pradyunsg]

Also, to pass arguments to pytest through tox, you can run tests as tox -e py36 -- {args} and they'll get passed to the underlying pytest call.

[techtonik]

.tox/py27/bin/py.test tests/functional/test_install_reqs.py fixes 2 tests and 3 still fail - https://pastebin.mozilla.org/?diff=9073266

tox -e py27 -- tests/functional/test_install_reqs.py gives the same result.

[pradyunsg]

One of them seems to be because SVN isn't installed. The other seems to be a more esoteric error. One of them is an expected failure.

[pradyunsg]

@techtonik Anything actionable here?

[pradyunsg]

(note: the documentation has been updated to say install everything in dev-requirements.txt)

[pradyunsg]

Closing due to lack of activity.

[techtonik]

@pradyunsg actionable - open https://pip.pypa.io/en/stable/development/#running-tests and run tests with all ways to confirm it is broken for you too.

[RonnyPfannschmidt]

i believe all thats needed is a note to run the test against the installed pip ^^

[pradyunsg]

So... I took too much time to come around to this. Thanks for filing this @techtonik!

---

I think we should just remove all the non-tox ways to running tests. I don't think any of the pip maintainers use those (which is why they've broken over time) and having one way to run tests is easier to understand, maintain, and help contributors with.

#5526 does this and (a lot) more. Would appreciate feedback on that PR. :)

[pfmoore]

+1 on this, but just to be clear, the docs should describe the "normal", or "recommended" workflow. That's not to say that we don't allow people to (say) run pytest manually, just that we expect people to work out the correct way to do so for themselves, not expect us to explain how to use pytest in the pip docs.

[pradyunsg]

That's not to say that we don't allow people to (say) run pytest manually, just that we expect people to work out the correct way to do so for themselves, not expect us to explain how to use pytest in the pip docs.

Agreed. That's exactly what I've done in #5526.

[techtonik]

Does tox address all use cases. For me the primary use case is to repeatedly run one or small group of tests while I debug some feature as fast as possible. If dev docs says run tests and they take 15+ minutes for each run, nobody will be interested to do tested development.

[pradyunsg]

Yeah. It does. It's setup such that it forwards arguments to pytest, so you can do tox -e py36 -- -k my_test or basically whatever you'd have done with pytest.

[pfmoore]

Also, nobody's stopping you running pytest by hand. We're just saying that tox sets everything up cleanly and lets you run whatever pytest invocation you want. But if you prefer to run pytest manually, that's fine - it's just up to you to make sure you do things like setting PYTHONPATH, etc.

By making tox the documented option, we avoid having to turn the pip docs into a pytest tutorial.

[techtonik]

Yeah. It does. It's setup such that it forwards arguments to pytest, so you can do tox -e py36 -- -k my_test or basically whatever you'd have done with pytest.

And how much time it takes to run a single test with tox?

Also, nobody's stopping you running pytest by hand.

Except that it doesn't run #4878.

[pradyunsg]

Yes, pytest doesn't work out of the box. This is a consequence of the fact that the project is structured with an src directory to explicitly push devs to test against the installed version of pip when they run the tests. You need to install the development version of pip before actually running the tests.

One way to do it:

$ pip install -e .
$ # hack, hack, hack
$ pytest

---

And how much time it takes to run a single test with tox?

It does take slightly more time, since tox checks the environment and then installs the development version of pip. On my machine, after setting up the venv on the first run, this takes under half a second. But if even that's an issue then after tox sets up an environment the first time, you can just call the pytest in that environment:

.tox/py36/bin/pytest -k test_expands_path

This is the same (functionally) as doing:

tox -e py36 -- -k test_expands_path

This works because pip's installed via setup.py develop in the tox env.

---

And, you've linked to the same issue. And we're switching away from suggesting that pytest works out of the box in #5526. Explaining why and how to use pytest to run the tests is not something I think we should do in pip's docs.

Also, nobody's stopping you running pytest by hand.

+ in this comment I've told you how to use pytest if you really want to. This isn't something I encourage; heck I'd say that don't do this unless you understand that this may cause other issues like a typo can cause your development virtual environment to break and they're something you want to deal with.

[lock]

This thread has been automatically locked since there has not been any recent activity after it was closed. Please open a new issue for related bugs.