pytest --collect-only: -q, -qq, --verbose

[twmr]

For projects where --verbose is set in pytest.ini or on the cmd-line, you have to use -qq if you want to see the collected test items in a one line per item style:

import unittest

class TestA(unittest.TestCase):
    def test_b(self):
        pass

def test_c():
    pass

(To make the output of pytest a bit better readable, I've removed the ====... header and footer lines from the following outputs)

#> pytest --verbose --collect-only 
platform linux -- Python 3.6.4, pytest-3.4.2, py-1.5.2, pluggy-0.6.0 -- /home/thomas/miniconda/envs/drina_py36/bin/python
cachedir: .pytest_cache
rootdir: /tmp, inifile:
plugins: cov-2.5.1
collected 2 items                                                                                                                                                                                                             
<Module 'testx.py'>
  <UnitTestCase 'TestA'>
    <TestCaseFunction 'test_b'>
  <Function 'test_c'>

#> pytest  --verbose --collect-only -q
platform linux -- Python 3.6.4, pytest-3.4.2, py-1.5.2, pluggy-0.6.0
rootdir: /tmp, inifile:
plugins: cov-2.5.1
collected 2 items                                                                                                                                                                                                             
<Module 'testx.py'>
  <UnitTestCase 'TestA'>
    <TestCaseFunction 'test_b'>
  <Function 'test_c'>

#> pytest --verbose --collect-only --verbose -qq
testx.py::TestA::test_b
testx.py::test_c

If --verbose if omitted from the cmd line (or pytest.ini) one gets the following behavior:

#> pytest --collect-only  -q
testx.py::TestA::test_b
testx.py::test_c

#> pytest --collect-only  -qq
testx.py: 2

I find this a bit inconsistent. I would prefer if the collect-only output is independent of the --verbose flag.

[pytestbot]

GitMate.io thinks possibly related issues are #214 (pytest blows up when collecting modules with uninspectable objects), #1123 (does pytest support to this way to collect test cases?), #1411 (Pytest stops), #1602 (pytest.print()), and #3065 (Pytest in Gitlab-ci Docker Runner does not collect testcases).

[twmr]

@pytestbot you are wrong! ;) None of your linked issues are related to this issue.

Only #3284 is related to this issue.

[nicoddemus]

I would prefer if the collect-only output is independent of the --verbose flag.

Actually they work like this by design:

pytest/_pytest/terminal.py

Lines 24 to 27 in 28df322

	group._addoption('-v', '--verbose', action="count",
	dest="verbose", default=0, help="increase verbosity."),
	group._addoption('-q', '--quiet', action="count",
	dest="quiet", default=0, help="decrease verbosity."),

One is the opposite of the other. 😉

In the end everything is working as it should, IMHO.

[twmr]

Thx @nicoddemus for having a look. This makes sense, but I don't like that the user needs to know whether a projects uses --verbose in the ini file or not. I don't really understand why the default output of --collect-only is so verbose.

[nicoddemus]

I think in "normal" mode it helps visualize and understand how collection works, I kind lean towards your preference but really does seem to be a matter of taste here.

Now that we clarified things, unless you disagree and want to propose to change this I guess we can close this issue then?

[twmr]

We need a user-survey for this ;) I want to propose a change but don't know how yet. Let's keep this ticket open for a few days.

[blueyed]

It might be helpful to allow for giving absolute verbosity.
See https://github.com/Vimjas/covimerage/blob/1e86ebf042b26b332e3d913edaa880a3ba2c3b7c/covimerage/cli.py#L19-L29 for an example - but I would name it --verbosity instead of --loglevel now/here.

[twmr]

Can be closed since #3296 was merged