Mixing relative and absolute paths to tests fails pytest.ini lookup

[peterlauri]

Please see the below simple reproduction of the problem. When mixing relative and absolute path configuration file resolution as per https://pytest.org/latest/customize.html

First execution just relative path (finds pytest.ini)
Second execution with relative and absolute (does not find pytest.ini)
Third execution with argument -s between relative and absolute path (does find pytest.ini)... strange...

➜  plauri@Peters-MacBook-Pro pytest-abs-rel-bug  ls -lrt dir1 dir2
dir1:
total 16
-rw-r--r--  1 plauri  staff   23 Aug 19 14:34 pytest.ini
-rw-r--r--  1 plauri  staff   35 Aug 19 14:36 test_hello.py
drwxr-xr-x  3 plauri  staff  102 Aug 19 14:37 __pycache__

dir2:
total 8
-rw-r--r--  1 plauri  staff   35 Aug 19 14:37 test_hola.py
drwxr-xr-x  3 plauri  staff  102 Aug 19 14:38 __pycache__
➜  plauri@Peters-MacBook-Pro pytest-abs-rel-bug  cat dir1/pytest.ini
[pytest]
addopts = -vv
➜  plauri@Peters-MacBook-Pro pytest-abs-rel-bug  cd dir1
➜  plauri@Peters-MacBook-Pro dir1  ../.ve/bin/py.test test_hello.py
=================================================== test session starts ====================================================
platform darwin -- Python 2.7.10 -- py-1.4.30 -- pytest-2.7.2 -- /Users/plauri/work/play/pytest-abs-rel-bug/.ve/bin/python2.7
rootdir: /Users/plauri/work/play/pytest-abs-rel-bug/dir1, inifile: pytest.ini
collected 1 items

test_hello.py::test_ok PASSED

================================================= 1 passed in 0.00 seconds =================================================
➜  plauri@Peters-MacBook-Pro dir1  ../.ve/bin/py.test test_hello.py /Users/plauri/work/play/pytest-abs-rel-bug/dir2
=================================================== test session starts ====================================================
platform darwin -- Python 2.7.10 -- py-1.4.30 -- pytest-2.7.2
rootdir: /Users/plauri/work/play/pytest-abs-rel-bug, inifile:
collected 2 items

test_hello.py .
../dir2/test_hola.py .

================================================= 2 passed in 0.01 seconds =================================================
➜  plauri@Peters-MacBook-Pro dir1  ../.ve/bin/py.test test_hello.py -s /Users/plauri/work/play/pytest-abs-rel-bug/dir2
=================================================== test session starts ====================================================
platform darwin -- Python 2.7.10 -- py-1.4.30 -- pytest-2.7.2 -- /Users/plauri/work/play/pytest-abs-rel-bug/.ve/bin/python2.7
rootdir: /Users/plauri/work/play/pytest-abs-rel-bug/dir1, inifile: pytest.ini
collected 2 items

test_hello.py::test_ok PASSED
::test_ok <- ../dir2/test_hola.py PASSED

================================================= 2 passed in 0.01 seconds =================================================
➜  plauri@Peters-MacBook-Pro dir1

[peterlauri]

As a workaround one could set the -c CONFIGFILE option.

[ionelmc]

One way to solve this would be changing determine_setup to look for conf files in CWD first, and then look in the common ancestors for all the test paths.

[nicoddemus]

Hmm accordingly to the docs in determining rootdir and inifile, because there are no pytest.ini file in common ancestors for both dir1 and dir2, the correct behavior in my opinion is the one you get whenever you pass both directories, either in absolute or relative form.

The fact that you pass -s and suddenly it finds a pytest.ini file seems to me to be a bug in how the arguments are parsed.

Consider:

def determine_setup(inifile, args):
    print('determine_setup', args)
    if inifile:
        iniconfig = py.iniconfig.IniConfig(inifile)
    ...

When I execute py.test passing the directories either as relative or absolute paths, I always get the same result, no pytest.ini file is found:

(omitting some output for brevity)

X:\temp\relbug\dir1>py.test  test_dir1.py x:\temp\relbug\dir2
determine_setup ['test_dir1.py', 'x:\\temp\\relbug\\dir2']
...
rootdir: x:\temp\relbug, inifile:

X:\temp\relbug\dir1>py.test  x:\temp\relbug\dir1 x:\temp\relbug\dir2
determine_setup ['x:\\temp\\relbug\\dir1', 'x:\\temp\\relbug\\dir2']
rootdir: x:\temp\relbug, inifile:

X:\temp\relbug\dir1>py.test  x:\temp\relbug\dir1 ..\dir2
determine_setup ['x:\\temp\\relbug\\dir1', '..\\dir2']
rootdir: X:\temp\relbug, inifile:

But if I pass a flag (say -s or -v) then suddenly the second path is not passed anymore to determine_setup function, and then it finds the pytest.ini:

X:\temp\relbug\dir1>py.test  x:\temp\relbug\dir1 -v ..\dir2
determine_setup ['x:\\temp\\relbug\\dir1']
rootdir: x:\temp\relbug\dir1, inifile: pytest.ini

Note that it doesn't find pytest.ini if I switch the argument order, as there's no pytest.ini in dir2:

X:\temp\relbug\dir1>py.test  ..\dir2 -v ..\dir1
determine_setup ['..\\dir2']
rootdir: X:\temp\relbug\dir2, inifile:

[peterlauri]

I don't think they are related. The example of having an option that makes it work, was just an extra side note. This is related to not finding the pytest.ini at all, without the option.

[peterlauri]

I think this issue is wrongly closed.

[nicoddemus]

Based on your example and the documentation, it was not supposed to find a pytest.ini at all.

The documentation says:

1. determine the common ancestor directory for the specified args.
2. look for pytest.ini, tox.ini and setup.cfg files in the ancestor directory and upwards.

In your example, your structure was as follows:

pytest-abs-rel-bug/
  dir1/
    pytest.ini
  dir2/  

Consider the following scenarios (let's assume CWD is pytest-abs-rel-bug for all examples, relative or absolute paths don't matter as well):

$ py.test dir1 dir2

In this case, the common ancestor of the files given in the command line is $CWD, which is pytest-abs-rel-bug, which does not contain a pytest.ini file, so none is considered for this test run. This is fine and it is the expected behavior.

$ py.test dir1 -s dir2

In this situation, pytest shouldn't find any pytest.ini file as well according to the documentation. But there was a bug where paths after an option (dir2, which comes after -s) were not being considered when finding the inifile. So what happened was that pytest was interpreting this as just py.test dir1, which then found the pytest.ini file under dir1 and used that for the session.

Hope this explanation makes sense. 😅 Sorry for not posting an explanation before!