Weird test names in pytest -v output

[poros]

This happened to me on at least three different projects when running pytest -v. Some of the test cases constantly appears in this weird form

test_dir/test_file.py <- absolute_path_of_test_file::TestClass::testcase PASSED

instead of the usual one.

I tried to look for something similar in the documentation, but I wasn't able to find anything. I can't spot any difference between the "weird" test cases and the regular ones.

Does anyone knows what's going on here?

[nicoddemus]

Could you provide a reproducible example? I found the code responsible for that, but couldn't reproduce it myself with some limited testing.

[poros]

You can try with this GitHub project https://github.com/Yelp/pyleus

This is the pastebin of my output
http://pastebin.com/v7qUkDcV

[nicoddemus]

Thanks, I'll try that later! Could post also what was the exact command you did, and what was the current directory?

[poros]

I'm in the top directory of the project. I run make test (which invokes tox which invokes pytest)

[nicoddemus]

After some digging and analyzing the code you provided, I could narrow down the problem with this simple test:

# contents of test_foo.py
import os
import mock

class TestConfiguration(object):

    @mock.patch.object(os.path, 'exists', autospec=True)
    def test_validate1(self, *args):
        pass

    def test_validate2(self):
        pass

Executing py.test test_foo.py -v yields:

$ py.test test_foo.py -v
test_foo.py::TestConfiguration::test_validate1 <- X:\pytest\test_foo.py PASSED
test_foo.py::TestConfiguration::test_validate2 <- X:\pytest\test_foo.py PASSED
========================== 2 passed in 0.01 seconds ===========================

Commenting out the line with the @mock.patch yields the expected output:

$ py.test test_foo.py -v
test_foo.py::TestConfiguration::test_validate1 PASSED
test_foo.py::TestConfiguration::test_validate2 PASSED
========================== 2 passed in 0.01 seconds ===========================

Applying @mock.patch on tests has some side effects in the pytest's internal machinery, that's why you see this output in some tests but not in others.

It's due to py.path.local.bestrelpath not supporting str objects and confusing the terminal code into thinking the collected item path and the session path are not relative to each other.

I've opened a PR in py lib with a fix. Thanks for reporting this and helping narrowing down the issue! 😄

Long Description

It boils down to the fact that py.path.bestrelpath doesn't support str objects. When a @mock.patch parameter is applied to a test function, reportinfo for function objects ends up returning a str for the file path:

def reportinfo(self):
        # XXX caching?
        obj = self.obj
        if hasattr(obj, 'compat_co_firstlineno'):  # <--- when mock.patch decorator is used
            # nose compatibility
            fspath = sys.modules[obj.__module__].__file__
            if fspath.endswith(".pyc"):
                fspath = fspath[:-1]
            lineno = obj.compat_co_firstlineno
        else: # normal cases          
            fspath, lineno = getfslineno(obj)
        modpath = self.getmodpath()
        assert isinstance(lineno, int)
        return fspath, lineno, modpath

When the mock.patch decorator is applied, the code above will return a fspath variable obtained from the __file__ attribute of the module, which is a full string instead of a py.path.local object which is the normal case.

Next the code that returns a location() for a test item now returns a full path string instead of a nice relative one because bestrelpath doesn't work with the str object returned earlier:

    def location(self):
        # <snip>
                fspath = self.session.fspath.bestrelpath(location[0])
                cache[location[0]] = fspath
            location = (fspath, location[1], str(location[2]))
            self._location = location
            return location

This in turn makes the terminal code think that the collected path and item path are not the same, and shows that distinction in the output:

def _locationline(self, nodeid, fspath, lineno, domain):
        # <snip>
        if fspath:
            res = mkrel(nodeid).replace("::()", "")  # parens-normalization
            if nodeid.split("::")[0] != fspath.replace("\\", "/"):
                res += " <- " + self.startdir.bestrelpath(fspath)
        else:
            res = "[location]"
        return res + " "

😅

[poros]

Thanks for looking into this! :)

[RonnyPfannschmidt]

@nicoddemus what's the state on this one?

[nicoddemus]

It was supposed to be fixed in this PR, but it got dropped... I suggest we drop this until this crops up again.

[poros]

@nicoddemus how about your https://bitbucket.org/pytest-dev/py/pull-requests/31? Are you going to reopen it again? I see the problem was just hg and not your code...

[RonnyPfannschmidt]

we want to drop py.path in the long term anyway, so i wonder how we should attack this

[nicoddemus]

Is just a bug fix, I think we should just re-apply it once #1199 is merged.

[poros]

@nicoddemus cool, thanks!