Regression: in 3.10 tests are no longer de-dupped resulting in mutations propagating through parameterized test runs

[Strilanc]

What happens

In python 2 with pytest 3.10, when you pass a file or directory argument to pytest multiple times, it executes tests multiple times instead of once. This can result in parameterized tests that mutate their parameters running again with the mutations visible, causing spurious test failures.

This is particularly problematic when using mocks, because the mock value gets re-used and checks such as "mock was only called once" start to fail.

This does not happen in python 3 and did not happen in previous versions of pytest.

Repro steps

0. Create a fresh python 2 virtual environment and install pytest 3.10

1. Create a file test.py containing the following code:

@pytest.mark.parametrize(
    'x,y',
    [
        [
            [], 'passed_in_correctly'
        ]
    ]
)
def test_repro(x, y):
    assert y == 'passed_in_correctly'
    x.append(1)
    assert x == [1]

2. Run pytest . .

3. The test runs twice in the same session and fails the second time

================ test session starts ==============================
platform linux2 -- Python 2.7.13, pytest-3.10.0, py-1.7.0, pluggy-0.8.0
rootdir: [...]
collected 2 items

test.py .F

===================== FAILURES ==============================
_________ test_repro[x0-test] _________________________

x = [1, 1], y = 'test'

    @pytest.mark.parametrize(
        'x,y',
        [
            [
                [], 'test'
            ]
        ]
    )
    def test_repro(x, y):
        assert y == 'test'
        x.append(1)
>       assert x == [1]
E       assert [1, 1] == [1]
E         Left contains more items, first extra item: 1
E         Use -v to get the full diff

test_repro.py:15: AssertionError
===================1 failed, 1 passed in 0.07 seconds ===================

[blueyed]

Closing as duplicate of #4310.
Thanks for the test case already!

[blueyed]

Please provide your config, and the output of pytest --collect-only . ..
I assume that you must have a custom config, since test.py appears to not be collected in the first place, when trying to reproduce this.

[blueyed]

I also assume it happens with py3, too - or is it really specific to py2?

[blueyed]

I assume this is fixed by #4319 (in master), please test and report back.