parametrize: allow Ellipsis to avoid repeating argument list

[mgeier]

This is an implementation for #518.

This is an example for the usage of parametrize before this PR:

import pytest

testparams = [
    (1, 2, 3, 4, 5, 6, 7, 8),
    (8, 7, 6, 5, 4, 3, 2, 1),
]


@pytest.mark.parametrize("a, b, c, d, e, f, g, h", testparams)
def test_many_args(a, b, c, d, e, f, g, h):
    assert False

The argument list has to be repeated, which is annoying.
After this PR, the argument list can be replaced by ... (or Ellipsis, which also works in Python 2.x):

import pytest

testparams = [
    (1, 2, 3, 4, 5, 6, 7, 8),
    (8, 7, 6, 5, 4, 3, 2, 1),
]


@pytest.mark.parametrize(..., testparams)
def test_many_args(a, b, c, d, e, f, g, h):
    assert False

The auto-deduced parameters must be in the beginning of the parameter list, but any other parameters can be used afterwards, e.g. a fixture:

import pytest

testparams = [
    (1, 2, 3, 4, 5, 6, 7, 8),
    (8, 7, 6, 5, 4, 3, 2, 1),
]


@pytest.fixture
def myfixture():
    pass


@pytest.mark.parametrize(..., testparams)
def test_many_args_and_fixture(a, b, c, d, e, f, g, h, myfixture):
    assert False

[The-Compiler]

There is some prior discussion in #94 as well.

FWIW, I'd like and use this feature - if others agree the idea makes sense, I think you should add some tests before merging however.

[mgeier]

Thanks for the positive response!

I'd like to create a test case, but to be honest, I'm a bit overwhelmed with the tests of pytest.
I guess testing a testing framework is a bit too meta for me ...

Can you give any pointers where I should add a test case?
Probably there are some similar tests already which I can extend?

I only found test_keyword_option_parametrize() in testing/test_mark.py, does this have anything to do with it?

[The-Compiler]

Hmm, it seems the existing pytest.mark.parametrize tests are without using pytest.mark in testing/python/metafunc.py (the test_parametrize_* methods).

[hpk42]

This kind of breaks the way how usually arguments work with pytest test functions. Normally, the order of arguments does not matter which is also mentioned in the docs. Are you sure you can not just write a decorator wrapper for pytest.mark.parametrize with your proposed logic and use that?

[nicoddemus]

Myself I like that pytest test functions are keyword-based so arguments can be given in any order, but at the same time I understand the motivation for the PR.

I like the idea of a separate decorator. @hpk42 are you proposing a to create a new decorator and include it into pytest? If not, this could easily be made available by a plugin then, and wouldn't even require the ellipsis:

@pytest.auto_parametrize(testparams)
def test_many_args_and_fixture(a, b, c, d, e, f, g, h, myfixture):
    assert False

[hpk42]

i would just include the decorator with the project and only create a plugin if multiple projects want to use it. So i am closing this PR here.

[nicoddemus]

Since @The-Compiler demonstrated interest in using it, I think a plugin would be worthwhile.

@mgeier, please don't get discouraged because the PR wasn't accepted into the pytest core, pytest is extensible enough exactly to allow third party plugins to fill any functionality they feel are missing.

Just to give an overview if you've never written a pytest plugin before, you would have to basically create a plugin file like this:

# contents of pytest_auto_parametrize.py
def auto_parametrize(func):
    # decorator implementation

def pytest_namespace():
    return {'auto_parametrize': auto_parametrize}

And write a setup.py file containing an appropriate entry point:

setup(
    name='pytest-auto-parametrize',
    entry_points={
        'pytest11': ['pytest_auto_parametrize = pytest_auto_parametrize'],
    },
    ...
)

Publishing this package on PyPI would allow people to simply pip install pytest-auto-parametrize and use it like I mentioned before:

@pytest.auto_parametrize(testparams)
def test_many_args_and_fixture(a, b, c, d, e, f, g, h, myfixture):
    assert False

That's pretty much it. 😄

Feel free to ask for help on [email protected] if you have any questions.

[mgeier]

@nicoddemus Thanks a lot for your help! I took me a very long time, but I didn't forget!

I tried implementing an extension as suggested above. Since it's quite short, I can include it here in its entirety:

import inspect
import pytest
from _pytest.mark import extract_argvalue


def auto_parametrize(argvalues, *args, **kwargs):
    def decorator(func):
        try:
            argvalue = argvalues[0]
        except IndexError:
            raise ValueError('argvalues must be non-empty')
        except TypeError:
            raise TypeError('argvalues must be a sequence')
        argvalue, _ = extract_argvalue(argvalue)
        argspec = inspect.getargspec(func)[0]
        if isinstance(argvalue, (list, tuple)):
            argnames = argspec[:len(argvalue)]
        else:
            argnames = argspec[0]
        return pytest.mark.parametrize(
            argnames, argvalues, *args, **kwargs)(func)
    return decorator


def pytest_namespace():
    return {'auto_parametrize': auto_parametrize}

This works quite well, but the problem is that I have to use the non-public function _pytest.mark.extract_argvalue(). I guess I'm not supposed to use _pytest, right?

Any comments on the implementation?

I can provide this on PyPI, but I still think this feature would be much nicer if it was built-in to py.test. Is there any chance that my original PR is re-considered?

[mgeier]

I put the code on Github, too: https://github.com/mgeier/pytest-auto-parametrize.

[nicoddemus]

@mgeier thanks for the follow up! 😁

If more people manifest to include in the core perhaps. 😉

[mgeier]

In case somebody is interested, I've uploaded my plugin to PyPI: https://pypi.python.org/pypi/pytest-auto-parametrize/.