Running tests by keywords throws exception when a method contains a number def test_2columns()

[ghost]

Pytest throws INTERNALERROR> SyntaxError: invalid syntax when running multiple selected tests via keywords separated with or when method substrings start with a number: e.g. pytest -k "2columns or 3columns"

Example:

def test_2columns():
    print("2 columns")

def test_3columns():
    print("3 columns")

Running the above code with:

pytest -k "2columns or 3columns" multiple_or_keywords.py

results in the stacktrace below:

======================================================== test session starts =========================================================
platform darwin -- Python 2.7.13, pytest-3.2.2, py-1.4.34, pluggy-0.4.0
sensitiveurl: .*
metadata: {'JAVA_HOME': '/Library/Java/JavaVirtualMachines/jdk1.8.0_121.jdk/Contents/Home', 'Python': '2.7.13', 'Driver': None, 'Capabilities': {}, 'Base URL': '', 'Platform': 'Darwin-16.7.0-x86_64-i386-64bit', 'Plugins': {'variables': '1.7.0', 'selenium': '1.11.1', 'capturelog': '0.7', 'html': '1.16.0', 'base-url': '1.4.1', 'metadata': '1.5.0'}, 'Packages': {'py': '1.4.34', 'pytest': '3.2.2', 'pluggy': '0.4.0'}}
rootdir: pytest-test, inifile:
plugins: variables-1.7.0,  metadata-1.5.0, html-1.16.0, capturelog-0.7, base-url-1.4.1
collected 2 items
INTERNALERROR> Traceback (most recent call last):
INTERNALERROR>   File "/usr/local/lib/python2.7/site-packages/_pytest/main.py", line 110, in wrap_session
INTERNALERROR>     session.exitstatus = doit(config, session) or 0
INTERNALERROR>   File "/usr/local/lib/python2.7/site-packages/_pytest/main.py", line 145, in _main
INTERNALERROR>     config.hook.pytest_collection(session=session)
INTERNALERROR>   File "/usr/local/lib/python2.7/site-packages/_pytest/vendored_packages/pluggy.py", line 745, in __call__
INTERNALERROR>     return self._hookexec(self, self._nonwrappers + self._wrappers, kwargs)
INTERNALERROR>   File "/usr/local/lib/python2.7/site-packages/_pytest/vendored_packages/pluggy.py", line 339, in _hookexec
INTERNALERROR>     return self._inner_hookexec(hook, methods, kwargs)
INTERNALERROR>   File "/usr/local/lib/python2.7/site-packages/_pytest/vendored_packages/pluggy.py", line 334, in <lambda>
INTERNALERROR>     _MultiCall(methods, kwargs, hook.spec_opts).execute()
INTERNALERROR>   File "/usr/local/lib/python2.7/site-packages/_pytest/vendored_packages/pluggy.py", line 614, in execute
INTERNALERROR>     res = hook_impl.function(*args)
INTERNALERROR>   File "/usr/local/lib/python2.7/site-packages/_pytest/main.py", line 155, in pytest_collection
INTERNALERROR>     return session.perform_collect()
INTERNALERROR>   File "/usr/local/lib/python2.7/site-packages/_pytest/main.py", line 648, in perform_collect
INTERNALERROR>     config=self.config, items=items)
INTERNALERROR>   File "/usr/local/lib/python2.7/site-packages/_pytest/vendored_packages/pluggy.py", line 745, in __call__
INTERNALERROR>     return self._hookexec(self, self._nonwrappers + self._wrappers, kwargs)
INTERNALERROR>   File "/usr/local/lib/python2.7/site-packages/_pytest/vendored_packages/pluggy.py", line 339, in _hookexec
INTERNALERROR>     return self._inner_hookexec(hook, methods, kwargs)
INTERNALERROR>   File "/usr/local/lib/python2.7/site-packages/_pytest/vendored_packages/pluggy.py", line 334, in <lambda>
INTERNALERROR>     _MultiCall(methods, kwargs, hook.spec_opts).execute()
INTERNALERROR>   File "/usr/local/lib/python2.7/site-packages/_pytest/vendored_packages/pluggy.py", line 614, in execute
INTERNALERROR>     res = hook_impl.function(*args)
INTERNALERROR>   File "/usr/local/lib/python2.7/site-packages/_pytest/mark.py", line 150, in pytest_collection_modifyitems
INTERNALERROR>     if keywordexpr and not matchkeyword(colitem, keywordexpr):
INTERNALERROR>   File "/usr/local/lib/python2.7/site-packages/_pytest/mark.py", line 233, in matchkeyword
INTERNALERROR>     return eval(keywordexpr, {}, mapping)
INTERNALERROR>   File "<string>", line 1
INTERNALERROR>     2columns or 3columns
INTERNALERROR>            ^
INTERNALERROR> SyntaxError: invalid syntax

===================================================== 3 warnings in 0.00 seconds =====================================================

NOTE: The error seems to be related to the way or works, since pytest -k "2columns" works fine.

[RonnyPfannschmidt]

as you can see in the line just above eval is used for those expressions, which really isnt nice or good to have, but the way it is right now since nobody came up with a backward compatible parser

you can work around by adding other characters in front like _2colums which is valid python syntax again

[ghost]

@RonnyPfannschmidt:

you can work around by adding other characters in front like _2colums which is valid python syntax again

that's exactly what I did; just thought it might help to have it reported in case anyone is up to taking it up

[robert-cody]

There is another problem where this workaround doesn't work. Assume we have these tests:

@pytest.mark.parametrize("boolean", (True, False))
def test_xxx(boolean):
    pass

@pytest.mark.parametrize("boolean", (True, False))
def test_yyy(boolean):
    pass

-k 'test_xxx or test_yyy' - works fine
-k 'test_xxx[False] or test_yyy[True]' - throws exception "TypeError: 'bool' object is not subscriptable".

[RonnyPfannschmidt]

well, - again this is due to using python syntax - test_xxx[False] is recognized as a normal python operation, not as a full string to match a keyword on

luckyly - if you want to run fill tests by name you can use pytest test_foo.py::test_xxx[False] test_foo.py::test_yyy[True] to explicitly list the items you want to run

[RonnyPfannschmidt]

closing as not actionable