ENH: -k with regex?

[tylerjereddy]

pytest -k is reasonably powerful for selecting tests to run, but there are certain selection tasks that are impossible with its current limitations (or at least, largely impractical). I think I've got one described in the README over at https://github.com/tylerjereddy/pytest-regex. I'm wondering if the team would be open to allowing regex pass-through in the -k ... option as an augmentation to the current selection language, or if i.e., the backwards compatibility concerns on allowing regexes on node ids would be prohibitive and it is best to experiment in a plugin for the foreseeable future?

[RonnyPfannschmidt]

Currently im -1 on making the expression language for -k and -m more complex

I'd rather see a new option for jq alike test selectors

[tylerjereddy]

Currently im -1 on making the expression language for -k and -m more complex

Maybe I was just naively motivated by the (admittedly very old) comment at #357 from @hpk42:

happy to review a PR which add an option to match tests via a regex on their id.

As for:

I'd rather see a new option for jq alike test selectors

Is that the JSON parser thing? I don't know much about that, I think I was hoping to maximize query power without learning a new syntax/tool, which I guess for me meant using something quite old/standard, like a regex.

So far, it sounds like I should indeed just develop the plugin then, which really does nothing other than compile a regular expression and filter node ids through it. For broader context, we were looking at useful plugins to try upstreaming/developing in scientific-python/summit-2023#10 for the summit next week, and I wanted to be sure that it would make sense to develop a plugin rather than say contribute directly upstream. It sounds like my intuition was correct.

[RonnyPfannschmidt]

The key reason why I am very wary of regex is, that node ids are actually something with structure that's tricky for regex

At the same time people want to query stuff like marks, parameter names/values

So a actual query language makes much more sense than just slapping regex at a part of the problem

[tylerjereddy]

Isn't a node id basically just: path/to/test_module.py::TestClass::test_name[parameter_value]?

I don't know about marks, but this seems to be pretty tractable with a regex parse, bounded pretty much only by the creativity of the user. I mean, I can see wanting to be user friendly and providing access to each level of the hierarchy (class/test/parameter) so you can query them individually without worrying about :: node components, but it also seems like pretty low-hanging fruit to just allow a regex to get you 90 % of the way there with an option. Certainly -k is not going to give you as much query power in its current state.

I also can't quite tell if you can really match the concision and power of regex with the JSON query approach.

[Zac-HD]

If we do add regex support, it should IMO be a new argument, not a change to the existing -k.

[Zac-HD]

Closing per #12465 (comment), since we can add that new argument in an external plugin 🙂