test intents

[pytestbot]

Originally reported by: Ronny Pfannschmidt (BitBucket: RonnyPfannschmidt, GitHub: RonnyPfannschmidt)

---

to unify various things i'd like to propose a reporting on a level of n subreports per test

the concept for that is an intent within a test

one intent should be something groups a set of operations within a context,
and decides how grave what kind of error is within that context

examples could be things like creating a certain state, working down one item of a user story or gernator test items

== example 1 - generative tests

#!python
def pytest_funcarg_call(item):
    func = item._obj
    if isgenerator(func):
        with intent(name='setup', error='fatal'):
            iter = func()

        for check in iter:
           name, func, args = split_check(check, item)
           with intent(name=name, error='report'):
               func(*args)
    else:
      func()

== example 2 - full capuring of sections

#!python

def test_subprocess(tmpdir):
   tmpdir.ensure('test')
   with intent('call', capture='fd', errors='xfail') as report
      subprocess.call(['ls'],)

   assert report.out.fnmatch_lines(['test'])

---

• Bitbucket: https://bitbucket.org/pytest-dev/pytest/issue/153

[pytestbot]

Original comment by holger krekel (BitBucket: hpk42, GitHub: hpk42):

---

This more looks like an implementation idea - i have a guess what actual use cases might be like but they are not spelled out here.

[nicoddemus]

Could you please post more details here @RonnyPfannschmidt? 😄

[RonnyPfannschmidt]

bascially the equivalent of https://docs.python.org/3/library/unittest.html#distinguishing-test-iterations-using-subtests

with an addition of turning tests into sections for acceptance tests

[nicoddemus]

Hmmm I see, thanks!

That seems doable, could be even be implemented as a plugin I think.

[nicoddemus]

How about?

def test_foo(subtests):
    for i in xrange(5):
        with subtests:
            assert i * 2 < 5 

[RonnyPfannschmidt]

A bit too minimal

Names, retrying and flakyness should be handleable as well

[RonnyPfannschmidt]

closing as unactionable wishlist item, wil lcome back with something more concrete