Make a pretty printer for errors

[michaelboulton]

Something like the default pytest one:

https://github.com/pytest-dev/pytest/blob/master/_pytest/terminal.py#L126

We want it to print out the yaml that caused the failure, the line, the file, etc. Colourised would be nice of course.

[michaelboulton]

Make it so each expected/request/verifier type has a log_fail which helpfully explains why it failed as well, or have some kind of pytest/user hook so it can do 'something' when it fails. At the moment it's essentially pointless

[ahopkins]

This would be awesome. The names are certainly a help, but I have taken to providing a sort of serial number in the name field so that I can easily identify where in my yaml the problem is. It is all manual right now, so it would be helpful if there was an easier way to track down the source of a failure.

Awesome work on this by the way, and thanks for open sourcing it!

[michaelboulton]

I started this with some help from the Pytest people: pytest-dev/pytest#3586

The real issue is trying to present the code in a nice format and showing which errors were caused at which point. This involves getting the yaml for the stage, probably post-format, which is currently a bit difficult.

It might be easier to generate a big output log file and then just parse it.

[michaelboulton]

Progress:

[michaelboulton]

@benhowes I was going to merge this into master and enable the 'new' reporting with a flag (eg --tavern-beta-rewrite-exceptions). I think the real 'big' step for this issue is to add metadata to exceptions so that we can highlight the line in the stage that failed for each error (in the case where there is multiple errors). Some note would go in the documentation about this, but also how it's a beta feature that might get changed in future.

[michaelboulton]

Also, print out the 'formatted' stage if possible alongside the raw input stage.

[benhowes]

@michaelboulton would it make sense for highlighting multiple errors to be a future feature? The current functionality is a big improvement on where we are at the moment, so it may make sense to get that out in a release. Printing the formatted stage alongside the raw one would be a big help for debugging too

[michaelboulton]

Highlighting multiple errors is definitely a fairly big thing so that can be left until a later release.

[michaelboulton]

This is now in 0.13.0 - use the --tavern-beta-new-traceback flag to enable it