tests: Generate better error messages with 'assertEqual'

[rbasso]

When a test fails, most of the exercises just display the expected value, without stating what was evaluated:

pangram
  isPangram
    sentence empty
    pangram with only lower case FAILED [1]

Failures:

  test/Tests.hs:16: 
  1) pangram.isPangram pangram with only lower case
       expected: True
        but got: False

Sometimes the test case's description is enough to figure out what happened, but usually the user has to open the test suite to see the actual code being evaluated:

...
    test Case{..} = it description $ isPangram input `shouldBe` expected
...
        , Case { description = "pangram with only lower case"
               , input       = "the quick brown fox jumps over the lazy dog"
               , expected    = True
...

In this hypothetical case, the information needed by the student was:

expression: isPangram "the quick brown fox jumps over the lazy dog"
expected  : True
but got   : False

Writing the test suite with assertEqual instead of shoudBe, it is easy to get all this output directly from the failed test:

    test Case{..} = it description $
                       assertEqual ("isPangram " ++ show input)
                                   expected
                                   (isPangram input)

An now we get all the information when the test fails:

pangram
  isPangram
    sentence empty
    pangram with only lower case FAILED [1]

Failures:

  1) pangram.isPangram pangram with only lower case
       isPangram "the quick brown fox jumps over the lazy dog"
       expected: True
        but got: False

I believe that this is a small change that can make things easier for the students.

Should we do it to allow students avoid reading the test suite if they don't want to?

[abo64]

On one hand a coder's life consists of reading and understanding other people's code for most of the time.

But preparing for real life is probably not the primary goal here, but preserving people's interest and fun.
So I think it is a good idea.

[petertseng]

Will be interesting to see how this works for pov, minesweeper, etc. Seems like a good improvement.

I hope it will not result in too much repetition in the test code, even if it is a secondary concern to providing information.

[lpalma]

On one hand a coder's life consists of reading and understanding other people's code for most of the time.

:true-story:

I always leave the tests opened while solving an exercise. I take it as a supplement of the description, plus it helps me learning about testing in Haskell.
If assertEqual improves the console output, them I'm up for it 👍

since we are talking about Tests / Learning Experience, did Exercism ever considered students writing tests (for an already implemented code) instead of implementing the solution? I know it can be highly framework biased, but learning to test an essential skill.

[petertseng]

since we are talking about Tests / Learning Experience, did Exercism ever considered students writing tests (for an already implemented code) instead of implementing the solution

exercism/problem-specifications#80