Rework golden tests #711
Rework golden tests #711
Conversation
jorisdral
requested review from
dcoutts,
mheinzel,
recursion-ninja and
wenkokke
as code owners
| instance EnumGolden SnapshotMetaData where | ||
| singGolden = SnapshotMetaData singGolden singGolden singGolden singGolden singGolden |
There was a problem hiding this comment.
This pattern of applying a constructor to a string of singGoldens looks like it could be generically derived. Maybe I should add a TODO
| nameGolden :: Typeable a => Proxy a -> Annotation -> String | ||
| nameGolden p ann = map spaceToUnderscore (show $ typeRep p) ++ "." ++ ann |
There was a problem hiding this comment.
The downside of using typeRep is that the name of the golden file/test changes when the type name changes. In some cases, we might want to rename the type without affecting the name, so maybe nameGolden should be a member of EnumGolden?
There was a problem hiding this comment.
The typeRep approach can still be the default, but the upside is that you can change the name per instance
There was a problem hiding this comment.
Would this be an issue? Seems easier to just recreate the relevant golden files than to maintain a type-name mapping forever.
There was a problem hiding this comment.
Maybe not, it's just something that came to mind. I'm fine with leaving it as is too
Previously, we were encoding each snapshot type in the context of a full metadata type. However, if one field somewhere down in the snapshot metadata changes, then the encoding of the full metadata stays the same except for that one field further down. As such, this commit changes the test setup so that we now encode (and golden-test) each type involved in the snapshot metadata separately. We also add a new `EnumGolden` class for enumerating values of snapshot types. With the more direct tests and `EnumGolden`, it's easier to prevent combinatorial explosion.
Example output when there is an unexpected file:
```
❯ cabal run lsm-tree-test -- -p "prop_noUnexpectedOrMissingGoldenFiles"
lsm-tree
Test.Database.LSMTree.Internal.Snapshot.Codec.Golden
prop_noUnexpectedOrMissingGoldenFiles: FAIL
*** Failed! Falsified (after 1 test):
Found unexpected files: fromList ["unexpected"]
Delete the unexpected files manually from test/golden-file-data/snapshot-codec
Use --quickcheck-replay="(SMGen 1489992322676650058 4671765810150648345,0)" to reproduce.
1 out of 1 tests failed (0.01s)
```
Example output when there is a missing file:
```
❯ cabal run lsm-tree-test -- -p "prop_noUnexpectedOrMissingGoldenFiles"
lsm-tree
Test.Database.LSMTree.Internal.Snapshot.Codec.Golden
prop_noUnexpectedOrMissingGoldenFiles: FAIL
*** Failed! Falsified (after 1 test):
Missing expected files: fromList ["NominalCredits.A.snapshot.golden"]
Run the golden tests to regenerate the missing files
Use --quickcheck-replay="(SMGen 11771750078239108519 7040896247146249591,0)" to reproduce.
1 out of 1 tests failed (0.01s)
```
jorisdral
force-pushed
the
jdral/rework-golden-combinatorial-explosion
branch
from
fe14e01 to
66d80ec
Compare
Previously, we were encoding each snapshot type in the context of a full
metadata type. However, if one field somewhere down in the snapshot metadata
changes, then the encoding of the full metadata stays the same except for that
one field further down. As such, this commit changes the test setup so that we
now encode (and golden-test) each type involved in the snapshot metadata
separately.
We also add a new
EnumGoldenclass for enumerating values of snapshot types.With the more direct tests and
EnumGolden, it's easier to preventcombinatorial explosion.
There is now also a new test that checks whether there are no missing or unexpected files in the golden data directory.
Example output when there is an unexpected file:
Example output when there is a missing file: