Skip to content

Commit 1d91ceb

Browse files
committed
travis: Build all stubs, and test all stubs except for untestable ones
In #421 we express a desire that all our stubs should compile. Not all of them can successfully run their tests, so we will allow a file that says not to run the tests. This file will be stored in the .meta directory. Support for this was added in Trackler in exercism/DEPRECATED.trackler#4 - in particular, it will not be served to students. Regardless, all stubs should build, ensuring that we have not made a syntax error! If we desire, we can evaluate from time to time whether it is legitimate for the exercises declaring DONT-TEST-STUB to remain that way. We don't use `--pedantic` since there are many cases where, say, a `Dummy` constructor is not used or a function argument is not used. Closes #421
1 parent 3622396 commit 1d91ceb

File tree

9 files changed

+38
-1
lines changed

9 files changed

+38
-1
lines changed

.travis.yml

Lines changed: 21 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -53,6 +53,27 @@ script:
5353
exercisename=$(basename "$exercise")
5454
pushd ${exercise}
5555
56+
examplename="stub"
57+
buildfolder="${TRAVIS_BUILD_DIR}/build/${exercisename}/${examplename}"
58+
mkdir -p "${buildfolder}"
59+
cp -rL stack.yaml package.yaml src test "${buildfolder}"
60+
61+
pushd $buildfolder
62+
63+
examplecache="${HOME}/.foldercache/${exercisename}/${examplename}/.stack-work"
64+
mkdir -p "$examplecache"
65+
ln -f -s "$examplecache"
66+
67+
if [ -f "${exercise}/.meta/DONT-TEST-STUB" ]; then
68+
echo "only building stub"
69+
stack build ${SET_RESOLVER} --install-ghc --no-terminal
70+
else
71+
echo "testing stub"
72+
stack test ${SET_RESOLVER} --install-ghc --no-terminal --no-run-tests
73+
fi
74+
75+
popd
76+
5677
if ! stat -t examples/*/ > /dev/null 2>&1; then
5778
echo "No examples for ${exercise}!"
5879
exit 1

README.md

Lines changed: 7 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -116,7 +116,13 @@ To fix a bug you should [create a pull request from a fork](https://help.github.
116116
You should have [Stack](http://docs.haskellstack.org/) installed in your system to make contributing to this repository easier.
117117

118118
### Stub solution
119-
The stub solution must compile together with the test suite. It should be as general as possible in order to not exclude any possible solutions. It should take Haskell specifics into account (for example use `Maybe` instead of a dummy return value). It should not contain any comments (people might forget to remove them), you can use the hints file instead.
119+
The stub solution should be as general as possible in order to not exclude any possible solutions. It should take Haskell specifics into account (for example use `Maybe` instead of a dummy return value). It should not contain any comments (people might forget to remove them), you can use the hints file instead.
120+
121+
The stub solution must compile by itself (with `stack build`).
122+
Ideally, it would also compile together with the test suite (with `stack test --no-run-tests`).
123+
These two conditions are enforced by Travis.
124+
If the second condition cannot be met for a good reason, place the explanation in `.meta/DONT-TEST-STUB` to circumvent the check.
125+
The first condition is always enforced and cannot be circumvented.
120126

121127
### Example solution
122128
The example solution could be inspiration for other language implementors. It doesn't need to be perfect or very elegant. But it should be efficient enough for the test suite to finish in only a few seconds.
Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1 @@
1+
We would have to export all the allergens in the stub; let students figure that out.

exercises/clock/.meta/DONT-TEST-STUB

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1 @@
1+
We ask for the Clock to be an instance of Num; let students figure this out.
Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,2 @@
1+
No type signatures on functions (TODO: Should we have them?),
2+
and this makes it impossible to ensure that the result of `grade` is `Eq`.

exercises/meetup/.meta/DONT-TEST-STUB

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,2 @@
1+
Can't build because the parameters of meetupDay are unused
2+
Would also have to export all the individual days of the week in stub; let students figure that out.
Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,2 @@
1+
No function signatures, since we give students the freedom to define mkTriplet how they want
2+
But this means we can't make sure that the result of `mkTriplet` is `Eq`
Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1 @@
1+
No signature, so students can figure out accepting strings and ints.
Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1 @@
1+
We would have to export all the planets in the stub; let students figure that out.

0 commit comments

Comments
 (0)