Skip to content

travis: test both Stack and non-stack (Cabal) #174

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 2 commits into from
Jul 3, 2016
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
133 changes: 97 additions & 36 deletions .travis.yml
Original file line number Diff line number Diff line change
@@ -1,50 +1,111 @@
# The Travis docs (https://docs.travis-ci.com/user/languages/haskell)
# suggest that we use `language: haskell` and `ghc:` to specify GHC versions.
# However, as of this writing, Travis only supported versions up to 7.8.
# In addition, `language: haskell` doesn't support using Stack, which we use.
# If Travis supports all versions we would like to test, then we could switch.
# Until then, we use the https://github.com/hvr/multi-ghc-travis setup.
# Until then, we use a hybrid of the https://github.com/hvr/multi-ghc-travis and
# https://www.fpcomplete.com/blog/2016/02/updated-haskell-travis-config setups.
language: c
sudo: false
addons:
apt: &apt
sources:
- hvr-ghc
cabal: &cabal cabal-install-1.22
env:
global:
- CABALVER=1.22
sudo: false # Use the container-based infrastructure.

cache:
directories:
- $HOME/.ghc # For BUILD=cabal
- $HOME/.cabal # For BUILD=cabal
- $HOME/.stack # Global stack's cache.
- $HOME/.foldercache # Per exercise `.stack-work` cache.

matrix:
include:
- env: GHCVER=7.6.3
addons:
# These are builds using cabal-install to install packages.
# The primary motivation is GHC 7.6 support (Stack does not support 7.6).
# We would like to keep support of 7.6 since various Linux distros still use it:
# - Debian stable (jessie)
# - Ubuntu 14.04 LTS (trusty)
# - Fedora 20 and 21.
# 7.6 is the minimum version we can test since our test runner uses mkdtemp from System.Posix.Temp.
# This was only added in unix 2.6.0.0, which ships with GHC 7.6.x.
- env: BUILD=cabal CABALVER=1.16 GHCVER=7.6.3
addons: {apt: {packages: [cabal-install-1.16, ghc-7.6.3], sources: [hvr-ghc]}}
- env: BUILD=cabal CABALVER=1.18 GHCVER=7.8.4
addons: {apt: {packages: [cabal-install-1.18, ghc-7.8.4], sources: [hvr-ghc]}}
- env: BUILD=cabal CABALVER=1.22 GHCVER=7.10.3
addons: {apt: {packages: [cabal-install-1.22, ghc-7.10.3], sources: [hvr-ghc]}}
- env: BUILD=cabal CABALVER=1.24 GHCVER=8.0.1
addons: {apt: {packages: [cabal-install-1.24, ghc-8.0.1], sources: [hvr-ghc]}}


# These are builds using Stack to install packages.
# The advantages:
# - reproducible builds
# - verification of per-exercise dependencies.
- env: BUILD=stack RESOLVER=lts-2.22 # Last GHC 7.8.x snapshot.
addons: &stack-addons
apt:
<<: *apt
packages: [ghc-7.6.3, *cabal]
- env: GHCVER=7.8.4
packages:
- libgmp-dev # Stack's GHC depends on this.
- env: BUILD=stack RESOLVER=lts-6.4 # Last checked LTS snapshot (GHC 7.10.3).
addons:
apt:
<<: *apt
packages: [ghc-7.8.4, *cabal]
- env: GHCVER=7.10.3
<<: *stack-addons
- env: BUILD=stack RESOLVER=nightly-2016-06-21 # Last checked nightly snapshot (GHC 8.0.1).
addons:
apt:
<<: *apt
packages: [ghc-7.10.3, *cabal]
- env: GHCVER=8.0.1
<<: *stack-addons
- env: BUILD=stack RESOLVER=lts # Latest LTS snapshot.
addons:
apt:
<<: *apt
packages: [ghc-8.0.1, *cabal]
cache:
directories:
- '$HOME/.ghc'
- '$HOME/.cabal'
<<: *stack-addons
- env: BUILD=stack RESOLVER=nightly # Latest nightly snapshot.
addons:
<<: *stack-addons
allow_failures: # The snapshots `lts` and `nightly` are
- env: BUILD=stack RESOLVER=lts # aliases to the most recent versions.
- env: BUILD=stack RESOLVER=nightly # We don't want to fail when a new version
fast_finish: true # is released, so we list them here.

before_install:
- './_test/bootstrap.sh'
- export PATH="${TRAVIS_BUILD_DIR}/bin:$PATH" # For {,fetch-}configlet.
- |
case "$BUILD" in
stack)
mkdir -p ${HOME}/bin # Create folder for stack.
export PATH="${HOME}/bin:$PATH" # For stack
export PATH="${TRAVIS_BUILD_DIR}/_test:$PATH" # For stackalize.
;;
cabal)
export PATH=/opt/cabal/$CABALVER/bin:/opt/ghc/$GHCVER/bin:$PATH
./_test/bootstrap.sh
;;
esac

install:
- true
- travis_retry fetch-configlet
- |
set -e
if [ "$BUILD" = "stack" ]; then
travis_retry curl -L https://www.stackage.org/stack/linux-x86_64 -o pack.tgz
tar xzf pack.tgz --wildcards --strip-components=1 -C ${HOME}/bin '*/stack'
Copy link
Contributor

@rbasso rbasso Jul 3, 2016

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

tar runs even if fetch-configlet and curl -L fail?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It might be best to fix that in #162 but I can do it here too. Trying to decide whether I should amend it into 9b73dee (yours), have a separate one, or amend it into 14f272b (mine), but perhaps I split hairs here

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

ok, just added a commit on

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

In #162 I used distinct lines of install so that Travis automatically fails everything if a line fails. When you moved the commands inside the if Travis cannot check intermediary steps.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

You don't have to base this PR on #162.
The change are so significant that maybe it's better to just base it directly on master.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

aha! a very subtle distinction that I did not notice! Thanks for the catch, then.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

in that case I'll remove the intermediate commit

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Maybe I should use && rather than set -e here, hmm

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

OK, I'll not base it on #162 so that _bootstrap.sh doesn't get removed and then re-added, which was the main weird thing about basing this on #162.

fi

script:
- 'export PATH=/opt/cabal/$CABALVER/bin:/opt/ghc/$GHCVER/bin:$PATH'
- 'runhaskell -Wall ./_test/check-exercises.hs'
- './bin/fetch-configlet'
- './bin/configlet .'
- |
set -e
configlet .
case "$BUILD" in
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

In script: Travis has a special behavior. If a command fails, Travis will fail everything, but it will not shortcut evaluation, so every entry will be executed.

I used set -e and only one command in #162 to avoid that.

Copy link
Member Author

@petertseng petertseng Jul 3, 2016

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ah interesting. Would we like this to happen? Consider this case:

If I'm adding a new exercise and have forgotten to add it to config.json AND its tests are failing: Maybe I want to see both of those things. If we stop the build immediately on configlet failing, then I have to fix that first, and only after I do that will I find out my tests are failing too. Is that OK?

Then again, the argument may be "well maybe I should have run the tests locally to see whether they were passing" which... is valid. Maybe the tests only fail on an old version of GHC though?

I'm OK with either of these behaviors, but we should make the explicit decision.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I prefer to fail ASAP:

  • Most of the time we are waiting for Travis, so faster is better.
  • Usually the changes are small, so we have just one error.
  • Travis is just a final check, it should be tested locally.
  • It saves resources for other projects.
  • It marginally decreases global warming. 😬

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

OK, in that case I'll move configlet back into the single command, after set -e

stack)
for exercise in ${TRAVIS_BUILD_DIR}/exercises/* ; do
pushd ${exercise}
# `stack --word-dir` fails if not targeting a subdirectory, so we just
# symbolic-link `.stack-work` to a subfolder in the cache directory.
mkdir -p "${HOME}/.foldercache/${exercise}/.stack-work"
ln -f -s "${HOME}/.foldercache/${exercise}/.stack-work"
stackalize --resolver lts-0.0 `# This resolver is not used. `
stack test --resolver "${RESOLVER}" `# Select the correct resolver. `\
--install-ghc `# Download GHC if not in cache.`\
--no-terminal `# Terminal detection is broken.`\
--pedantic `# Enable -Wall and -Werror. `
popd
done
;;
cabal)
runhaskell -Wall ./_test/check-exercises.hs
;;
esac
6 changes: 0 additions & 6 deletions _test/bootstrap.sh
Original file line number Diff line number Diff line change
@@ -1,12 +1,6 @@
#!/bin/bash
# This installs a the prerequisites for a given GHC/Cabal config
set -e
if [ ! -z "$GHCVER" ]; then
export PATH=/opt/ghc/${GHCVER}/bin:$PATH
fi
if [ ! -z "$CABALVER" ]; then
export PATH=/opt/cabal/${CABALVER}/bin:$PATH
fi

Copy link
Member Author

@petertseng petertseng Jul 3, 2016

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

no longer needed as now handled by before_install. Can make it a separate commit if desired, though it will be a bit tricky as there will be one commit that has PATH being set in script/cabal, then one that moves it up to before_install/cabal

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I will try to completely rewrite bootstrap.sh to use dependencies if I get the time. In #162 I removed it but, if we want to keep tests with cabal, we need to at least to centralize the data in dependencies.txt.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Seems like a good idea.

# What directory is this script in?
DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
Expand Down