cabal install --enable-tests does not run test suite?

[snoyberg]

$ cabal --version
cabal-install version 1.20.0.3
using version 1.20.0.0 of the Cabal library 

When I have a project with a failing test suite (i.e., cabal test fails), cabal install --enable-tests still succeeds, and I see no evidence that the test suite was ever actually run.

By contrast, with:

$ cabal --version
cabal-install version 1.18.0.4
using version 1.18.1.3 of the Cabal library 

running cabal install --enable-tests fails as I would have expected.

[23Skidoo]

Try --run-tests.

[snoyberg]

That seems to work. Was this change in behavior mentioned in any release announcement? And what's the motivation behind it? I can't think of a good reason to build the test suite without running it, especially since simply building the test suite won't cause the executables to be installed anywhere.

If I could make a request for future such changes: it would be nice if cabal-install command line flags were treated somewhat more like an API, where silently changing behavior is not recommended, and instead a new flag with the modified behavior be provided instead. This switch did actually cause me quite a few problems, as I ended up installing versions of libraries in development that were not fully functional, assuming the test suites had already been run.

[23Skidoo]

Was this change in behavior mentioned in any release announcement?

Hmm, looks like Johan forgot to mention it. Sorry for that!

And what's the motivation behind it? I can't think of a good reason to build the test suite without running it, especially since simply building the test suite won't cause the executables to be installed anywhere.

See #1647.

[snoyberg]

Motivation understood. I'd still like to request that future changes avoid backwards-incompatible changes like this. When giving instructions to new users, it's very difficult to say "OK, if you're on version X of cabal, do A, but if you're on version Y, A actually does something else, so do B instead."