./Setup configure eats memory, cabal configure does not

[nomeata]

Debian is observing a problem with git-annex, where using ./Setup configure would eat up all memory in the solver (judging from the heap profile, related to flag assignment), while cabal configure does not. This is Cabal 1.22.1.1 and cabal.-install 1.22.0.1.

I will add a link to a tarball that contains a home directory with a few packages installed (just the package config) and a cabal directory. You can test it this way:

/tmp/bug-home $ ghc --make Setup.hs -rtsopts && HOME=/tmp/bug-home/ ./Setup configure +RTS -M100M
[1 of 1] Compiling Main             ( Setup.hs, Setup.o )
Linking Setup ...
Configuring git-annex-5.20150812...
Setup: Heap exhausted;             
Current maximum heap size is 104857600 bytes (100 MB);
use `+RTS -M<size>' to increase it.
[jojo@kirk:3] Di, der 18.08.2015 um 11:35 Uhr Don't Panic! (251)
/tmp/bug-home $ HOME=/tmp/bug-home/ cabal configure
Warning: The package list for 'hackage.haskell.org' does not exist. Run 'cabal
update' to download it.
Resolving dependencies...
[1 of 1] Compiling Main             ( dist/setup/setup.hs, dist/setup/Main.o )
Linking ./dist/setup/setup ...
Configuring git-annex-5.20150812...
Warning: The 'license-file' field refers to the file 'COPYRIGHT' which does
not exist.
Warning: 'include-dirs: Utility' directory does not exist.

I’m surprised that cabal does better than Cabal here.

[nomeata]

https://people.debian.org/~nomeata/cabal-bug-2777.tar.gz

[ttuegel]

I’m surprised that cabal does better than Cabal here.

At first this is surprising because git-annex uses build-type: Custom, i.e. cabal-install must actually invoke ghc --make Setup.hs etc. My first guess is that cabal-install solves the flag assignment more efficiently because it runs through the dependency solver. I'll take a look.

[joeyh]

Looking at what cabal configure does, it finds a solution and passes it, including the --flags settings and --dependency versions to setup configure.

nomeata found earlier that telling setup configure the flag settings avoids this blowup.

BTW, I've also got a report of this on Fedora: http://git-annex.branchable.com/bugs/OOM_while_configuring_git-annex/

[RichiH]

@ttuegel Any luck with this?

[ttuegel]

Yes and no. I confirmed that the problem is as I expected:

My first guess is that cabal-install solves the flag assignment more efficiently because it runs through the dependency solver.

I haven't done anything toward fixing it. There is some work being done to move the solver to its own library, but I'd actually like to see it moved into the Cabal library so we could solve this problem more easily.

[RichiH]

@ttuegel Thanks; reading #2768, it seems there's not even a rough guesstimate of an ETA, correct?

[ttuegel]

@RichiH There hasn't been any movement on it recently, but if it doesn't start moving soon, I'll give it a push.

[RichiH]

@ttuegel Reminder-poke :)

[grayjay]

@ttuegel Does it make sense to make a smaller improvement to Cabal's algorithm before the solver is available? Cabal currently tries every combination of flags until it finds a solution. I changed the algorithm so that it backtracks as soon as one flag introduces a dependency that isn't available, and it seems to work in this case. Would you like me to work on a pull request?

[RichiH]

I would like to have the small version sooner rather than later if you guys are willing to do the work. Richard Sent by mobile; excuse my brevity.

[23Skidoo]

@grayraw Yes, I think your patch would be accepted.

[ttuegel]

@grayjay Thank you for working on that. Yes, please make a pull request. I've been overruled and the solver is never going to be part of Cabal.

[grayjay]

@ttuegel No problem. I created pull request #2925 , but I'm still trying to improve the error message that is generated without an exhaustive search.

[RichiH]

You could print pseudologic "Install ( foo || bar ) && quux && ( foo1 || foo2 || foo3) and try again." Richard Sent by mobile; excuse my brevity.

[grayjay]

@RichiH Thanks for the suggestion. I think that that message would become too long when all flag combinations lead to different sets of dependencies. I just updated my branch with an improvement over the previous message (#2925).

[23Skidoo]

Fixed in #3076.