Faster cabal builds by caching the build directory

[nh2]

Imported from reddit:

I got annoyed that every time I build something in a new cabal sandbox I have to wait for the same versions of lens or haskell-src-exts to compile forever.

Couldn't we just cache the build directory from which we installed a package instead of throwing it away after every build?

Cabal tries to guarantee that running cabal install in a source directory always produces the correct result without us having to run cabal clean first (so no matter what the state of the dist/ directory is). Cabal does this by using ghc --make for building, which has very good recompilation avoidance and flag change detection.

This means that this kind of cache introduces no additional correctness guarantees we would have to provide, it is only relying on guarantees that cabal already tries to provide.

The cache does take some space in my home directory, but I accept that for the 20x faster build it allows.

For example, a no-op rebuild of for haskell-src-exts is 7 seconds on my computer vs. a full build that takes 132 seconds.

This method will pick up changed dependencies (e.g. it will build correctly when the version of haskell-src-exts's dependency pretty changes between two builds) because ghc --make notices that.

This method will also provide for cases where other approaches cannot. For example, as mentioned here and here, it can cache more than a nix-like approach can: With nix caching ends completely at the first dependency that has changed, while ghc --make caches on the module level, recompiling only those modules that depend on changed code, not packages. Nevertheless, the biggest speedup to be expected is for CI systems like TravisCI, CircleCI etc., and sandboxes, as those compile share huge parts of the dependency graph where all code is exactly the same.

Here's a proof of concept patch: nh2/cabal@nh2:before-buildcache...buildcache

What do you think?

---

Some things we would need:

• Are you aware of any recent or past cabal bugs mentioning that cabal clean was necessary in order to build correctly?
• Also I think that if we want to try this out, the best might be to hide it behind a cabal install --cached option defaulting to False, and if it works out well, make it the default at some point in the future. Do you agree?

[nh2]

@23Skidoo The reason it doesn't work with sandboxes yet is this:

    -- 'cabal sdist' puts pre-generated files in the 'dist'
    -- directory. This fails when a nonstandard build directory name
    -- is used (as is the case with sandboxes), so we need to rename
    -- the 'dist' dir here.
    --
    -- TODO: 'cabal get happy && cd sandbox && cabal install ../happy' still
    -- fails even with this workaround. We probably can live with that.
    maybeRenameDistDir :: FilePath -> IO ()

I don't understand this yet - could you explain me a bit? Why does the placement of files created by cabal sdist fail when a nonstandard build directory name is used? Couldn't cabal sdist just put the files into whatever the build directory is?

[23Skidoo]

Haven't looked at the patch, but this sounds like an interesting idea. I agree that this feature should be optional, at least in the beginning.

Are you aware of any recent or past cabal bugs mentioning that cabal clean was necessary in order to build correctly?

No.

I don't understand this yet - could you explain me a bit? Why does the placement of files created by cabal sdist fail when a nonstandard build directory name is used? Couldn't cabal sdist just put the files into whatever the build directory is?

When using sandboxes, cabal install foo actually means cabal install foo --builddir dist-$hash(/path/to/sandbox). Not sure if install supports --builddir, but you get the idea. This was done to prevent add-source deps clashing with each other and the normal usage (#1281). See also #2106.

If you want to experiment with sandbox build dir caching, try disabling this feature by replacing sandboxBuildDir with id and removing maybeRenameDistDir.

[nh2]

@23Skidoo Thanks for your explanation.

So if I understand it right, in #2106 you suggest that we probably don't want to use dist/dist-$HASH for non-add-source dependencies.

In that case, we should probably treat #2106 as a blocker to this, since then my patch could stay as simple as it is and wouldn't have to deal with maybeRenameDistDir.

Haven't looked at the patch

Please do - it's just ~5 lines ;) :D

[ezyang]

This should be fixed by the nix-local-build branch, which introduces "projects" which have a shared (monotonically growing) global database and a per-project database for local sources / vendored things.

[dcoutts]

Yes, I think we can legitimately say that this is fixed by the new nix style builds. So I'll close this ticket. Yes it's currently still a tech preview, but we've got other tracker tickets so don't need to keep this one open.