Any way to override a Haskell package using Nix expressions?

[dhess]

With nixpkgs.haskellPackages, it's really easy to override a particular package(s) in the Haskell package set using "pure Nix"; that is, without needing to resort to using Cabal or Stackage tooling. For example, I often need to override a dependency (temporarily) while it's in development and hasn't yet been pushed out to Hackage.

Here's an example with nixpkgs.haskellPackges and Agda, where I wanted some features in the latest version (now 2.6.1) that weren't available on Hackage for nearly 9 months:

https://github.com/hackworthltd/hacknix/blob/e8f64c291a076b48569463af32b5ca6fd18e18a3/overlays/haskell-packages.nix#L31

One nice thing about this approach is that the overridden haskellPackages set can then be put in an overlay and shared by multiple downstream Nix projects, and that it can be built and cached by CI just like any other nixpkgs package.

With haskell.nix, I can do this very easily for a top-level package. For example, here's what I do to build ghcide from its GitHub repo:

https://github.com/hackworthltd/haskell-hacknix/blob/4efd4d58cf6d7c24e8cc7123fdc0fee650e8ba90/nix/pkgs/ghcide.nix#L22

The version pin is trivial to maintain using niv, just like all of my other pinned Nix dependencies.

That works great for ghcide because all I need from that package is the ghcide executable. But what if I wanted to use that version of ghcide as a library dependency in my local haskell.nix project? I can't figure out whether there's any existing way to do that. It looked like mkCacheModule was going to do that for me, but it appears that packages in a cabalProject's modules do not override the packages in the haskell.nix Hackage package set. In my brief testing, Cabal ignored a package override in my cabalProject.modules and used the one in haskell.nix's Hackage package set, instead.

Judging from a conversation on IRC, it sounds like haskell.nix-based projects are expected to pin things in their Cabal file using source-repository-package, and while I agree that this is nice because it means that the package override will be visible to users who aren't using the Nix tooling:

a) As these overrides are normally only temporary and aren't meant to be used by downstream consumers of the package, I don't particularly care that it's not visible to non-Nix users;

b) It sounds like the overridden package is built in the project's shellFor when you're interactively developing with cabal build etc., even if it's available in the Nix cache (I have not yet verified this myself);

c) If you're already using something like niv to pin dependencies in a Nix project, the Cabal pins will need to be maintained separately and will need to be manually updated; and

d) If this overridden package version is needed by multiple downstream haskell.nix projects, each one of those haskell.nix projects will need to specify that override in their Cabal file, they'll have to be manually kept in sync across projects, etc.

For these reasons, I think that being able to specify Haskell package overrides in Nix is preferable to the, "do it in Cabal" approach.

So: am I missing something? Is there is a way to override dependencies a la haskellPackages.override in nixpkgs, or would this need to be added to haskell.nix?

[dhess]

Also, I've just verified that with source-respository-packages of type git and using a revision as a tag, whenever you make a change to the project, nix-build will fetch each such repository every time, even if the pins haven't changed! That drastically slows down development time when you have 2 or 3 of those.

[dhess]

This approach also appears to have issues with Hydra. For the project I'm testing this with, I need the GitHub version of pandoc-sidenote. Here's the Hydra evaluation log:

hydra-eval-jobs returned exit code 1:
trace: Using latest index state for bpvx01bwg8bwz4nrlhpgzasdjgcx3ikx-272c3a4ab8abb8679151b7ac8e17e0bb09b60b4343125ac765295370f3eea2f2-94818cb!
trace: [bpvx01bwg8bwz4nrlhpgzasdjgcx3ikx-272c3a4ab8abb8679151b7ac8e17e0bb09b60b4343125ac765295370f3eea2f2-94818cb-plan-to-nix-pkgs] cabal new-configure --with-ghc=ghc --with-ghc-pkg=ghc-pkg
trace: Using index-state: 2020-03-23T00:00:00Z for bpvx01bwg8bwz4nrlhpgzasdjgcx3ikx-272c3a4ab8abb8679151b7ac8e17e0bb09b60b4343125ac765295370f3eea2f2-94818cb
trace: [alex-plan-to-nix-pkgs] cabal new-configure --with-ghc=ghc --with-ghc-pkg=ghc-pkg
trace: [happy-plan-to-nix-pkgs] cabal new-configure --with-ghc=ghc --with-ghc-pkg=ghc-pkg
trace: [hscolour-plan-to-nix-pkgs] cabal new-configure --with-ghc=ghc --with-ghc-pkg=ghc-pkg
building '/nix/store/v742qdpazn0xn0rc7c98i5cplpxnkxns-bpvx01bwg8bwz4nrlhpgzasdjgcx3ikx-272c3a4ab8abb8679151b7ac8e17e0bb09b60b4343125ac765295370f3eea2f2-94818cb-plan-to-nix-pkgs.drv' on 'ssh://remote-builder@builder'...
Cloning into '/build/tmp.FAKzhvLViq/dist-newstyle/src/pandoc-si_-d9b1006bd95a418'...
fatal: unable to access 'https://github.com/jez/pandoc-sidenote/': Could not resolve host: github.com
builder for '/nix/store/v742qdpazn0xn0rc7c98i5cplpxnkxns-bpvx01bwg8bwz4nrlhpgzasdjgcx3ikx-272c3a4ab8abb8679151b7ac8e17e0bb09b60b4343125ac765295370f3eea2f2-94818cb-plan-to-nix-pkgs.drv' failed with exit code 128
error: build of '/nix/store/v742qdpazn0xn0rc7c98i5cplpxnkxns-bpvx01bwg8bwz4nrlhpgzasdjgcx3ikx-272c3a4ab8abb8679151b7ac8e17e0bb09b60b4343125ac765295370f3eea2f2-94818cb-plan-to-nix-pkgs.drv' on 'ssh://remote-builder@builder' failed: builder for '/nix/store/v742qdpazn0xn0rc7c98i5cplpxnkxns-bpvx01bwg8bwz4nrlhpgzasdjgcx3ikx-272c3a4ab8abb8679151b7ac8e17e0bb09b60b4343125ac765295370f3eea2f2-94818cb-plan-to-nix-pkgs.drv' failed with exit code 128
builder for '/nix/store/v742qdpazn0xn0rc7c98i5cplpxnkxns-bpvx01bwg8bwz4nrlhpgzasdjgcx3ikx-272c3a4ab8abb8679151b7ac8e17e0bb09b60b4343125ac765295370f3eea2f2-94818cb-plan-to-nix-pkgs.drv' failed with exit code 1
error: build of '/nix/store/v742qdpazn0xn0rc7c98i5cplpxnkxns-bpvx01bwg8bwz4nrlhpgzasdjgcx3ikx-272c3a4ab8abb8679151b7ac8e17e0bb09b60b4343125ac765295370f3eea2f2-94818cb-plan-to-nix-pkgs.drv' failed

I assume this is due to restricted mode?

[michaelpj]

Ah yes, you need to use this unfortunate hack to make it into a fixed-output derivation.

I think this is a major wart. I remember thinking we should be able to use builtins.fetchGit to avoid this issue, but I can't remember what happened about it.

[michaelpj]

#309

[dhess]

OK, that seems to have fixed the Hydra evaluation. I'm tempted to switch to git submodules for these packages, though, as it would probably be less maintenance until/unless a haskell.nix provides a pure Nix solution.

[Rizary]

Hi, I have similar case, but in this case, my library located in different folder. I have this structure:

haskell-project
|--> backend
|----> backend.cabal
|--> common
|----> common.cabal

when building backend, cabal couldn't find common which is obvious. I thought that haskellPackages behave the same with nixpkgs' haskellPackages, so either I use extend or override as mentioned above.

I already try to add the common library into my overlays, and still doesn't work.

I wonder how to add my local package in haskell-nix. I usually do using nixpkgs.haskellPackages very often and so simple like:

pkgs.haskell.packages.ghc8102.extend (self: super: {
  common = self.callCabal2nix "common" ./common { };
  backend = self.callCabal2nix "backend" ./backend { };
})

with haskell-nix, I'm not sure where to add my common (which is not git repo).

[michaelpj]

I think the combination of doing things in cabal.project and the options in modules should mostly cover this, please open more specific issues if there are still problems.