IFD-less evaluation of local packages

[srid]

IFD can really slow down the evaluation of the flake, such as to make the launching of nix develop noticeably slow -- this is currently the case for https://github.com/nammayatri/nammayatri which has gazillion local packages with lots of dependencies.

The idea here is to do something similar to the hpack -> cabal workflow, but for cabal -> nix. ie., use cabal2nix to generate a default.nix in the local package directory (example), for which we provide a nix run'able flake app that the developer will run anytime changing the .cabal file. There will also be a flake check (analogous to #97) to make sure that generated default.nix is in sync with the .cabal file (useful in CI). (pre-commit-hooks-nix can do this).

We could actually rename the default.nix to something unique to haskell-flake: haskell-package.nix.

Then, the following configuration:

{
  haskellProjects.default = {
    packages = {
      foo.source = ./foo;
    };
}

... would look for haskell-packages.nix and use that; if not, it looks for the .cabal file falling back to using IFD (callCabal2nix).

cc @roberth @locallycompact @NorfairKing for feedback/suggestions/ideas.

[srid]

Alternatively, is it possible to make callCabal2nix's drv re-evaluate only when the cabal file changes (less often), but not when the sources do (more often)?

[locallycompact]

The hpack -> cabal : cabal -> nix analog is how horizon-gen-nix works currently, although it only works by targetting remote urls, tarballs and hackage. Horizon did have FromLocal constructor at one point that could be used for overlays using local repositories.

https://gitlab.horizon-haskell.net/dhall/horizon-spec/-/blob/e7f19fc16cb5b47773bcca49fb1cab0d5c0011b9/horizon-spec/package.dhall#L23

But I got rid of it because I didn't have the bandwidth to think about how the pathing should work (pkgs/myPkg.nix should refer to ../something/myPkg or so) and also the workflow was rubbish without a way to check for staleness of the data. In the case of hpack staleness is obvious because the cabal file can be regenerated purely and cheaply, but in the case of cabal2nix over a gazillion projects - checking for staleness is just as expensive as the IFD it replaces. A lockfile would also confuse the matter here, where as a lockfile for urls, tarballs and hackage works fine because any change to those would dirty the lockfile and hence trigger a new cabal2nix regen for that item, where as with a FromLocal, the entry wouldn't change, so it would still require a call to cabal2nix everytime. So I think IFD is the only real option for local packages unfortunately.

Example of a horizon overlay style here: https://gitlab.horizon-haskell.net/package-sets/horizon-plutus-apps

[NorfairKing]

@srid I use https://github.com/cachix/pre-commit-hooks.nix to keep the default.nix up to date so that there is no IFD.

[srid]

Zulip discussion: https://nixos.zulipchat.com/#narrow/stream/413949-haskell-flake/topic/Removing.20IFD