stackProject: builtins.fetchGit to avoid manual sha256s and allow private repos

[roberth]

It seems like stackProject could support git packages without the user messing about with hashes. A nix expression could read the stack.yaml and prefetch all git repos using the evaluation-time builtins.fetchGit. This function only needs a rev and supports private repos.

[domenkozar]

Should be just a matter of changing the call at https://github.com/input-output-hk/nix-tools/blob/47ab47199194990681d24afd725e3f0e942e4f3d/lib/Cabal2Nix.hs#L223

[purefn]

This alone wouldn't enable stackProject to work with git repositories, private or otherwise, unless I'm missing something. It would be fine for running stack-to-nix outside of a nix expression and then building nix expressions that can fetch from private git repos, but as part of a derivation stackProject wouldn't be able to access the network and so wouldn't be able to do any fetching.

Again, that is unless I am missing something. If I am, please tell me as I would love for this to work!

[purefn]

Oh nevermind, I see this is intended to work with the cache parameter. Ignore me.

[angerman]

@roberth using fetchGit doesn’t work in restricted/pure mode iirc. My memory is a bit hazy here, but what I believe to recall is that fetchGit did not work on our hydra and hence we needed to make sure everything ends up being fixed output derivations.

[considerate]

Can anyone give some insight into why I'm unable to override the src attribute of a package when using stackProject but I'm able to when manually generating the .nix files with stack-to-nix and then using mkStackPkgSet?

Is this because the src attribute is evaluated in importAndFilterProject? I can't verify for sure but it seems like the git clone is started already when the projectNix is imported on this line

haskell.nix/lib/import-and-filter-project.nix

Line 7 in 8e248b3

project = import "${projectNix}";

[roberth]

@angerman fetchGit has specific behavior for restricted and pure mode. Maybe this was added recently? It works:

$ nix-build --option allowed-uris 'https://github.com/hercules-ci' --option restrict-eval true --option pure-eval true --expr 'builtins.fetchGit { url = "https://github.com/hercules-ci/warp-systemd.git"; rev = "49b17630eceb59febd95cc56ab6c588e68e163be"; }'
[ no error and it was fetched but not printend because it's a path; not a derivation ]

It won't work if the stack.yaml doesn't have a sha for it of course, but that's not desirable anyway.

So if I get it right, it should be a matter of extending the cache format to allow an optional path to the source at that version; no need to fetch it if you already have it. Or perhaps just change the stack yaml to use local paths. Or maybe something else. You're better at estimating which is better.

[domenkozar]

Don't think this was fixed :)

[angerman]

🤣 GitHub failed at parsing the fixes line.

[michaelpj]

It looks like fetchGit in restricted evaluation mode works without a sha, but only if the URI is in allowed-uris. That is, it doesn't get through the restricted mode loophole for fixed-output derivations.

In practice, this means you still need the sha if you're building on hydra (since adding to allowed-uris isn't trivial).

(This always bites me: I keep being surprised that fetchGit without a sha works in pure mode but not restricted mode...)

[roberth]

@michaelpj You can work around it with haskell.nix's nix-sha256: <hash> comments. I don't think there's anything else haskell.nix can do.

[michaelpj]

Yes, but this issue is about avoiding those hashes :)

I agree there's nothing haskell.nix can do. This seems like a nix bug to me if anything.

[michaelpj]

I finally complained about it: NixOS/nix#3469