-
Notifications
You must be signed in to change notification settings - Fork 99
Add support for Nix flakes and haskell.nix #1570
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Merged
Merged
Changes from all commits
Commits
Show all changes
56 commits
Select commit
Hold shift + click to select a range
a3a8d73
Initial version of flake.nix
jwiegley b87fe81
Further work on flake.nix
jwiegley ea46017
Update flake.nix and cabal.project
jwiegley 3bd2bdc
Add old Nix files back, they don't conflict
jwiegley c8089b0
Merge branch 'master' into johnw/nix-flake
jwiegley 7292803
Merge branch 'master' into johnw/nix-flake
jwiegley 5d9695e
Merge branch 'master' into johnw/nix-flake
jwiegley 5161f61
Update the cabal.project hashes
enobayram 75030b2
Align default.nix with flake.nix
enobayram 3d154e6
Merge branch 'master' into johnw/nix-flake
enobayram bb5b8ab
Avoid the haddock error from http2
enobayram 0720e79
Merge branch 'master' into johnw/nix-flake
jwiegley 1dd99bd
Pin explicit commit of nixpkgs
enobayram 024569f
Add a GitHub action for cached nix builds
enobayram 86855e2
Point "on push" branch to master
enobayram 3db800f
Run on all new commits
enobayram 753fadd
Convert tabs to spaces in default.nix
enobayram 4f2cce7
Rename Nix github action
enobayram 6f6e3b7
Add a workflow_dispatch trigger
enobayram ce0beee
Try a different way to set the trigger
enobayram 726e663
Revert "Try a different way to set the trigger"
enobayram 921085d
Make sure default.nix sees same src as flake.nix
enobayram f1c0491
Update nix.yml
DevopsGoth e96160e
Update nix.yml
DevopsGoth 33c6ac0
Update nix.yml
DevopsGoth e9646b3
increase timeout for nix build
DevopsGoth 4922da9
Fix the nix develop invocation
enobayram 4539616
Update action versions to avoid warnings
enobayram e559343
Merge remote-tracking branch 'origin/master' into johnw/nix-flake
enobayram 3fb5746
Avoid dowloading the entire nix develop closure
enobayram c27c482
Avoid depending on the chainweb-node closure too
enobayram ef3de89
Filter out irrelevant folders in project' src
enobayram 1112bab
Fix Nix issues
enobayram 0dad325
Clean up Nix files
enobayram 5865020
Prefetch haskell.nix IFD closure
enobayram 78801a1
Revert "Prefetch haskell.nix IFD closure"
enobayram 54961bb
Revert "Clean up Nix files"
enobayram f681a83
Rename default.nix to default-flake.nix
enobayram 6d6d927
Restore the old default.nix
enobayram fbdbb33
Add note to cabal.project on how to determine the sha values
jwiegley e170492
Convert cabal.project sha256s to base32
enobayram 1a00b59
Check the cabal.project hashes during nix build
enobayram 4712fde
Move the cabal.project check outside of build
enobayram eca7552
Separate the executable and devShell checks
enobayram 4281ac9
Add jq to check-cabal-project PATH
enobayram 8f5d288
Move check-cabal-project to default-flake.nix
enobayram eb31863
Fake commit to test cabal.project check
enobayram c5d4d12
Revert "Fake commit to test cabal.project check"
enobayram ef7e468
Expose the haskell source folder derivation
enobayram 043c25a
Remove the cabal.project check
enobayram 6e124e4
Revert "Convert cabal.project sha256s to base32"
enobayram 2666565
Fake commit to test outdated sha256
enobayram 1921ca2
Revert "Fake commit to test outdated sha256"
enobayram ae32d93
Fix and improve the source filtering
enobayram 9a2cd1c
Fake commit to test source filtering
enobayram d9634a6
Revert "Fake commit to test source filtering"
enobayram File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -1,4 +1,31 @@ | ||
name: Build and cache with Nix | ||
|
||
on: | ||
workflow_dispatch: | ||
workflow_dispatch: | ||
push: | ||
|
||
jobs: | ||
build-and-cache: | ||
runs-on: ubuntu-latest | ||
steps: | ||
- name: Checkout repository | ||
uses: actions/checkout@v3 | ||
|
||
- name: Set up Nix with caching | ||
uses: kadena-io/setup-nix-with-cache@v1 | ||
with: | ||
cache_url: s3://nixcache.chainweb.com?region=us-east-1 | ||
signing_private_key: ${{ secrets.NIX_CACHE_PRIVATE_KEY }} | ||
|
||
- name: Set up AWS credentials | ||
uses: aws-actions/configure-aws-credentials@v2 | ||
with: | ||
aws-access-key-id: ${{ secrets.NIX_CACHE_AWS_ACCESS_KEY_ID }} | ||
aws-secret-access-key: ${{ secrets.NIX_CACHE_AWS_SECRET_ACCESS_KEY }} | ||
aws-region: us-east-1 | ||
|
||
- name: Build and cache artifacts | ||
timeout-minutes: 740 | ||
run: | | ||
echo Building the project and its devShell | ||
nix build .#check |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,50 @@ | ||
let flakeDefaultNix = (import ( | ||
fetchTarball { | ||
url = "https://github.com/edolstra/flake-compat/archive/35bb57c0c8d8b62bbfd284272c928ceb64ddbde9.tar.gz"; | ||
sha256 = "1prd9b1xx8c0sfwnyzkspplh30m613j42l1k789s521f4kv4c2z2"; } | ||
larskuhtz marked this conversation as resolved.
Show resolved
Hide resolved
|
||
) { | ||
src = ./.; | ||
}).defaultNix; | ||
inputs = flakeDefaultNix.inputs; | ||
pkgsDef = import inputs.nixpkgs (import inputs.haskellNix {}).nixpkgsArgs; | ||
in | ||
{ pkgs ? pkgsDef | ||
, compiler ? "ghc8107" | ||
, flakePath ? flakeDefaultNix.outPath | ||
, nix-filter ? inputs.nix-filter | ||
, ... | ||
}: | ||
let haskellSrc = with nix-filter.lib; filter { | ||
root = flakePath; | ||
exclude = [ | ||
".github" | ||
".gitignore" | ||
".gitattributes" | ||
"docs" | ||
"examples" | ||
(matchExt "nix") | ||
"flake.lock" | ||
]; | ||
}; | ||
chainweb-node = pkgs.haskell-nix.project' { | ||
src = haskellSrc; | ||
compiler-nix-name = compiler; | ||
projectFileName = "cabal.project"; | ||
shell.tools = { | ||
cabal = {}; | ||
}; | ||
shell.buildInputs = with pkgs; [ | ||
zlib | ||
pkgconfig | ||
]; | ||
modules = [ | ||
{ | ||
packages.http2.doHaddock = false; | ||
} | ||
]; | ||
}; | ||
flake = chainweb-node.flake {}; | ||
default = flake.packages."chainweb:exe:chainweb-node"; | ||
in { | ||
inherit flake default haskellSrc; | ||
} |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
|
@@ -135,4 +135,4 @@ pkgs.haskell.packages.${compiler}.developPackage { | |
}); | ||
|
||
inherit returnShellEnv; | ||
} | ||
} |
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
How are those pins determined when updating the source
tag
for a package? Is that somewhere documented?In the past we would just wait for the zeus nix build to fail and copy and paste from the failure message? That wasn't ideal (because of the extra CI roundtrip) but worked. Will that method still work?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Is there any mechanism that guarantees that the source version represented by the
--sha
value matches the sources of the sourcetag
?In the past (with the old nix infra) we ran into issues where nix pins didn't exactly match the sources of the respective Hackage version and there was no easy way to detected that.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
The command to determine the sha locally is:
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Could you add that as a comment to the cabal.project file? So when people update that file they find that info?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Could we include an action in the CI workflow that runs that command to double checks that the nix
--sha
actually matches the sourcetag
?(You may take a look at cabal CI workflows; there are examples for actions that check that the git tree is still clean after performing some action or test.)
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
@larskuhtz Earlier today, I've added a step to check the hashes during the github action build, but then later I realized that the check was redundant, because it doesn't seem to be possible to start the build with a wrong
sha256
anyway. I've tried multiple ways to fool it, like changing the source-repository-package commit hash without updating thesha256
, but the nix build step always failed quickly, telling the correctsha256
. As an example, check out this run: https://github.com/kadena-io/chainweb-node/actions/runs/4831581150/jobs/8609217888The build took 2 minutes to fail and the error message tells you how you need to update the hash:
I can imagine how the old nix infra running on Zeus could perform misleading builds, picking the old source folder from the nix store due to the outdated hash, but I don't think that kind of inconsistency is possible with this new setup.
That's why I've removed the separate
cabal.project
checks.There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Admittedly, the error message you get from the Nix build is not ideal because it doesn't tell you what exactly you need to do. So if somebody encounters this message out of the blue, it could be frustrating to try to debug it.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
We could add a hint about how to read this error to the comment that describes how to compute the sha.