Skip to content

Alex and Happy in more recent versions start to break #31

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
angerman opened this issue Jan 23, 2019 · 18 comments
Closed

Alex and Happy in more recent versions start to break #31

angerman opened this issue Jan 23, 2019 · 18 comments
Labels

Comments

@angerman
Copy link
Collaborator

Specifically we see messages like:

alex: /nix/store/iq8y91pg75x1j3zb0mchpkxh82y97awh-alex-3.2.4-exe-alex/share/x86_64-linux-ghc-8.4.4/alex-3.2.4-4b3blgwsCz88ZBWTyKoMOl-alex/AlexWrapper-monadUserState-bytestring: openFile: does not exist (No such file or directory)

There seems to be some mismatch between paths nix generates and cabal expects.

@angerman
Copy link
Collaborator Author

This should be fixed with input-output-hk/iohk-nix#30 which includes the patch to haskell/cabal#5862.

@ocharles
Copy link
Contributor

ocharles commented Jul 14, 2019

https://github.com/simonmar/alex/blob/75d0ab55f6b056049c8d80e47c8b93674a704128/src/Main.hs#L149 makes me think it is that Cabal bug - but I wonder if alex isn't getting rebuilt or something as it's a boot package? I don't think alex comes with GHC, so really not sure what's up!

@ocharles
Copy link
Contributor

Ok, I think the problem is changing packages.Cabal.patches has no affect on the result of hsPkgs.alex.components.exes.alex.

If I remove my packages.Cabal.patches override, I get exactly the same derivation for hsPkgs.alex.components.exes.alex. Furthermore, if I add more patches, the derivation still doesn't change. So it looks like that overriding doesn't go deep enough.

@ocharles
Copy link
Contributor

A bit more debugging - I think anything with build-type: Simple doesn't see the overriden Cabal, because no custom cabal.config is produced, and thus the Cabal used is the one bundled with ghc (which is borked).

@ocharles
Copy link
Contributor

diff --git a/builder/default.nix b/builder/default.nix
index e231cab..1009ef5 100644
--- a/builder/default.nix
+++ b/builder/default.nix
@@ -57,7 +57,7 @@ in {
   # Build a Haskell package from its config.
   # TODO: this pkgs is the adjusted pkgs, but pkgs.pkgs is unadjusted
   build-package = haskellLib.weakCallPackage pkgs ./hspkg-builder.nix {
-    inherit haskellLib ghc buildGHC comp-builder setup-builder;
+    inherit haskellLib ghc buildGHC comp-builder setup-builder makeSetupConfigFiles;
   };
 
   inherit shellFor;
diff --git a/builder/hspkg-builder.nix b/builder/hspkg-builder.nix
index 922902e..5c169af 100644
--- a/builder/hspkg-builder.nix
+++ b/builder/hspkg-builder.nix
@@ -1,4 +1,4 @@
-{ pkgs, buildPackages, stdenv, lib, haskellLib, ghc, buildGHC, fetchurl, runCommand, comp-builder, setup-builder }:
+{ pkgs, buildPackages, stdenv, lib, haskellLib, ghc, buildGHC, fetchurl, runCommand, comp-builder, setup-builder, makeSetupConfigFiles }:
 
 
 { flags
@@ -12,6 +12,7 @@
 , revision
 , revisionSha256
 , patches
+, hardeningDisable
 
 , shellHook
 
@@ -30,22 +31,40 @@ let
     import Distribution.Simple
     main = defaultMain
   '';
-  defaultSetup = buildPackages.runCommand "default-Setup" { nativeBuildInputs = [buildGHC]; } ''
-    cat ${defaultSetupSrc} > Setup.hs
-    mkdir -p $out/bin
-    ${buildGHC.targetPrefix}ghc Setup.hs --make -o $out/bin/Setup
-  '';
+  defaultConfig = makeSetupConfigFiles {
+    inherit (package) identifier;
+    inherit (config) flags;
+    fullName = "${name}-setup";
+    component = {
+      depends = [];
+      libs = [];
+      frameworks = [];
+      doExactConfig = false;
+    };
+  };
+  defaultSetup =
+    buildPackages.runCommand
+      "default-Setup"
+      { nativeBuildInputs = [buildGHC];
+        CABAL_CONFIG = defaultConfig + /cabal.config;
+      }
+      ''
+        cat ${defaultSetupSrc} > Setup.hs
+        mkdir -p $out/bin
+        ${buildGHC.targetPrefix}ghc Setup.hs --make -o $out/bin/Setup
+      '';
 
   setup = if package.buildType == "Simple"
     then defaultSetup
     else setup-builder {
       setup-depends = package.setup-depends;
       inherit package name src flags;
+      postUnpack = config.postUnpack;
     };
 
   buildComp = componentId: component: comp-builder {
     inherit componentId component package name src flags setup cabalFile cabal-generator patches revision
-            shellHook
+            shellHook hardeningDisable
             ;
   };

Appears to fix this. Will test more and open a PR if so.

@angerman
Copy link
Collaborator Author

angerman commented Jul 14, 2019

Great analysis @ocharles!

thus the Cabal used is the one bundled with ghc (which is borked).

Oh! yes of course, that makes sense! Guess we'll have to always build
against a non-stock lib:Cabal. Ignoring most of what ghc ships with
seems to be the best options sometimes :(

@ocharles
Copy link
Contributor

What I've done so far is taken the above patch, and only for alex added:

alex.package.setup-depends = [ config.hsPkgs.buildPackages.Cabal ];

so at least only alex has this special alternative. But anything that uses Build-Type: simple and cabal data directories is going to have this problem, so we probably do need to all packages to be configured with the reinstallable Cabal, to work around this.

ttuegel added a commit to ttuegel/haskell.nix that referenced this issue Jul 29, 2019
@ttuegel
Copy link
Contributor

ttuegel commented Jul 29, 2019

@ocharles I'm unable to evaluate with your patch:

error: anonymous function at /nix/store/6yn95c21hd3k1a88xm4jx156x96246gw-source/builder/hspkg-builder.nix:4:1 called without required argument 'hardeningDisable', at /nix/store/6yn95c21hd3k1a88xm4jx156x96246gw-source/modules/component-driver.nix:56:49

This issue has been open for some time; is there a plan to make this work? Being unable to use any package that depends on Happy or Alex is a pretty serious blocker...

@ocharles
Copy link
Contributor

@ttuegel I'll try and dig my experimental work out and see what PRs I can submit. I do have a few local fixes.

@considerate
Copy link
Contributor

To add to this issue: I'm running into a very similar problem when trying to compile the pretty-show package. The path to the happy executable is incorrect. I get the following error:

happy: /nix/store/spkhdfzan9jv2nqdbka2smsadn0dkwjb-happy-1.19.11-exe-happy/share/x86_64-linux-ghc-8.6.5/happy-1.19.11-KZRTzlNxnMbFxOwKYvlW3t-happy/HappyTemplate-arrays-coerce: openFile: does not exist (No such file or directory)

However, the binary is located at /nix/store/spkhdfzan9jv2nqdbka2smsadn0dkwjb-happy-1.19.11-exe-happy/share/x86_64-linux-ghc-8.6.5/happy-1.19.11/HappyTemplate-arrays-coerce.

I applied the following changes to haskell.nix: 7797c05...considerate:master

and modified the modules to be

    modules = [
	{ packages.alex.package.setup-depends = [pkgSet.config.hsPkgs.Cabal]; }
	{ packages.pretty-show.package.setup-depends = [pkgSet.config.hsPkgs.Cabal]; }
    ];

Building alex seems to work correctly with the specified changes however.

@ocharles
Copy link
Contributor

ocharles commented Jul 31, 2019

I haven't tidied up my work into PRs yet, but the diff is master...circuithub:master. Perhaps try with the master branch of https://github.com/circuithub/haskell.nix and see if that works?

@considerate
Copy link
Contributor

@ocharles Thank you for preparing this diff. Unfortunately I'm running into the same issue with your work as well.

minimal.nix:

{ pkgs ? import <nixpkgs> {} }:

let
  haskell = import (builtins.fetchTarball https://github.com/circuithub/haskell.nix/archive/master.tar.gz) { inherit pkgs; };

  pkgSet = haskell.mkStackPkgSet {
    stack-pkgs = {
      extras = hackage:
        {
          packages = {
          };
        };
        resolver = "lts-13.26";
      };
    pkg-def-extras = [
      (hackage: {
        hsc2hs = hackage.hsc2hs."0.68.4".revisions.default;
      })
    ];
    modules = [
        { packages.pretty-show.package.setup-depends = [pkgSet.config.hsPkgs.Cabal]; }
    ];
  };
in
  pkgSet.config.hsPkgs

Running with $ nix-build minimal.nix -A pretty-show.components.library yields:

these derivations will be built:
  /nix/store/nydm5iv41j98pvkjjllfr1r8iixsak7c-pretty-show-1.9.5-lib-pretty-show.drv
building '/nix/store/nydm5iv41j98pvkjjllfr1r8iixsak7c-pretty-show-1.9.5-lib-pretty-show.drv'...
unpacking sources
unpacking source archive /nix/store/awplvl5acy8j8hvhqhn2iax2haw8hyxi-pretty-show-1.9.5.tar.gz
source root is pretty-show-1.9.5
setting SOURCE_DATE_EPOCH to timestamp 1000000000 of file pretty-show-1.9.5/Text/Show/Value.hs
patching sources
configuring
Configure flags:
--prefix=/nix/store/yps8rkr9bs8rbjcffxx2sav03vvi8wy0-pretty-show-1.9.5-lib-pretty-show lib:pretty-show --package-db=clear --package-db=/nix/store/bna6835vgm2gpbry34dskbzin1xp95q7-pretty-show-1.9.5-lib-pretty-show-config/package.conf.d --exact-configuration --dependency=haskell-lexer=haskell-lexer-1.0.2-H2COOgYCyCAEveADIbCypJ --dependency=text=text-1.2.3.1-GCSbsQArgWY8ErCWIXJbi2 --dependency=filepath=filepath-1.4.2.1-8vpM2IXSLfHBqDAUich3wV --dependency=rts=rts --dependency=ghc-heap=ghc-heap-8.6.5 --dependency=ghc-prim=ghc-prim-0.5.3 --dependency=integer-gmp=integer-gmp-1.0.2.0 --dependency=base=base-4.12.0.0 --dependency=deepseq=deepseq-1.4.4.0 --dependency=array=array-0.5.3.0 --dependency=ghc-boot-th=ghc-boot-th-8.6.5 --dependency=pretty=pretty-1.1.3.6 --dependency=template-haskell=template-haskell-2.14.0.0 --dependency=ghc=ghc-8.6.5 --with-ghc=ghc --with-ghc-pkg=ghc-pkg --with-hsc2hs=hsc2hs --with-gcc=cc --with-ld=ld --with-ar=ar --with-strip=strip --disable-executable-stripping --disable-library-stripping --disable-library-profiling --disable-executable-profiling --docdir=/nix/store/1k10x708yyplka6mhn734hjz1jd4lah8-pretty-show-1.9.5-lib-pretty-show-doc/share/doc/pretty-show --enable-split-sections
Configuring library for pretty-show-1.9.5..
Warning: The flag --disable-executable-profiling is deprecated. Please use
--disable-profiling instead.
building
Preprocessing library for pretty-show-1.9.5..
happy: /nix/store/rfgv0gzcxbi40mb6d6lkny3km57snq8w-happy-1.19.11-exe-happy/share/x86_64-linux-ghc-8.6.5/happy-1.19.11-KZRTzlNxnMbFxOwKYvlW3t-happy/HappyTemplate-arrays-coerce: openFile: does not exist (No such file or directory)
builder for '/nix/store/nydm5iv41j98pvkjjllfr1r8iixsak7c-pretty-show-1.9.5-lib-pretty-show.drv' failed with exit code 1
error: build of '/nix/store/nydm5iv41j98pvkjjllfr1r8iixsak7c-pretty-show-1.9.5-lib-pretty-show.drv' failed

@considerate
Copy link
Contributor

I realized that I needed to apply the patch to the Cabal package linked in the above cabal issue as well (haskell/cabal#5862).

After applying that patch, I can now build pretty-show. 😄

For reference if someone else runs into this issue this is now the updated version:

minimal.nix:

{ pkgs ? import <nixpkgs> {} }:

let
  haskell = import (builtins.fetchTarball https://github.com/circuithub/haskell.nix/archive/master.tar.gz) { inherit pkgs; };

  pkgSet = haskell.mkStackPkgSet {
    stack-pkgs = {
      extras = hackage:
	{
	  packages = {
	  };
	};
  	resolver = "lts-13.26";
      };
    pkg-def-extras = [
      (hackage: {
	hsc2hs = hackage.hsc2hs."0.68.4".revisions.default;
      })
    ];
    modules = [
	{ packages.Cabal.patches = [./cabal.patch]; }
	{ packages.happy.package.setup-depends = [pkgSet.config.hsPkgs.Cabal]; }
	{ packages.pretty-show.package.setup-depends = [pkgSet.config.hsPkgs.Cabal]; }
    ];
  };
in
  pkgSet.config.hsPkgs

cabal.patch:

diff --git a/Distribution/Simple/Build/PathsModule.hs b/Distribution/Simple/Build/PathsModule.hs
index f6a203874..2aa406f74 100644
--- a/Distribution/Simple/Build/PathsModule.hs
+++ b/Distribution/Simple/Build/PathsModule.hs
@@ -192,10 +192,14 @@ generate pkg_descr lbi clbi =
           bindir     = flat_bindir,
           libdir     = flat_libdir,
           dynlibdir  = flat_dynlibdir,
-          datadir    = flat_datadir,
           libexecdir = flat_libexecdir,
           sysconfdir = flat_sysconfdir
         } = absoluteComponentInstallDirs pkg_descr lbi cid NoCopyDest
+
+        InstallDirs {
+          datadir    = flat_datadir
+        } = absoluteInstallDirs pkg_descr lbi NoCopyDest
+
         InstallDirs {
           bindir     = flat_bindirrel,
           libdir     = flat_libdirrel,

Thank you for digging into this and solving this issue.

@ttuegel
Copy link
Contributor

ttuegel commented Aug 1, 2019

@considerate I was stumped about the Cabal patch, too. Thanks for posting your resolution! 🎉

@fendor
Copy link

fendor commented Aug 4, 2019

I am trying to package lambdabot with haskell.nix and got it to run.
Following Changes:

  • In pkgs.nix I set the happy version to 1.19.9
  • Apply the Cabal.patch
  • Add non-reinstallable packages due to dependency on ghc-boot, etc...: [ "ghc-boot" "binary" "bytestring" "filepath" "directory" "containers" "time" "unix" "Win32" "ghci" "hpc" "process" "terminfo" "transformers" "mtl" "parsec" "];
    • Note, following the error message I initially got after adding [ "ghc-boot" "binary" "bytestring" "filepath" "directory" "containers" "time" "unix" "Win32"] to non-reinstallable packages, I also added Cabal to non-reinstallable packages. Thus, the patch was not applied and the build error didnt change.

Final default.nix:

{ pkgs ? import <nixpkgs> {} }:
# default.nix
let
  # Import the Haskell.nix library,
  haskell = import (builtins.fetchTarball https://github.com/input-output-hk/haskell.nix/archive/master.tar.gz) { inherit pkgs; };

  # Import the file you will create in the stack-to-nix or cabal-to-nix step.
  my-pkgs = import ./pkgs.nix;

  # Cabal projects use this:
  pkgSet = haskell.mkCabalProjectPkgSet {
    plan-pkgs = my-pkgs;
    pkg-def-extras = [];
    modules = [
      { nonReinstallablePkgs =
         [ "ghc-boot" "binary" "bytestring" "filepath" "directory" "containers" "time" "unix" "Win32"
           "ghci" "hpc" "process" "terminfo" "transformers" "mtl" "parsec" "text"
         ];
      }

      { packages.Cabal.patches = [./Cabal.patch]; }
      { packages.happy.package.setup-depends = [pkgSet.config.hsPkgs.Cabal]; }
      { packages.haskell.src.exts.package.setup-depends = [pkgSet.config.hsPkgs.Cabal]; }
    ];
  };

in pkgSet.config.hsPkgs // { _config = pkgSet.config; }

Would be glad to document that somewhere, but I dont know what the best location would be.

enobayram added a commit to enobayram/haskell.nix that referenced this issue Oct 2, 2019
enobayram pushed a commit to enobayram/haskell.nix that referenced this issue Oct 2, 2019
@domenkozar
Copy link
Contributor

I don't think this is still needed.

@domenkozar
Copy link
Contributor

domenkozar commented Dec 12, 2020

This was upstreamed in Cabal 3.0.0.0 and GHC 8.8.4 shiped Cabal 3.0.1.0

andreabedini pushed a commit to andreabedini/haskell.nix that referenced this issue Sep 14, 2022
andreabedini pushed a commit to andreabedini/haskell.nix that referenced this issue Sep 14, 2022
@stale
Copy link

stale bot commented Sep 28, 2022

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.

@stale stale bot added the wontfix label Sep 28, 2022
@stale stale bot closed this as completed Nov 27, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

No branches or pull requests

6 participants