Copying executables when installing is fragile with packages relying on the paths in Cabal's Paths_<pkg-name> module

[DanielG]

Since stack install is merely an alias for copying executables after building them this means that the Paths_ module doesn't get re-generated for the actual installation prefix but rather remains at pointing to somewhere in .stack-work. This is a real problems for programs or libraries that rely on these paths being correct.

cabal install does this correctly and rebuilds the whole package with the proper paths in this module.

My specific use case is cabal-helper where, in the library, I have to use libexecdir to find the wrapper executable (since I don't want that to show up on the user's PATH). Since there isn't such a thing as LIBEXECPATH I'm kind of stuck here as I can't think of another way to find my executables in libexec.

[snoyberg]

cabal install and stack are doing the same thing here: neither one is recompiling executables. You can see that, for example, by running cabal install inside a sandbox, which is essentially what stack install is doing. I'm having trouble understanding what the problem is here: are you saying that running the generated executable doesn't work, or that you don't like the paths it's using internally?

[DanielG]

Ok so if I do the following with cabal-install:

$ cabal configure --prefix=/tmp/prefix
$ cabal build -j

$ cabal install .

Then cabal install . will actually reconfigure and rebuild the project with prefix set to the default path i.e. ~/.cabal instead of /tmp/prefix.

The problem is that stack build initializes the prefix to somewhere in .stack-work and then doesn't change it to wherever the executables will actually end up on stack install.

[borsboom]

What would you suggest the preferred behaviour be?

[snoyberg]

And perhaps you can describe the problem you're actually trying to solve? I read through the linked issue, and couldn't understand what problem the libexecdir is trying to work around.

[DanielG]

Hmm it seems like this issue is more weird than I though. On second look the Paths_ module as generated by stack build does contain the "right" path to $libexec, namely .cabal/libexec but all the other paths (bindir, libdir, etc.) are pointing to ./.stack-work/....

If I add a plain stack.yaml to cabal-helper (https://github.com/DanielG/cabal-helper) and stack build inside that repo it does install the cabal-helper-wrapper executable into .cabal/libexec (which it probably shouldn't):

$ stack build
cabal-helper-0.5.2.0: configure  
Configuring cabal-helper-0.5.2.0...
cabal-helper-0.5.2.0: build
Building cabal-helper-0.5.2.0...
Preprocessing library cabal-helper-0.5.2.0...
[1 of 4] Compiling CabalHelper.Sandbox ( CabalHelper/Sandbox.hs, .stack-work/dist/x86_64-linux/Cabal-1.22.4.0/build/CabalHelper/Sandbox.o )
[2 of 4] Compiling CabalHelper.Types ( CabalHelper/Types.hs, .stack-work/dist/x86_64-linux/Cabal-1.22.4.0/build/CabalHelper/Types.o )
[3 of 4] Compiling Paths_cabal_helper ( .stack-work/dist/x86_64-linux/Cabal-1.22.4.0/build/autogen/Paths_cabal_helper.hs, .stack-work/dist/x86_64-linux/Cabal-1.22.4.0/build/Paths_cabal_helper.o )
[4 of 4] Compiling Distribution.Helper ( Distribution/Helper.hs, .stack-work/dist/x86_64-linux/Cabal-1.22.4.0/build/Distribution/Helper.o )
In-place registering cabal-helper-0.5.2.0...
Preprocessing executable 'cabal-helper-wrapper' for cabal-helper-0.5.2.0...
[1 of 9] Compiling CabalHelper.Data ( CabalHelper/Data.hs, .stack-work/dist/x86_64-linux/Cabal-1.22.4.0/build/cabal-helper-wrapper/cabal-helper-wrapper-tmp/CabalHelper/Data.o )
[2 of 9] Compiling CabalHelper.Sandbox ( CabalHelper/Sandbox.hs, .stack-work/dist/x86_64-linux/Cabal-1.22.4.0/build/cabal-helper-wrapper/cabal-helper-wrapper-tmp/CabalHelper/Sandbox.o )
[3 of 9] Compiling CabalHelper.Types ( CabalHelper/Types.hs, .stack-work/dist/x86_64-linux/Cabal-1.22.4.0/build/cabal-helper-wrapper/cabal-helper-wrapper-tmp/CabalHelper/Types.o )
[4 of 9] Compiling CabalHelper.Log  ( CabalHelper/Log.hs, .stack-work/dist/x86_64-linux/Cabal-1.22.4.0/build/cabal-helper-wrapper/cabal-helper-wrapper-tmp/CabalHelper/Log.o )
[5 of 9] Compiling CabalHelper.GuessGhc ( CabalHelper/GuessGhc.hs, .stack-work/dist/x86_64-linux/Cabal-1.22.4.0/build/cabal-helper-wrapper/cabal-helper-wrapper-tmp/CabalHelper/GuessGhc.o )
[6 of 9] Compiling CabalHelper.Common ( CabalHelper/Common.hs, .stack-work/dist/x86_64-linux/Cabal-1.22.4.0/build/cabal-helper-wrapper/cabal-helper-wrapper-tmp/CabalHelper/Common.o )
[7 of 9] Compiling Paths_cabal_helper ( .stack-work/dist/x86_64-linux/Cabal-1.22.4.0/build/autogen/Paths_cabal_helper.hs, .stack-work/dist/x86_64-linux/Cabal-1.22.4.0/build/cabal-helper-wrapper/cabal-helper-wrapper-tmp/Paths_cabal_helper.o )
[8 of 9] Compiling CabalHelper.Compile ( CabalHelper/Compile.hs, .stack-work/dist/x86_64-linux/Cabal-1.22.4.0/build/cabal-helper-wrapper/cabal-helper-wrapper-tmp/CabalHelper/Compile.o )
[9 of 9] Compiling Main             ( CabalHelper/Wrapper.hs, .stack-work/dist/x86_64-linux/Cabal-1.22.4.0/build/cabal-helper-wrapper/cabal-helper-wrapper-tmp/Main.o )
[7 of 9] Compiling CabalHelper.Data ( CabalHelper/Data.hs, .stack-work/dist/x86_64-linux/Cabal-1.22.4.0/build/cabal-helper-wrapper/cabal-helper-wrapper-tmp/CabalHelper/Data.o )
Linking .stack-work/dist/x86_64-linux/Cabal-1.22.4.0/build/cabal-helper-wrapper/cabal-helper-wrapper ...
cabal-helper-0.5.2.0: install
extended "cabal-helper-wrapper"
Installing executable(s) in /home/dxld/.cabal/libexec
Warning: The directory /home/dxld/.cabal/libexec is not in the system search
path.
Installing library in
/home/dxld/share/dev/hs/cabal-helper/.stack-work/install/x86_64-linux/lts-3.1/7.10.2/lib/x86_64-linux-ghc-7.10.2/cabal-helper-0.5.2.0-6OC4KX1HZFK46by8bcnSLv
Registering cabal-helper-0.5.2.0...

And now stack install will error out because cabal-helper-wrapper wasn't installed into where it wants it to be:

Couldn't find executable cabal-helper-wrapper in directory /home/dxld/share/dev/hs/cabal-helper/.stack-work/install/x86_64-linux/lts-3.1/7.10.2/bin/

I use a custom Setup.hs to implement the X-Install-Target field in my .cabal file https://github.com/DanielG/cabal-helper/blob/master/Setup.hs that overrides the inst and copy hooks so I'm not sure what can really be done about this other than adding Install-Target to Cabal proper :/

[DanielG]

I'm starting to think the initial bug report was bogus, if I install ghc-mod with stack locally everything just happens to end up in the right path.

I still believe the deviation from what cabal-install would do with bindir, libdir etc. is a bug though. How to solve that properly I have no idea. What I would do is 1) make stack install a separate command that generates the sdist, unpacks it into a temporary directory, configures the installation prefix for the Paths_ stuff to caba-install's defaults instead of the .stack-work stuff and then does the build like normal. Actually generating the sdist ensures that stack install is exactly equivalent to what downloading the package from hackage would do, I've run into forgetting to add some test data to Extra-Source-Files wayyyyy too often already ;)

stack install failing when Setup.hs is doing weird stuff with the installtion directories I can understand and so long as it doesn't actually make the installation fail it's fine I suppose.

[snoyberg]

I've fixed the libexecdir and sysconfdir not being set, thanks for reporting that.

[DanielG]

That's all nice and well but this makes the issue of what happens on stack install even more pressing because now the $libexec directory will disappear as soon as the user decides to delete the build directory...

[snoyberg]

I'm really missing the forest for the trees here, I don't have any idea why this problem exists in the first place. Why isn't cabal-helper just being installed into the bin directory like all other executables?

[snoyberg]

Alright, reviewing this again, I think I'm starting to understand what you're saying, and where the misunderstanding is starting. For future reference, it would be a lot easier to follow a bug report if it followed the recommended bug reporting guidelines.

Current situation: all paths in the Paths_* module refer to local directories inside of the stack-work directory. AFAICT, despite the implications in the initial bug report, this doesn't break anything: all executables are generated with paths that actually exist. Is this correct?

The concern that you have is that, when copying an executable from .stack-work/foo/bar to ~/.local/bin, it will still refer to data-files and other such files in .stack-work. This, again, doesn't result in anything being broken. Your concern is that, if in the future the user deletes the .stack-work directory, things will break. Are we still on the same page?

And just to round things out: it's not technically true that paths will always be under .stack-work. If the package is part of the snapshot, it will instead be placed inside ~/.stack. Perhaps that's part of the problem you're worried about.

I have comments to make about this situation, but I want to confirm that we're actually talking about the same thing.

[DanielG]

Alright that makes a lot more sense. I can live with breakage for as long as the package isn't in a snapshot, thanks for the explaination and sorry about the confusion.

[snoyberg]

No problem. One last thing that might be useful to know: as long as you're building outside a project, the executable ends up under ~/.stack, e.g.:

$ stack build ghc-mod cabal-helper-0.5.2.0
$ ls .stack/global/.stack-work/install/x86_64-linux/lts-3.0/7.10.2/libexec/cabal-helper-wrapper
.stack/global/.stack-work/install/x86_64-linux/lts-3.0/7.10.2/libexec/cabal-helper-wrapper
$ stack exec which ghc-mod
Run from outside a project, using implicit global config
Using resolver: lts-3.0 from global config file: /home/vagrant/.stack/global/stack.yaml
/home/vagrant/.stack/global/.stack-work/install/x86_64-linux/lts-3.0/7.10.2/bin/ghc-mod