Skip to content

Copying executables when installing is fragile with packages relying on the paths in Cabal's Paths_<pkg-name> module #828

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
DanielG opened this issue Aug 21, 2015 · 15 comments

Comments

@DanielG
Copy link
Contributor

DanielG commented Aug 21, 2015

Since stack install is merely an alias for copying executables after building them this means that the Paths_ module doesn't get re-generated for the actual installation prefix but rather remains at pointing to somewhere in .stack-work. This is a real problems for programs or libraries that rely on these paths being correct.

cabal install does this correctly and rebuilds the whole package with the proper paths in this module.

My specific use case is cabal-helper where, in the library, I have to use libexecdir to find the wrapper executable (since I don't want that to show up on the user's PATH). Since there isn't such a thing as LIBEXECPATH I'm kind of stuck here as I can't think of another way to find my executables in libexec.

@snoyberg
Copy link
Contributor

cabal install and stack are doing the same thing here: neither one is recompiling executables. You can see that, for example, by running cabal install inside a sandbox, which is essentially what stack install is doing. I'm having trouble understanding what the problem is here: are you saying that running the generated executable doesn't work, or that you don't like the paths it's using internally?

@DanielG
Copy link
Contributor Author

DanielG commented Aug 21, 2015

Ok so if I do the following with cabal-install:

$ cabal configure --prefix=/tmp/prefix
$ cabal build -j

$ cabal install .

Then cabal install . will actually reconfigure and rebuild the project with prefix set to the default path i.e. ~/.cabal instead of /tmp/prefix.

The problem is that stack build initializes the prefix to somewhere in .stack-work and then doesn't change it to wherever the executables will actually end up on stack install.

@borsboom
Copy link
Contributor

What would you suggest the preferred behaviour be?

@snoyberg
Copy link
Contributor

And perhaps you can describe the problem you're actually trying to solve? I read through the linked issue, and couldn't understand what problem the libexecdir is trying to work around.

@DanielG
Copy link
Contributor Author

DanielG commented Aug 24, 2015

Hmm it seems like this issue is more weird than I though. On second look the Paths_ module as generated by stack build does contain the "right" path to $libexec, namely .cabal/libexec but all the other paths (bindir, libdir, etc.) are pointing to ./.stack-work/....

If I add a plain stack.yaml to cabal-helper (https://github.com/DanielG/cabal-helper) and stack build inside that repo it does install the cabal-helper-wrapper executable into .cabal/libexec (which it probably shouldn't):

$ stack build
cabal-helper-0.5.2.0: configure  
Configuring cabal-helper-0.5.2.0...
cabal-helper-0.5.2.0: build
Building cabal-helper-0.5.2.0...
Preprocessing library cabal-helper-0.5.2.0...
[1 of 4] Compiling CabalHelper.Sandbox ( CabalHelper/Sandbox.hs, .stack-work/dist/x86_64-linux/Cabal-1.22.4.0/build/CabalHelper/Sandbox.o )
[2 of 4] Compiling CabalHelper.Types ( CabalHelper/Types.hs, .stack-work/dist/x86_64-linux/Cabal-1.22.4.0/build/CabalHelper/Types.o )
[3 of 4] Compiling Paths_cabal_helper ( .stack-work/dist/x86_64-linux/Cabal-1.22.4.0/build/autogen/Paths_cabal_helper.hs, .stack-work/dist/x86_64-linux/Cabal-1.22.4.0/build/Paths_cabal_helper.o )
[4 of 4] Compiling Distribution.Helper ( Distribution/Helper.hs, .stack-work/dist/x86_64-linux/Cabal-1.22.4.0/build/Distribution/Helper.o )
In-place registering cabal-helper-0.5.2.0...
Preprocessing executable 'cabal-helper-wrapper' for cabal-helper-0.5.2.0...
[1 of 9] Compiling CabalHelper.Data ( CabalHelper/Data.hs, .stack-work/dist/x86_64-linux/Cabal-1.22.4.0/build/cabal-helper-wrapper/cabal-helper-wrapper-tmp/CabalHelper/Data.o )
[2 of 9] Compiling CabalHelper.Sandbox ( CabalHelper/Sandbox.hs, .stack-work/dist/x86_64-linux/Cabal-1.22.4.0/build/cabal-helper-wrapper/cabal-helper-wrapper-tmp/CabalHelper/Sandbox.o )
[3 of 9] Compiling CabalHelper.Types ( CabalHelper/Types.hs, .stack-work/dist/x86_64-linux/Cabal-1.22.4.0/build/cabal-helper-wrapper/cabal-helper-wrapper-tmp/CabalHelper/Types.o )
[4 of 9] Compiling CabalHelper.Log  ( CabalHelper/Log.hs, .stack-work/dist/x86_64-linux/Cabal-1.22.4.0/build/cabal-helper-wrapper/cabal-helper-wrapper-tmp/CabalHelper/Log.o )
[5 of 9] Compiling CabalHelper.GuessGhc ( CabalHelper/GuessGhc.hs, .stack-work/dist/x86_64-linux/Cabal-1.22.4.0/build/cabal-helper-wrapper/cabal-helper-wrapper-tmp/CabalHelper/GuessGhc.o )
[6 of 9] Compiling CabalHelper.Common ( CabalHelper/Common.hs, .stack-work/dist/x86_64-linux/Cabal-1.22.4.0/build/cabal-helper-wrapper/cabal-helper-wrapper-tmp/CabalHelper/Common.o )
[7 of 9] Compiling Paths_cabal_helper ( .stack-work/dist/x86_64-linux/Cabal-1.22.4.0/build/autogen/Paths_cabal_helper.hs, .stack-work/dist/x86_64-linux/Cabal-1.22.4.0/build/cabal-helper-wrapper/cabal-helper-wrapper-tmp/Paths_cabal_helper.o )
[8 of 9] Compiling CabalHelper.Compile ( CabalHelper/Compile.hs, .stack-work/dist/x86_64-linux/Cabal-1.22.4.0/build/cabal-helper-wrapper/cabal-helper-wrapper-tmp/CabalHelper/Compile.o )
[9 of 9] Compiling Main             ( CabalHelper/Wrapper.hs, .stack-work/dist/x86_64-linux/Cabal-1.22.4.0/build/cabal-helper-wrapper/cabal-helper-wrapper-tmp/Main.o )
[7 of 9] Compiling CabalHelper.Data ( CabalHelper/Data.hs, .stack-work/dist/x86_64-linux/Cabal-1.22.4.0/build/cabal-helper-wrapper/cabal-helper-wrapper-tmp/CabalHelper/Data.o )
Linking .stack-work/dist/x86_64-linux/Cabal-1.22.4.0/build/cabal-helper-wrapper/cabal-helper-wrapper ...
cabal-helper-0.5.2.0: install
extended "cabal-helper-wrapper"
Installing executable(s) in /home/dxld/.cabal/libexec
Warning: The directory /home/dxld/.cabal/libexec is not in the system search
path.
Installing library in
/home/dxld/share/dev/hs/cabal-helper/.stack-work/install/x86_64-linux/lts-3.1/7.10.2/lib/x86_64-linux-ghc-7.10.2/cabal-helper-0.5.2.0-6OC4KX1HZFK46by8bcnSLv
Registering cabal-helper-0.5.2.0...

And now stack install will error out because cabal-helper-wrapper wasn't installed into where it wants it to be:

Couldn't find executable cabal-helper-wrapper in directory /home/dxld/share/dev/hs/cabal-helper/.stack-work/install/x86_64-linux/lts-3.1/7.10.2/bin/

I use a custom Setup.hs to implement the X-Install-Target field in my .cabal file https://github.com/DanielG/cabal-helper/blob/master/Setup.hs that overrides the inst and copy hooks so I'm not sure what can really be done about this other than adding Install-Target to Cabal proper :/

@DanielG
Copy link
Contributor Author

DanielG commented Aug 24, 2015

I'm starting to think the initial bug report was bogus, if I install ghc-mod with stack locally everything just happens to end up in the right path.

I still believe the deviation from what cabal-install would do with bindir, libdir etc. is a bug though. How to solve that properly I have no idea. What I would do is 1) make stack install a separate command that generates the sdist, unpacks it into a temporary directory, configures the installation prefix for the Paths_ stuff to caba-install's defaults instead of the .stack-work stuff and then does the build like normal. Actually generating the sdist ensures that stack install is exactly equivalent to what downloading the package from hackage would do, I've run into forgetting to add some test data to Extra-Source-Files wayyyyy too often already ;)

stack install failing when Setup.hs is doing weird stuff with the installtion directories I can understand and so long as it doesn't actually make the installation fail it's fine I suppose.

@snoyberg
Copy link
Contributor

I've fixed the libexecdir and sysconfdir not being set, thanks for reporting that.

@DanielG
Copy link
Contributor Author

DanielG commented Aug 24, 2015

That's all nice and well but this makes the issue of what happens on stack install even more pressing because now the $libexec directory will disappear as soon as the user decides to delete the build directory...

@snoyberg
Copy link
Contributor

I'm really missing the forest for the trees here, I don't have any idea why this problem exists in the first place. Why isn't cabal-helper just being installed into the bin directory like all other executables?

@snoyberg
Copy link
Contributor

Alright, reviewing this again, I think I'm starting to understand what you're saying, and where the misunderstanding is starting. For future reference, it would be a lot easier to follow a bug report if it followed the recommended bug reporting guidelines.

Current situation: all paths in the Paths_* module refer to local directories inside of the stack-work directory. AFAICT, despite the implications in the initial bug report, this doesn't break anything: all executables are generated with paths that actually exist. Is this correct?

The concern that you have is that, when copying an executable from .stack-work/foo/bar to ~/.local/bin, it will still refer to data-files and other such files in .stack-work. This, again, doesn't result in anything being broken. Your concern is that, if in the future the user deletes the .stack-work directory, things will break. Are we still on the same page?

And just to round things out: it's not technically true that paths will always be under .stack-work. If the package is part of the snapshot, it will instead be placed inside ~/.stack. Perhaps that's part of the problem you're worried about.

I have comments to make about this situation, but I want to confirm that we're actually talking about the same thing.

@DanielG
Copy link
Contributor Author

DanielG commented Aug 24, 2015

Alright that makes a lot more sense. I can live with breakage for as long as the package isn't in a snapshot, thanks for the explaination and sorry about the confusion.

@DanielG DanielG closed this as completed Aug 24, 2015
@snoyberg
Copy link
Contributor

No problem. One last thing that might be useful to know: as long as you're building outside a project, the executable ends up under ~/.stack, e.g.:

$ stack build ghc-mod cabal-helper-0.5.2.0
$ ls .stack/global/.stack-work/install/x86_64-linux/lts-3.0/7.10.2/libexec/cabal-helper-wrapper
.stack/global/.stack-work/install/x86_64-linux/lts-3.0/7.10.2/libexec/cabal-helper-wrapper
$ stack exec which ghc-mod
Run from outside a project, using implicit global config
Using resolver: lts-3.0 from global config file: /home/vagrant/.stack/global/stack.yaml
/home/vagrant/.stack/global/.stack-work/install/x86_64-linux/lts-3.0/7.10.2/bin/ghc-mod

@DanielG
Copy link
Contributor Author

DanielG commented Aug 24, 2015

Ah I was wondering where those were coming from.

@pavelkogan
Copy link

What about if you want to install executables that require access to the directories in Paths_* on a different system to the one it's built on?

With cabal you can do

cabal configure --prefix=/path/to/prefix
cabal build
cabal copy --destdir=./dist/pkg

then just copy the pkg directory over.

Is something similar possible with stack?

@borsboom
Copy link
Contributor

@pavelkogan: See #848, we're looking to address this.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

4 participants