local options do not work with install on local packages (was: Run okay with profiling enabled, but installed exe has NO profiling enabled) #7297

complyue · 2021-02-23T08:39:26Z

Describe the bug
Can't install profiling-enabled exe with cabal 3.2.0.0, while direct running works correctly.

To Reproduce
Steps to reproduce the behavior:

$ cabal v2-run --enable-profiling --enable-executable-profiling  exe:els -- +RTS --info                                  
Resolving dependencies...
Up to date
 [("GHC RTS", "YES")
 ,("GHC version", "8.8.3")
 ,("RTS way", "rts_thr_p")
 ,("Build platform", "x86_64-apple-darwin")
 ,("Build architecture", "x86_64")
 ,("Build OS", "darwin")
 ,("Build vendor", "apple")
 ,("Host platform", "x86_64-apple-darwin")
 ,("Host architecture", "x86_64")
 ,("Host OS", "darwin")
 ,("Host vendor", "apple")
 ,("Target platform", "x86_64-apple-darwin")
 ,("Target architecture", "x86_64")
 ,("Target OS", "darwin")
 ,("Target vendor", "apple")
 ,("Word size", "64")
 ,("Compiler unregisterised", "NO")
 ,("Tables next to code", "YES")
 ,("Flag -with-rtsopts", "-maxN31 -qg")
 ]
$ cabal v2-install --enable-profiling --overwrite-policy=always --enable-executable-profiling exe:els && els +RTS --info 
Wrote tarball sdist to
 ...
Resolving dependencies...
Up to date
Copying 'els'
 [("GHC RTS", "YES")
 ,("GHC version", "8.8.3")
 ,("RTS way", "rts_thr")
 ,("Build platform", "x86_64-apple-darwin")
 ,("Build architecture", "x86_64")
 ,("Build OS", "darwin")
 ,("Build vendor", "apple")
 ,("Host platform", "x86_64-apple-darwin")
 ,("Host architecture", "x86_64")
 ,("Host OS", "darwin")
 ,("Host vendor", "apple")
 ,("Target platform", "x86_64-apple-darwin")
 ,("Target architecture", "x86_64")
 ,("Target OS", "darwin")
 ,("Target vendor", "apple")
 ,("Word size", "64")
 ,("Compiler unregisterised", "NO")
 ,("Tables next to code", "YES")
 ,("Flag -with-rtsopts", "-maxN31 -qg")
 ]
$ els +RTS -p
els: the flag -p requires the program to be built with -prof
els: 
els: Usage: <prog> <args> [+RTS <rtsopts> | -RTS <args>] ... --RTS <args>
els: ...
$

Expected behavior
The built executable can have +RTS -p enabled

System information

Operating system

macOS Mojave

cabal, ghc versions

$ cabal --version
cabal-install version 3.2.0.0
compiled using version 3.2.0.0 of the Cabal library 
$ ghc --version
The Glorious Glasgow Haskell Compilation System, version 8.8.3

Additional context
Add any other context about the problem here.

The text was updated successfully, but these errors were encountered:

fendor · 2021-02-23T22:45:20Z

Looks like a duplicate of #5982

complyue · 2021-02-24T07:08:38Z

In addition to the cmdl options being ignored (as with #5982), I also instrumented in the cabal.project.local for profiling, hope this add more information (or concerns) for the diagnostics.

$ cat cabal.project.local 

profiling: True
profiling-detail: all-functions

optimization: 0

$ cat cabal.project

-- Please change Cabal project settings per your needs

-- FOLLOWING CONTENTS GONNA BE OVERWRITTEN BY EPM --

--  * this file should be:
--  /fw/m3cyue/edh-universe/cabal.project

packages:
 ./e-wrks/edh/lossless-decimal/lossless-decimal.cabal
 ./e-wrks/edh/elr/elr.cabal
 ./e-wrks/edh/host.hs/edh.cabal
 ./e-wrks/nedh/host.hs/nedh.cabal
 ./e-wrks/sedh/host.hs/sedh.cabal
 ./e-wrks/hasdim/host.hs/hasdim.cabal
 ./e-wrks/haskit/host.hs/haskit.cabal
 ./e-wrks/els/host.hs/els.cabal
$

complyue · 2021-02-24T07:12:55Z

BTW, Albert Y.C. Lai suggests in a Cafe list mail that such a workaround should work, but seemingly not for my case.

https://mail.haskell.org/pipermail/haskell-cafe/2021-February/133453.html

amigalemming · 2021-03-04T20:50:36Z

Do you have profiling disabled globally in .cabal/config?

complyue · 2021-03-05T07:04:55Z

@amigalemming I'm not sure, here is the content:

$ cat .cabal/config | grep -e '^\s*--' -v


repository mirrors.tuna.tsinghua.edu.cn
  url: http://mirrors.tuna.tsinghua.edu.cn/hackage




remote-repo-cache: /Users/cyue/.cabal/packages
world-file: /Users/cyue/.cabal/world
extra-prog-path: /Users/cyue/.cabal/bin
build-summary: /Users/cyue/.cabal/logs/build.log
remote-build-reporting: anonymous
jobs: $ncpus
install-method: copy
installdir: /Users/cyue/.cabal/bin

haddock

init

install-dirs user

install-dirs global

program-locations

program-default-options
$

Did I?

amigalemming · 2021-03-05T07:34:52Z

On Thu, 4 Mar 2021, Compl Yue wrote: @amigalemming I'm not sure, here is the content: $ cat .cabal/config | grep -e '^\s*--' -v

I would grep for 'profil', but it looks like you have not touched the profiling settings.

romanofski · 2021-09-28T01:46:54Z

I actually tripped over this too when working on purebred-mua/purebred#299. Also not explicitly profiling disabled. What I thought was interesting was that setting profiling: True in my cabal.project.local would not have an effect and neither would --enable-library-profiling. And by affect I mean, that under dist-newstyle I find profiling information, but in the installed package under ~/.cabal/store just the normal libraries.

Even the build progress does not show that profiling is enabled. If I build with:

$ cabal new-install -p -j --overwrite-policy=always --lib purebred-icu

I get:

In order, the following will be built (use -v for more details):
 - purebred-0.1.0.0 (lib) (requires build)
 - purebred-icu-0.1.0.0 (lib) (requires build)

Using the workaround linked from the mailing list, I get:

In order, the following will be built (use -v for more details):
 - purebred-0.1.0.0 (lib)  --enable-profiling (requires build)
 - purebred-icu-0.1.0.0 (lib)  --enable-profiling (requires build)

Perhaps it has something to do with multiple packages that even the command line arguments are not applied? My config that enabled profiling:

packages: .
          ../purebred-icu

package *
  profiling: True

The one which does not enable does not have the package * line.

Mikolaj · 2021-09-28T07:53:43Z

That's a long shot, but it may be related to #7236, where static compilation works with cabal build, but is ignored with cabal install.

fgaz · 2021-10-09T09:19:49Z

I have an hypothesis for why this happens:

v2-install sdists the packages before installing them
- this means the packages are treated as remote packages
command line options are equivalent to top-level cabal.project options
- top-level cabal.project options apply to local packages
=> the cli and top-level options are not applied to v2-installed packages!

If that's true, a possible workaround could be to set the desired options per-package in cabal.project. I did not test this yet.

fgaz · 2021-10-09T09:35:01Z

I did a few tests and it looks like my hypothesis is correct.

Workaround

for every local package x, add this snippet to your cabal.project

package x
  profiling: True

The same applies for other flags.

edit: oops, I just noticed that a similar workaround was posted above (though that will apply to all dependencies too)

I'll edit the title to reflect this.
This is very likely to be a part of what's happening in #7236

jneira · 2021-10-09T11:58:16Z

I am starting to think that actual implementation of cabal install is giving us more problems than a raw build + copy bins would do. Another symptom: the popularization of the pattern cabal build && cp $(cabal list-bins myexec) somepath

Afaik the actual mechanism was implemented to make the code reuse the cabal install my.local.or.remote.tar variation. am I correct?

Mikolaj · 2021-10-09T12:08:57Z

Afaik the actual mechanism was implemented to make the code reuse the cabal install my.local.or.remote.tar variation

I'm not sure what the original intention was, but I think cabal install does too many things (even without the --lib flags), such as installing from Hackage, installing from local package, installing from repo or archive, etc. Creating separate simple aliases for each of these might be less error prone that guessing what the user wants (e.g., whether the package from Hackage should be installed according to the configuration from local directory, which is what happens by default right now).

See #7471 (comment) and other comments.

fgaz · 2021-10-09T12:22:16Z

Afaik the actual mechanism was implemented to make the code reuse the cabal install my.local.or.remote.tar variation. am I correct?

More or less. I originally wrote v2-install in #4825, and it supported remote packages only, then @typedrat extended it to local packages (and libraries) by combining that mechanism with sdist in #5399. The original design document is #4558

gbaz · 2021-10-09T16:31:49Z

Kudos to @fgaz for the excellent sleuthing here.

I think this is really a bad interaction of how install treats cli flags -- it should probably translate them to package local flags for the package being installed, rather than top level project flags.

Mikolaj · 2021-10-09T17:13:34Z

I think users expect flags to do the same for cabal build and cabal install, even just to be able to easily switch between the two in commandline invocations. Does your proposal ensure that for each use case of cabal install? If not, perhaps that's the criterion for which of many functionalities of cabal install should be split off and renamed to a less generic verb so that it's clear the semantics is very different from cabal build.

Actually, perhaps users even expect cabal install to be equilvalent to what @jneira cited: cabal build && cp $(cabal list-bins myexec) somepath.

phadej · 2021-10-09T17:21:04Z

Note that

Actually, perhaps users even expect cabal install to be equilvalent to what @jneira cited: cabal build && cp $(cabal list-bins myexec) somepath.

cannot be completely true, unit-ids (of local libraries) are different, Paths_ point to different directories, maybe something else I forget.

The goal is good, but there are subtle differences, which won't go away (especially the unit-id). Maybe that false expectation should rather be explained in the manual.

fgaz · 2021-11-06T17:11:33Z

By the way this is also super counterintuive wrt explicitly named remote targets. For example cabal install --enable-profiling some-package-from-hackage will not enable profiling for that package!

jneira · 2022-02-14T17:10:06Z

to let it clear, the welcomed pr would follow @gbaz proposal?:

I think this is really a bad interaction of how install treats cli flags -- it should probably translate them to package local flags for the package being installed, rather than top level project flags.

Either for installing within a local project or from hackage/tarball?

jneira · 2022-02-23T13:38:50Z

hmm not sure what is the meaning of consulted here, it is not being applied (or it would be a workaround for the issue) but it is available in the config for the build prior to the install (like the package entries in the cabal.project, the actual workaround).

So we could use it like we want to do with the cli flags (which also are available and are not applied); the question is if doing it would be sensible.

It would make a little bit simpler the fix cause it will not have to discern between installing local packages from remote ones.
I think the top level config should be applied when installing local packages for sure or it would be another unexpected behaviour.

jneira · 2022-02-23T13:45:46Z

I think the top level config should be applied when installing local packages for sure or it would be another unexpected behaviour.

If users dont want to apply them they always could do a cabal install package-from-hackage-or-tarball --ignore-project

EDIT: Add the previous comment as a quote to make clear it refers to a feature (install --ignore-project` to be added and dont work inthe current version of cabal-install

gbaz · 2022-02-23T14:09:43Z

I'm honestly not sure of the right behavior here and could be swayed either way. Would your proposal amount to effectively treating the sdist'd tarball being installed as a local package?

Mikolaj · 2022-02-23T20:44:54Z

Slightly related: we really should split cabal install-local and cabal install-from-hackage-or-tarball both in UI and as much as feasable in the code paths.

jneira · 2022-02-23T21:02:20Z

Well, to be fair i personally would expect cabal.project top level build config be applicable only installing local packages executables and no remote ones, even if they are included in dependencies. It is clear to me than top project config only is applicable to local packages and i think it is described in docs. But i am not totally convinced so asked to know other's opinion.

So to resuming in a table:

	build local pkg	install local pkg	install remote pkg
top project config	✔️	✔️	❌
cli args	✔️	✔️	✔️

All would be a little bit clearer (and less convenient) if top level project config did not exist and you had to do package :locals\n xxx

jneira · 2022-02-24T07:19:13Z

I'm honestly not sure of the right behavior here and could be swayed either way. Would your proposal amount to effectively treating the sdist'd tarball being installed as a local package?

yeah so adding the package(s) as a local package(s) in the project config could do the fix for all target types

I foresee that would be simpler than distinguish cli flags (they are not marked as such in the actual project config afaics) from project config top level ones and apply them to remote packages. In that case we still could add local packages targets as such in the project config

jneira · 2022-03-03T17:20:39Z

This issue is about --extra-lib-dirs and --extra-include-dirs being ignored as cli args, in cabal.project inside package sections but is has interesting insights (from hvr, phadej, emilpy and others) about they being ignored as cli args for cabal install specifically

Currently, there are three kinds of cabal configurations considered when determining an option of an `ElaboratedConfiguredPackage`: * Global configuration, in `.cabal/config` * Local configuration, in - Options passed in the cabal invocation, e.g. `cabal build --enable-executable-dynamic` - Fields in the top level `cabal.project`, or `cabal.project.local`, e.g. `extra-include-dirs: /opt/homebrew/include` Note thus that top-level cabal.project flags and cli flags are treated all together at the same level (`local`). * Per package configuration, as in package HsOpenSSL extra-include-dirs: /opt/homebrew/Cellar/openssl@3/3.2.0_1/include extra-lib-dirs: /opt/homebrew/Cellar/openssl@3/3.2.0_1/lib Then, we have a definition for whether a package is local to the project. The local packages are the packages listed in the project which have a specific source package, rather than just being listed by name in a `source-repository-stanza`, or in a `package <package-name>` stanza that configures installed packages. In this patch, we try fix the mistmatch between the `local` flags and the packages which are deemed `local`, and define a specification for what exactly should happen..... TODO Fixes haskell#7297, haskell#8909, haskell#2997

Currently, there are three kinds of cabal configurations considered when determining an option of an `ElaboratedConfiguredPackage`: * Global configuration, in `.cabal/config` * Local configuration, in - Options passed in the cabal invocation, e.g. `cabal build --enable-executable-dynamic` - Fields in the top level `cabal.project`, or `cabal.project.local`, e.g. `extra-include-dirs: /opt/homebrew/include` Note thus that top-level cabal.project flags and cli flags are treated all together at the same level (`local`). * Per package configuration, as in package HsOpenSSL extra-include-dirs: /opt/homebrew/Cellar/openssl@3/3.2.0_1/include extra-lib-dirs: /opt/homebrew/Cellar/openssl@3/3.2.0_1/lib Then, we have a definition for whether a package is local to the project. The local packages are the packages listed in the project which have a specific source package, rather than just being listed by name in a `source-repository-stanza`, or in a `package <package-name>` stanza that configures installed packages. The reason why local packages being installed are treated as non-local is that TODO In this patch, we try fix the mistmatch between the `local` flags and the packages which are deemed `local`, and define a specification for what exactly should happen..... TODO Fixes haskell#7297, haskell#8909, haskell#2997

alt-romes · 2024-02-02T14:28:27Z

I've observed some quite interesting behaviour regarding this and related issues (#7236, #8909), which report that flags passed to install are ignored and not applied to the executable being built.

Namely, note the following

cabal install --enable-profiling # NOT profiled
cabal install <my-pkg> --enable-profiling # DOES profile
cabal install exe:<my-pkg> --enable-profiling # NOT profile

I'm investigating...

The target of `cabal install` is not considered to be a local package, which means local configuration (e.g. in cabal.project, or flags like --enable-profiling) does not apply to it. In 76670eb, we changed the behaviour to applying the local flags to cabal install targets, but it used the literal target string as a package name to which the flags were additionally applied. However, `cabal install` targets are NOT necessarily package names, so, e.g., if we did `cabal install exe:mycomp`, the local flags would not apply since "exe:mycomp" is not a recognized /package/. The solution is to parse the target selectors first, and apply the local flags to the package of the resolved targets. Fixes haskell#7297 haskell#8909 and the install part of haskell#7236

alt-romes · 2024-02-05T18:06:47Z

Fix in #9697.

alt-romes · 2024-02-05T18:10:16Z

To add a bit to the discussion:
At a certain point (76670eb) we started applying local configuration (such as cli arguments and top level cabal.project fields) to the packages being installed. However, that patch only accounted for exact package names being specified as cabal install targets.

So

cabal install <my-pkg> --enable-profiling

would apply --enable-profiling to the package, but any other form of target like all:exes, no explicit target, exe:my-exe, ... would not get the local configuration.

What I've done is properly account for cabal install targets, and apply the local configuration to the appropriate packages (the limitation is that there is no way to apply configuration per-component, but that's fine. it means that building the exe with --enable-profiling will also build the profiled sources of a library sharing the package with said executable).

The target of `cabal install` is not considered to be a local package, which means local configuration (e.g. in cabal.project, or flags like --enable-profiling) does not apply to it. In 76670eb, we changed the behaviour to applying the local flags to cabal install targets, but it used the literal target string as a package name to which the flags were additionally applied. However, `cabal install` targets are NOT necessarily package names, so, e.g., if we did `cabal install exe:mycomp`, the local flags would not apply since "exe:mycomp" is not a recognized /package/. The solution is to parse the target selectors first, and apply the local flags to the package of the resolved targets. Fixes haskell#7297 haskell#8909 and the install part of haskell#7236

The target of `cabal install` is not considered to be a local package, which means local configuration (e.g. in cabal.project, or flags like --enable-profiling) does not apply to it. In 76670eb, we changed the behaviour to applying the local flags to cabal install targets, but it used the literal target string as a package name to which the flags were additionally applied. However, `cabal install` targets are NOT necessarily package names, so, e.g., if we did `cabal install exe:mycomp`, the local flags would not apply since "exe:mycomp" is not a recognized /package/. The solution is to parse the target selectors first, and apply the local flags to the package of the resolved targets. Fixes haskell#7297, haskell#8909, the install part of haskell#7236, haskell#8529, haskell#7832

fgaz added cabal-install: cmd/install type: bug labels Feb 24, 2021

amigalemming mentioned this issue Mar 4, 2021

cabal-install local profiling settings do not override global settings #7314

Open

romanofski mentioned this issue Sep 27, 2021

Documentation request: enable profiling purebred-mua/purebred#299

Closed

fgaz added the can-workaround There is a (maybe partial) workaround for the issue or missing feature label Oct 9, 2021

fgaz changed the title ~~Run okay with profiling enabled, but installed exe has NO profiling enabled~~ local options do not work with install on local packages (was: Run okay with profiling enabled, but installed exe has NO profiling enabled) Oct 9, 2021

fgaz mentioned this issue Oct 9, 2021

cabal install/build: static executables and stripping #7236

Closed

jneira mentioned this issue Oct 10, 2021

Install test executables with cabal install when "--enable-tests" present #7693

Open

fgaz mentioned this issue Oct 10, 2021

cabal install run immediately after cabal build unnecessarily rebuilds the project #6919

Open

gbaz added the attention: pr-welcome label Nov 23, 2021

gbaz mentioned this issue Nov 23, 2021

Cannot install a profiled library with cabal install --lib -p pkgname #7832

Closed

jneira mentioned this issue Feb 13, 2022

cabal v2-install --enable-profiling does not profile #5982

Closed

jneira mentioned this issue Mar 3, 2022

--extra-lib-dirs and --extra-include-dirs ignored? #2997

Open

andreasabel mentioned this issue Apr 9, 2022

cabal v2-install --ignore-project crashes when project file is present #8094

Closed

Mikolaj mentioned this issue Jun 1, 2022

Cabal 3.4.0.0 release on Windows lacks extensions #7298

Closed

bairyn mentioned this issue Sep 25, 2022

Track static vs. dynamic dependencies. #8461

Closed

nomeata added a commit to nomeata/cabal-plan-bounds that referenced this issue Jan 5, 2023

Point to issue haskell/cabal#7297

d7d79f3

alt-romes mentioned this issue Jan 8, 2024

cabal install ignores --enable-executable-dynamic flag #8909

Closed

alt-romes self-assigned this Feb 1, 2024

alt-romes mentioned this issue Feb 5, 2024

Apply local configuration to install targets #9697

Merged

mergify bot closed this as completed in #9697 Feb 9, 2024

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

local options do not work with install on local packages (was: Run okay with profiling enabled, but installed exe has NO profiling enabled) #7297

local options do not work with install on local packages (was: Run okay with profiling enabled, but installed exe has NO profiling enabled) #7297

complyue commented Feb 23, 2021

fendor commented Feb 23, 2021

complyue commented Feb 24, 2021

complyue commented Feb 24, 2021

amigalemming commented Mar 4, 2021

complyue commented Mar 5, 2021

amigalemming commented Mar 5, 2021 via email

romanofski commented Sep 28, 2021

Mikolaj commented Sep 28, 2021

fgaz commented Oct 9, 2021

fgaz commented Oct 9, 2021 •

edited

Loading

jneira commented Oct 9, 2021 •

edited

Loading

Mikolaj commented Oct 9, 2021 •

edited

Loading

fgaz commented Oct 9, 2021

gbaz commented Oct 9, 2021

Mikolaj commented Oct 9, 2021

phadej commented Oct 9, 2021

fgaz commented Nov 6, 2021

jneira commented Feb 14, 2022

jneira commented Feb 23, 2022 •

edited

Loading

jneira commented Feb 23, 2022 •

edited

Loading

gbaz commented Feb 23, 2022

Mikolaj commented Feb 23, 2022

jneira commented Feb 23, 2022

jneira commented Feb 24, 2022

jneira commented Mar 3, 2022

alt-romes commented Feb 2, 2024

alt-romes commented Feb 5, 2024

alt-romes commented Feb 5, 2024 •

edited

Loading

local options do not work with install on local packages (was: Run okay with profiling enabled, but installed exe has NO profiling enabled) #7297

local options do not work with install on local packages (was: Run okay with profiling enabled, but installed exe has NO profiling enabled) #7297

Comments

complyue commented Feb 23, 2021

fendor commented Feb 23, 2021

complyue commented Feb 24, 2021

complyue commented Feb 24, 2021

amigalemming commented Mar 4, 2021

complyue commented Mar 5, 2021

amigalemming commented Mar 5, 2021 via email

romanofski commented Sep 28, 2021

Mikolaj commented Sep 28, 2021

fgaz commented Oct 9, 2021

fgaz commented Oct 9, 2021 • edited Loading

Workaround

jneira commented Oct 9, 2021 • edited Loading

Mikolaj commented Oct 9, 2021 • edited Loading

fgaz commented Oct 9, 2021

gbaz commented Oct 9, 2021

Mikolaj commented Oct 9, 2021

phadej commented Oct 9, 2021

fgaz commented Nov 6, 2021

jneira commented Feb 14, 2022

jneira commented Feb 23, 2022 • edited Loading

jneira commented Feb 23, 2022 • edited Loading

gbaz commented Feb 23, 2022

Mikolaj commented Feb 23, 2022

jneira commented Feb 23, 2022

jneira commented Feb 24, 2022

jneira commented Mar 3, 2022

alt-romes commented Feb 2, 2024

alt-romes commented Feb 5, 2024

alt-romes commented Feb 5, 2024 • edited Loading

fgaz commented Oct 9, 2021 •

edited

Loading

jneira commented Oct 9, 2021 •

edited

Loading

Mikolaj commented Oct 9, 2021 •

edited

Loading

jneira commented Feb 23, 2022 •

edited

Loading

jneira commented Feb 23, 2022 •

edited

Loading

alt-romes commented Feb 5, 2024 •

edited

Loading