docs: cabal.project search path is not documented

[ulysses4ever]

My understanding is that cabal.project affects a build even if the file lives in a parent directory, not only the current directory. Maybe this is well-known, but not for me, and I've been experiencing spooky actions at a distance for a little while. The closest the docs get to is (if I'm not mistaken):

The full configuration of a project is determined by combining the following sources (later entries override earlier ones, except for appendable options):

• ~/.cabal/config (the user-wide global configuration)

• cabal.project (the project configuration)

• cabal.project.freeze (the output of cabal v2-freeze)

Which, if anything, suggests that the global config file and the two from the current directory affect the build. At least, having an absolute path in the first bullet point indicates to me that the rest are probably relative paths.

If you agree that it's an omission, I could supply some kind of amendment to the above text.

[Mikolaj]

Related: #7695

Yes, we need better and consistent docs for that: in code, readthedocs, commandline help, if appropriate.

[andreasabel]

@ulysses4ever

My understanding is that cabal.project affects a build even if the file lives in a parent directory

This sounds like a misfeature to me. Can we get rid of it? What motivated it in the first place?

[Mikolaj]

I guess, cabal needs context to build a package and that context is provided by a project. If that package is not part of any project, some default context is assumed. Project can be determined by files in parent directories as well as by commandline overrides. An example of the contest info is the location of some package dependencies (that, e.g., are not on Hackage at all).

I guess one may ask why in such cases packages are not built by staying in the directory where the project file is. I don't know --- perhaps cabal cds to subdirectories where it builds things, e.g., because tools it calls assume they are run in the same directory as the source they operate on?

[ulysses4ever]

It's probably clear from my original message that I also find this feature confusing. The argument for it I got from someone is that many build tools work this way today (I think bazel does); also, git.

[fgaz]

I think the feature can be useful, though I don't use it often. Like with git, it's nice to not have to cd back to the root of the project when invoking a command in a subdirectory.

What kind of confusing behaviour did you experience?

[ulysses4ever]

I had a completely unrelated cabal.project file hanging up the directory, which changed the way the current package builds in a subtle way.

[ulysses4ever]

I guess, ideally I'd expect cabal-install messaging me about loading a cabal.project file.

[fgaz]

I had a completely unrelated cabal.project file hanging up the directory, which changed the way the current package builds in a subtle way.

Ah I see... this can indeed happen if the inner project does not have a cabal.project file.

[andreasabel]

@ulysses4ever

It's probably clear from my original message that I also find this feature confusing.

Me too. I only have one project with subpackages (https://github.com/BNFC/bnfc/blob/master/cabal.project), but in my working directory I mostly have the cabal.project file removed because I only want to build the BNFC subpackage (and not the testing subpackage). So, in practice, I have to delete and restore the cabal.project file back an forth...

@Mikolaj:

I guess, cabal needs context to build a package and that context is provided by a project.

Exactly, and I think the context should be determined by the directory where cabal has been invoked.
This means that I can consider this directory as package, or subpackage of projects that have this directory as subdirectory, depending from where I invoke cabal. Thanks to linking this can be several different projects.

@fgaz wrote:

Like with git, it's nice to not have to cd back to the root of the project when invoking a command in a subdirectory.

I can see that such behavior can be useful in situations where you work with both projects and individual files.

• E.g. in git, we might want to git add <file> and thus it has to be determined which project this file belongs to.
• In Agda, we want to load a file into the editor and to interpret it (to resolve the modules it imports0, we have to determine the project it belongs to, so we search the parent directories for a .agda-lib file.
• Similar for GHC and Haskell's hierarchical modules.

cabal isn't really working on individual files, but on projects only. So the behavior to "search above" for a context is less necessary and it is (imho) confusing. There are further reasons why such a behavior is unexpected:

• It is not that cabal would look above for .cabal files. Many users still work with .cabal files only and no cabal.projectfiles.
• The ubiquitous build tool make doesn't do though either. Instead it has a flag -C <dir> that allows to temporarily to <dir>, thus, providing the context for the build.

The "looking above" (mis?)feature has these harms:

• Making the behavior of cabal harder to understand, as it becomes more context-sensitive.
• It could be used in principle for sabotage: the BOFH could place a cabal.project[.local] in / and thus sabotage cabal for every user (sitting under /home/...) who does not use own cabal.project files.

[ulysses4ever]

A recent example: just-released Nix 2.6 has this in release notes:

• The Nix CLI now searches for a flake.nix up until the root of the current Git repository or a filesystem boundary rather than just in the current directory.

[andreasabel]

A recent example: just-released Nix 2.6 has this in release notes:

* The Nix CLI now searches for a flake.nix up until the root of the current Git repository or a filesystem boundary rather than just in the current directory.

The quoted release notes do not link to an issue or a more detailed documentation.
I would wonder what exactly "or" means there: Does it stop at the git-root always if started inside a git repository, or only when it found a flake until then?

Anyway, this would not help if I want to cabal build a non-root package that does not have a project file. It would still take me to the root.

[fgaz]

#7057 would help with that

[fgaz]

I guess, ideally I'd expect cabal-install messaging me about loading a cabal.project file.

Yeah this could be useful. Maybe it could only do that when the project file is not in the current directory (and be clear about that, eg. "Notice: loaded a project file from an upper directory: /blah/blah")

[andreasabel]

I think searching upward for a cabal.project could easily be implemented by tooling outside of cabal, like a shell script.
However, if I want to prevent cabal to do so atm, I need to create a cabal.project in the local directory, changing the state of my file system. I don't like state change---I am a FP person.

[Mikolaj]

I there are really no compelling reason for this behaviour, we can certainly drop it, with some backward compatibility scheme. Simpler is better. We'd need an RFC and some time to collect feedback.

[andreasabel]

I have been experimenting a bit more just now, and found something I had overlooked. It works well if one does everything right like:

/cabal.project  -- containing: "packages: A B"
/A/A.cabal 
/A/cabal.project.local
/B/B.cabal

Then I can invoke cabal build fine in either A/ or B/ and in builds the right package.

Yet I tripped quite often in the past.
So it would be good if cabal told which project and cabal files it is using for each step, like e.g. in A/:

Package configured by:
- ../cabal.project
- cabal.project.local
- A.cabal

[ulysses4ever]

Big plus for cabal reporting which project files are loaded. Ambivalent (slightly leaning no: it may be handy perhaps) for removing the feature.

[Mikolaj]

Let's report the project file in use and, independenly, continue discussing a possible enhancement of the functionality. Discussion should not paralyse improvements.

[ratherforky]

I just got burned by this. It took me ~5 mins to figure out some spooky action at a distance was happening, ~20 mins to work out cabal was being affected by a parent directory, and then another hour to find a workaround.

I'm using an automarker I wrote in Haskell to mark student's Haskell coursework. The student cabal projects go in a subdirectory of the automarker cabal project. In previous years, this wasn't a problem, but this time when I tried to run the student projects I was getting Error: cabal: No targets given and there is no package in the current directory.. Turns out I used to have a cabal.project for the student projects, but this time I didn't. @andreasabel hit the nail on the head with what my confusion was (thank you!):

It is not that cabal would look above for .cabal files. Many users still work with .cabal files only and no cabal.project files.

I almost never write cabal.project files unless I need to specify something out of the ordinary so this interaction was very surprising. I don't have much else to add (though a flag to disable this behaviour would be nice), just want it on the record.

[geekosaur]

Does --ignore-project work?

[geekosaur]

Also, I will note that this semantic is common for tools such as git, and it is often useful to run cabal in a directory containing a cabal file and expect the project configuration to be used.

[ratherforky]

I saw --ignore-project mentioned in places but I can't figure out where I'm supposed to use it. cabal build --ignore-project gives me: Error: cabal: unrecognized 'build' option '--ignore-project'

I agree it's not entirely unprecedented behaviour and I originally suspected it was a .gitignore problem (the subdir is .gitignored, so I thought cabal might be trying to respect the parent .gitignore and not seeing the .cabal file), so I grep'd for any flags with 'ignore' and --ignore-project didn't come up. I tried lots of different flags and nothing worked.

I'm not opposed to cabal checking the parent directories, but the behaviour I expected was for it to prioritise the most local project (like git does and like cabal does if there's a cabal.project file). I don't feel strongly that should be the behaviour, but it's what I wanted.

[ulysses4ever]

@ratherforky thanks a lot for carefully documenting your experience. Short of changing the default behavior (which will probably be too disruptive), what would have saved you is #8519 (display within which project file we build). What do you think? Unfortunately, that feature turned out to be tricky to integrate into our humongous and over-complicated testing infrastructure (what?? yes!!)

Unfortunately, --ignore-project currently doesn't work with build: #7057

[ratherforky]

Yeah I think that would've saved me a lot of time figuring out what the problem was. Thankfully adding a basic cabal.project to the subdir is a pretty easy workaround so it's not a big deal, just a papercut