Missing package database if only installing boot packages

[noughtmare]

Describe the bug

I recently installed GHC 9.6.3 and now I ran the command cabal install --lib --package-env . base ghc. This creates a environment file like this:

clear-package-db
global-package-db
package-db /Users/jaro/.cabal/store/ghc-9.6.3/package.db
package-id base-4.18.1.0
package-id ghc-9.6.3

However, I did not actually have a package.db for ghc-9.6.3 yet and this command does not create it. So, now when I try to run GHC using this environment file I get the error:

ghc-9.6.3: can't find a package database at /Users/jaro/.cabal/store/ghc-9.6.3/package.db

I would expect that this command either would actually ensure that the package db is created (or alternatively leave out that package-db line from the environment file).

To work around this issue I can just run a command that does actually build a package, e.g.: cabal repl -b tasty.

System information

• Operating system: MacOS
• cabal 3.10.1.0, ghc 9.6.3

[andreabedini]

I can reproduce and confirm that cabal should not write an invalid environment file.

Mind if I ask: what is the use case of installing base and ghc in a local environment? They are always available to ghci.

[noughtmare]

HLS (outside a cabal or stack project) doesn't pick up on ghc without an environment file (and I believe there is/was also a bug that making an environment file without base resulted compiler errors so I've been adding it ever since). Maybe that is an HLS issue?

[andreabedini]

HLS (outside a cabal or stack project) doesn't pick up on ghc without an environment file (and I believe there is/was also a bug that making an environment file without base resulted compiler errors so I've been adding it ever since). Maybe that is an HLS issue?

It seems to work for me. But I had to admit I had to do some troubleshooting to get it to work. I had a stray .cabal file in a directory above which was confusing everything. I usually do haskell-language-server-wrapper Test.hs and hie-bios debug Test.hs to keep what is going on. There is also a way to specify manually how to call GHC, although the documentation seems a bit lacking.

❯ cat hie.yaml
cradle:
  direct:
    arguments: ["-XGADTs"]

Anyway, thank you for reporting this bug.
Also

[Mikolaj]

(and I believe there is/was also a bug that making an environment file without base resulted compiler errors so I've been adding it ever since)

BTW, the bug was #8894, fixed in 3.10.1.0 via a backport of #8903.

[Mikolaj]

And the PR claims "Also resolves #8825 by checking that filepaths of specificpackagedbs exist before adding them to the env file." so it should prevent the very issue that package ghc is added but not in the db, unless I'm mixing up the dbs in question. Are these the same paths and dbs as in this here ticket?

[noughtmare]

It seems to work for me.

So, you can run haskell-language-server-wrapper on a file like this:

-- Test.hs
import GHC.Tc.Plugin
main = putStrLn "Hello, World!"

After a few seconds, I'm getting an error like this:

2023-10-05T09:49:55.813808Z | Info | updateFileDiagnostics published different from new diagnostics - file diagnostics: File:     /private/tmp/Test.hs
Hidden:   no
Range:    1:8-1:21
Source:   not found
Severity: DiagnosticSeverity_Error
Message: 
  Could not load module ‘GHC.Tc.Plugin’
  It is a member of the hidden package ‘ghc-9.6.3’.
  You can run ‘:set -package ghc’ to expose it.
  (Note: this unloads all the modules in the current scope.)
Files that failed:
 * /private/tmp/Test.hs

I'll try hie-bios next, but it takes some time to install. Edit: hie-bios debug Test.hs produces this although I'm not sure how to interpret this:

[Info] invoking build tool to determine build flags (this may take some time depending on the cache)
[Debug] ghc --print-libdir
[Debug] ghc --numeric-version
Root directory:        /private/tmp
Component directory:   /private/tmp
GHC options:           
GHC library directory: CradleSuccess "/Users/jaro/.ghcup/ghc/9.6.3/lib/ghc-9.6.3/lib"
GHC version:           CradleSuccess "9.6.3"
Config Location:       No explicit config found
Cradle:                Cradle{ cradleRootDir = "/private/tmp", cradleOptsProg = CradleAction: Default}
Dependencies:    

[andreabedini]

I can reproduce with this script

❯ sh -c 'cd $(mktemp -d); ghc --version; cabal --version; cabal --store-dir $(mktemp -d) install --package-env . --lib base ghc; ghci; cat .ghc.environment.*'
The Glorious Glasgow Haskell Compilation System, version 9.4.7
cabal-install version 3.10.1.0
compiled using version 3.10.1.0 of the Cabal library
Resolving dependencies...
Loaded package environment from /tmp/tmp.SL2PriUZ9h/.ghc.environment.x86_64-linux-9.4.7
GHCi, version 9.4.7: https://www.haskell.org/ghc/  :? for help
ghc-9.4.7: can't find a package database at /tmp/tmp.YGNtPJSZyl/ghc-9.4.7/package.db
clear-package-db
global-package-db
package-db /tmp/tmp.YGNtPJSZyl/ghc-9.4.7/package.db
package-id base-4.17.2.0
package-id ghc-9.4.7

[andreabedini]

And the PR claims "Also resolves #8825 by checking that filepaths of specificpackagedbs exist before adding them to the env file." so it should prevent the very issue that package ghc is added but not in the db, unless I'm mixing up the dbs in question. Are these the same paths and dbs as in this here ticket?

Did we have test? As a mathematician once told me: untested code is wrong with probability 1-epsilon

[Mikolaj]

Yeah, no automated tests, though a couple of people manually tested it resolves their particular issues. This may just be yet another slightly different iteration of the problem needing a slightly different tweak.

[noughtmare]

@Mikolaj

BTW, the bug was #8894, fixed in 3.10.1.0 via a backport of #8903.

That does not seem right, the bug was reported against 3.10.1.0 (and I can also reproduce it myself with 3.10.1.0), so I don't think there has been a release yet with a fix.

[Mikolaj]

@noughtmare: good catch! So it's backported to 3.10 branch, but will only we released in 3.10.2.0. Does 3.10.2.0 fix it for you? An pre-release can be obtained with "ghcup install cabal -u 'https://downloads.haskell.org/~cabal/cabal-install-3.10.2.0/cabal-install-3.10.2.0-x86_64-linux-alpine.tar.xz' 3.10.2.0-a1".

[noughtmare]

@Mikolaj yes that version works. Although I noticed it reports itself as being 3.8.1.0 and it shows warnings like this:

Warning: Unknown/unsupported 'ghc' version detected (Cabal 3.8.1.0 supports
'ghc' version < 9.6): /Users/jaro/.ghcup/bin/ghc is version 9.6.3

Edit: Oh, I'm using MacOS and that version doesn't seem to work on my platform so it selects 3.8.1.0 as a fallback. And I'm using NixOS on my other PC, so I can't test it there either.

Edit 2: I've now installed cabal-install-3.10.2.0-x86_64-darwin.tar.xz and that does indeed also work.

[Mikolaj]

Apologies, that was Linux-centric of me. But I'm confused: did you say above that 3.8.1.0 worked in this attempt? Is the bug non-deterministic?

[noughtmare]

3.8.1.0 indeed does seem to work. I think the bug was introduced in 3.10.

[Mikolaj]

Oh, I see, thank you. Works in 3.8.*, fails in 3.10.1.0, works again in 3.10.2.0.