Regression test T4025 fails (macOS)

[andreasabel]

The regression test for #4025 fails for me (macOS Mojave). The report is:

stderr:
*** Exception: Command /cabal-testsuite/PackageTests/Regression/T4025/setup.dist/work/./dist/build/exe/exe succeeded.
Output:
23

The intention of the test is to build an executable using --enable-executable-dynamic that is then not supposed to run because it is not supposed to find a dynamic component.

However, on my system the executable does run successfully and prints its output 23.

$ cd cabal-testsuite/PackageTests/Regression/T4025
$ echo "packages: ." > cabal.project
$ cabal run --enable-executable-dynamic
Resolving dependencies...
Build profile: -w ghc-8.10.6 -O1
In order, the following will be built (use -v for more details):
 - T4025-1.0 (lib) (first run)
 - T4025-1.0 (exe:exe) (first run)
Configuring library for T4025-1.0..
Preprocessing library for T4025-1.0..
Building library for T4025-1.0..
[1 of 1] Compiling A                ( A.hs, cabal-testsuite/PackageTests/Regression/T4025/dist-newstyle/build/x86_64-osx/ghc-8.10.6/T4025-1.0/build/A.o, cabal-testsuite/PackageTests/Regression/T4025/dist-newstyle/build/x86_64-osx/ghc-8.10.6/T4025-1.0/build/A.dyn_o )
Configuring executable 'exe' for T4025-1.0..
Preprocessing executable 'exe' for T4025-1.0..
Building executable 'exe' for T4025-1.0..
[1 of 1] Compiling Main             ( exe/Main.hs, cabal-testsuite/PackageTests/Regression/T4025/dist-newstyle/build/x86_64-osx/ghc-8.10.6/T4025-1.0/x/exe/build/exe/exe-tmp/Main.dyn_o )
Linking cabal-testsuite/PackageTests/Regression/T4025/dist-newstyle/build/x86_64-osx/ghc-8.10.6/T4025-1.0/x/exe/build/exe/exe ...
23

(The rest of the cabal-testsuite passes...)

The test script explains the purpose of the test:

cabal/cabal-testsuite/PackageTests/Regression/T4025/setup.test.hs

Lines 2 to 13 in 95a9241

	-- Test that we don't accidentally add the inplace directory to
	-- an executable RPATH. Don't test on Windows, which doesn't
	-- support RPATH.
	main = setupAndCabalTest $ do
	skipIfWindows
	setup "configure" ["--enable-executable-dynamic"]
	setup "build" []
	-- This should fail as it we should NOT be able to find the
	-- dynamic library for the internal library (since we didn't
	-- install it). If we incorrectly encoded our local dist
	-- dir in the RPATH, this will succeed.
	recordMode DoNotRecord . fails $ runExe "exe" []

Just to make sure, the generated exe does depend on the dylib:

$ otool -L ./dist-newstyle/build/x86_64-osx/ghc-8.10.6/T4025-1.0/x/exe/build/exe/exe
...
	@rpath/libHST4025-1.0-inplace-ghc8.10.6.dylib (compatibility version 0.0.0, current version 0.0.0)
...

I am however puzzled why such an innocent looking project should not just work with --enable-executable-dynamic:

cabal/cabal-testsuite/PackageTests/Regression/T4025/T4025.cabal

Lines 1 to 15 in 95a9241

	name: T4025
	version: 1.0
	build-type: Simple
	cabal-version: >= 1.10
	library
	default-language: Haskell2010
	build-depends: base
	exposed-modules: A
	executable exe
	default-language: Haskell2010
	build-depends: T4025, base
	hs-source-dirs: exe
	main-is: Main.hs

[Mikolaj]

There is definitely some chaos in this area, e.g., some flags are ignored (or reversed?) in v2- commands: #7605 [edit: wrong ticket, should be #7236]

[andreasabel]

There is definitely some chaos in this area, e.g., some flags are ignored (or reversed?) in v2- commands: #7605

At this PR, I didn't find anything I could recognize as connected to the OP.

Should T4025 be retired then, or do I have to configure my machine in a special way to get it passed?

[Mikolaj]

Uh, apologies, github is auto-suggesting when I press '#' and changing the number. The ticket is #7236.

I guess it would be great to get to the bottom of that and open a ticket [edit: doh, this is the ticket :)]. In particular, it would help understand why it passes in OSX CIs, but not on your machine.

[andreasabel]

#6688 could be related.

[gbaz]

I wonder if it has to do with the OS on the builders vs user machines. Just ran into this myself (big sur)

[andreasabel]

The last call to ghc that build the executable is:

/usr/local/bin/ghc-8.10.6
  --make
  -fbuilding-cabal-package
  -O
  -dynamic
  -fPIC

  <<settings about suffixes, directories, includes>>
  
  '-dynload deploy'
  -optl-Wl,-rpath,/Users/abel/bin/src/cabal/cabal-testsuite/PackageTests/Regression/T4025/setup.dist/usr/lib/x86_64-osx-ghc-8.10.6
  -optl-Wl,-rpath,/usr/local/lib/ghc-8.10.6/base-4.14.3.0
  -optl-Wl,-rpath,/usr/local/lib/ghc-8.10.6/ghc-prim-0.6.1
  -optl-Wl,-rpath,/usr/local/lib/ghc-8.10.6/integer-gmp-1.0.3.0
  -optl-Wl,-rpath,/usr/local/lib/ghc-8.10.6/rts

  <<settings about packages>>
  ...
  -package-id T4025-1.0
  ...

  exe/Main.hs
  -o setup.dist/work/dist/build/exe/exe

Linking setup.dist/work/dist/build/exe/exe ...

I wonder whether

  -optl-Wl,-rpath,/Users/abel/bin/src/cabal/cabal-testsuite/PackageTests/Regression/T4025/setup.dist/usr/lib/x86_64-osx-ghc-8.10.6

should be exactly not passed to the linker in this testcase, so that local dynlibs are not found.

[Mikolaj]

More information in #7988.