cmd/link: wrong c-archive architecture using GNU binutils ar on macOS Mojave 10.14.1

[DenWav]

What version of Go are you using (go version)?

$ go version
go version go1.11.2 darwin/amd64

Does this issue reproduce with the latest release?

Yes

What operating system and processor architecture are you using (go env)?

go env Output

$ go env
GOARCH="amd64"
GOBIN=""
GOCACHE="/Users/kyle/Library/Caches/go-build"
GOEXE=""
GOFLAGS=""
GOHOSTARCH="amd64"
GOHOSTOS="darwin"
GOOS="darwin"
GOPATH="/Users/kyle/go"
GOPROXY=""
GORACE=""
GOROOT="/usr/local/go"
GOTMPDIR=""
GOTOOLDIR="/usr/local/go/pkg/tool/darwin_amd64"
GCCGO="gccgo"
CC="clang"
CXX="clang++"
CGO_ENABLED="1"
GOMOD=""
CGO_CFLAGS="-g -O2"
CGO_CPPFLAGS=""
CGO_CXXFLAGS="-g -O2"
CGO_FFLAGS="-g -O2"
CGO_LDFLAGS="-g -O2"
PKG_CONFIG="pkg-config"
GOGCCFLAGS="-fPIC -m64 -pthread -fno-caret-diagnostics -Qunused-arguments -fmessage-length=0 -fdebug-prefix-map=/var/folders/vw/ht92mdqd1f54_c6ys42ltc5c0000gn/T/go-build035928823=/tmp/go-build -gno-record-gcc-switches -fno-common"

What did you do?

Following the instructions at https://golang.org/doc/install/source, with the latest 1.11.2 darwin/amd64 Go binary distribution as the bootstrap toolchain:

git clone https://go.googlesource.com/go gosrc
cd gosrc
git checkout go1.11.2
cd src
./all.bash

What did you expect to see?

Go project should build successfully.

What did you see instead?

Error log: https://gist.github.com/DemonWav/8caf4fbdf9642133f53502614a231a24

Issues occur with ##### ../misc/cgo/testcarchive. I believe the issue is with this line:

ld: warning: ignoring file pkg/darwin_amd64/libgo.a, file was built for archive which is not the architecture being linked (x86_64): pkg/darwin_amd64/libgo.a

Since the architecture doesn't match, it doesn't link, and the linker can't find the missing symbols.

If I instead ask for the 386 architecture (GOARCH=386 ./all.bash), it builds successfully: https://gist.github.com/DemonWav/533894d4426e6a9dc7cc84cc613ee66d

The host is still recognized as amd64 in the build log, though.

Building packages and commands for host, darwin/amd64.
Building packages and commands for target, darwin/386

It appears libgo is always compiled as i386 architecture, even when both host and target are defined as amd64, GOARCH=amd64 GOHOSTARCH=amd64 ./all.bash gives the same result.

I'm not sure if there's something wrong with my environment or what, but I've asked on Slack and IRC and no one seems to know. I'm sorry if this isn't an issue.

[ianlancetaylor]

What ar program is first on your PATH?

[DenWav]

My ar is the GNU utils command from the binutils package from Homebrew:

┌─┤✓├─┤kyle├─┤~│
└──▶ ar --version
GNU ar (GNU Binutils) 2.31.1
Copyright (C) 2018 Free Software Foundation, Inc.
This program is free software; you may redistribute it under the terms of
the GNU General Public License version 3 or (at your option) any later version.
This program has absolutely no warranty.

┌─┤✓├─┤kyle├─┤~│
└──▶ which ar
/usr/local/bin/ar

┌─┤✓├─┤kyle├─┤~│
└──▶ ls -l /usr/local/bin/ar
lrwxr-xr-x 1 kyle admin 34 Nov  9 10:01 /usr/local/bin/ar -> ../Cellar/binutils/2.31.1_2/bin/ar

┌─┤✓├─┤kyle├─┤~│
└──▶ /usr/bin/ar --version
usage:  ar -d [-TLsv] archive file ...
	ar -m [-TLsv] archive file ...
	ar -m [-abiTLsv] position archive file ...
	ar -p [-TLsv] archive [file ...]
	ar -q [-cTLsv] archive file ...
	ar -r [-cuTLsv] archive file ...
	ar -r [-abciuTLsv] position archive file ...
	ar -t [-TLsv] archive [file ...]
	ar -x [-ouTLsv] archive [file ...]

So if I prioritize the default utils instead, it actually does work:

$ PATH="/usr/bin:$PATH" ./all.bash
.
.
.
ALL TESTS PASSED
---
Installed Go for darwin/amd64 in /Users/kyle/gosrc
Installed commands in /Users/kyle/gosrc/bin
*** You need to add /Users/kyle/gosrc/bin to your PATH.

So something about the GNU utils doesn't work with the Go build. I can keep that in mind, but it would be really nice if that could be supported directly, if possible.

[ianlancetaylor]

Could you examine the output of the system default ar program and the GNU binutils ar program and see what the difference is? Is there some option we could pass to the GNU ar program that will make it do the right thing?

[DenWav]

I'm not familiar with ar, sorry. Some quick research didn't reveal anything interesting, they seem to be supposed to do the same thing. It's easier for me to just uninstall the binutils package, I only had it installed for the sake of using GNU utils, but I don't need it directly. I don't know if you want to keep the issue open for the sake of supporting GNU utils on macOS or just close it for now, so I'll leave that to you.

[ianlancetaylor]

I will leave this open in the hopes that someone using MacOS can look into it. Thanks.

[liuzheng]

same issue
MacOS : 10.14.6 (18G87)
$ go version
go version go1.12.7 darwin/amd64

I am using golang(1.12.7) with pkg installed to build golang(1.12.7) from source code

##### ../misc/cgo/testcarchive
--- FAIL: TestInstall (8.92s)
    carchive_test.go:214: [go install -i -buildmode=c-archive libgo]
    carchive_test.go:214: [clang -fPIC -m64 -pthread -fno-caret-diagnostics -Qunused-arguments -fmessage-length=0 -fdebug-prefix-map=/var/folders/0d/8cgyz4bx1s771vg_611m4cfm0000gn/T/go-build071877855=/tmp/go-build -gno-record-gcc-switches -fno-common -framework CoreFoundation -framework Foundation -I pkg/darwin_amd64 -o ./testp1 main.c main_unix.c pkg/darwin_amd64/libgo.a]
    carchive_test.go:214: main_unix.c:39:22: warning: result of comparison against a string literal is unspecified (use strncmp instead) [-Wstring-compare]
        ld: warning: ignoring file pkg/darwin_amd64/libgo.a, file was built for archive which is not the architecture being linked (x86_64): pkg/darwin_amd64/libgo.a
        Undefined symbols for architecture x86_64:
          "_CheckArgs", referenced from:
              _main in main-af3d9d.o
          "_DidInitRun", referenced from:
              _main in main-af3d9d.o
          "_DidMainRun", referenced from:
              _main in main-af3d9d.o
          "_FromPkg", referenced from:
              _main in main-af3d9d.o
        ld: symbol(s) not found for architecture x86_64
        clang: error: linker command failed with exit code 1 (use -v to see invocation)
    carchive_test.go:214: exit status 1
--- FAIL: TestEarlySignalHandler (4.62s)
    carchive_test.go:259: ld: warning: ignoring file libgo2.a, file was built for archive which is not the architecture being linked (x86_64): libgo2.a
        Undefined symbols for architecture x86_64:
          "_GoRaiseSIGPIPE", referenced from:
              _main in main2-18be2a.o
          "_RunGoroutines", referenced from:
              _main in main2-18be2a.o
          "_TestSEGV", referenced from:
              _main in main2-18be2a.o
        ld: symbol(s) not found for architecture x86_64
        clang: error: linker command failed with exit code 1 (use -v to see invocation)
    carchive_test.go:260: exit status 1
--- FAIL: TestSignalForwarding (3.33s)
    carchive_test.go:292: main5.c:42:25: warning: indirection of non-volatile null pointer will be deleted, not trap [-Wnull-dereference]
        main5.c:42:25: note: consider using __builtin_trap() or qualifying pointer with 'volatile'
        ld: warning: ignoring file libgo2.a, file was built for archive which is not the architecture being linked (x86_64): libgo2.a
        Undefined symbols for architecture x86_64:
          "_Noop", referenced from:
              _main in main5-b6fac3.o
        ld: symbol(s) not found for architecture x86_64
        clang: error: linker command failed with exit code 1 (use -v to see invocation)
    carchive_test.go:293: exit status 1
--- FAIL: TestSignalForwardingExternal (2.86s)
    carchive_test.go:336: main5.c:42:25: warning: indirection of non-volatile null pointer will be deleted, not trap [-Wnull-dereference]
        main5.c:42:25: note: consider using __builtin_trap() or qualifying pointer with 'volatile'
        ld: warning: ignoring file libgo2.a, file was built for archive which is not the architecture being linked (x86_64): libgo2.a
        Undefined symbols for architecture x86_64:
          "_Noop", referenced from:
              _main in main5-d45a27.o
        ld: symbol(s) not found for architecture x86_64
        clang: error: linker command failed with exit code 1 (use -v to see invocation)
    carchive_test.go:337: exit status 1
--- FAIL: TestOsSignal (4.29s)
    carchive_test.go:452: ld: warning: ignoring file libgo3.a, file was built for archive which is not the architecture being linked (x86_64): libgo3.a
        Undefined symbols for architecture x86_64:
          "_CatchSIGIO", referenced from:
              _main in main3-1c2445.o
          "_ProvokeSIGPIPE", referenced from:
              _main in main3-1c2445.o
          "_ResetSIGIO", referenced from:
              _main in main3-1c2445.o
          "_SawSIGIO", referenced from:
              _main in main3-1c2445.o
        ld: symbol(s) not found for architecture x86_64
        clang: error: linker command failed with exit code 1 (use -v to see invocation)
    carchive_test.go:453: exit status 1
--- FAIL: TestSigaltstack (4.54s)
    carchive_test.go:488: ld: warning: ignoring file libgo4.a, file was built for archive which is not the architecture being linked (x86_64): libgo4.a
        Undefined symbols for architecture x86_64:
          "_GoCatchSIGIO", referenced from:
              _main in main4-3d73de.o
          "_GoRaiseSIGIO", referenced from:
              _thread2 in main4-3d73de.o
          "_SIGIOCount", referenced from:
              _thread1 in main4-3d73de.o
              _thread2 in main4-3d73de.o
        ld: symbol(s) not found for architecture x86_64
        clang: error: linker command failed with exit code 1 (use -v to see invocation)
    carchive_test.go:489: exit status 1
--- FAIL: TestCompileWithoutShared (3.89s)
    carchive_test.go:688: [go build -buildmode=c-archive -gcflags=-shared=false -o libgo2.a libgo2]
    carchive_test.go:690:
    carchive_test.go:704: [clang -fPIC -m64 -pthread -fno-caret-diagnostics -Qunused-arguments -fmessage-length=0 -fdebug-prefix-map=/var/folders/0d/8cgyz4bx1s771vg_611m4cfm0000gn/T/go-build071877855=/tmp/go-build -gno-record-gcc-switches -fno-common -framework CoreFoundation -framework Foundation -I pkg/darwin_amd64 -o ./testnoshared -no-pie main5.c libgo2.a]
    carchive_test.go:720: main5.c:42:25: warning: indirection of non-volatile null pointer will be deleted, not trap [-Wnull-dereference]
        main5.c:42:25: note: consider using __builtin_trap() or qualifying pointer with 'volatile'
        ld: warning: ignoring file libgo2.a, file was built for archive which is not the architecture being linked (x86_64): libgo2.a
        Undefined symbols for architecture x86_64:
          "_Noop", referenced from:
              _main in main5-d3ec3d.o
        ld: symbol(s) not found for architecture x86_64
        clang: error: linker command failed with exit code 1 (use -v to see invocation)
    carchive_test.go:722: exit status 1
FAIL
2019/08/12 22:23:47 Failed: exit status 1