cmd/go: macOS on arm64 requires codesigning

[FiloSottile]

On the production Apple Silicon machines, Go binaries are killed at start. #38485 (comment)

It looks like all binaries need to be codesigned now, and indeed running codesign -s - on them lets them run correctly.

This stops go test and go run from working, and requires an extra step after go build to get a functional binary.

This also affects the bootstrapped compiler itself.

[FiloSottile]

@cherrymui indeed, binaries produced by the machine's clang are codesigned, in what looks like the same exact way as the output of codesign -s -.

filippo@Filippos-MacBook-Pro tmp % clang hello.c
filippo@Filippos-MacBook-Pro tmp % ./a.out
Hello M1!
filippo@Filippos-MacBook-Pro tmp % codesign -d -v a.out                                 
Executable=/Users/filippo/tmp/a.out
Identifier=a.out
Format=Mach-O thin (arm64)
CodeDirectory v=20400 size=510 flags=0x20002(adhoc,linker-signed) hashes=13+0 location=embedded
Signature=adhoc
Info.plist=not bound
TeamIdentifier=not set
Sealed Resources=none
Internal requirements=none

[FiloSottile]

Actually, codesign outputs don't have the linker-signed flag.

[cherrymui]

Thanks, @FiloSottile !

Interesting. The C compiler on the DTK doesn't do that...

Could you try one more thing: run clang -v hello.c to see if the C compiler invokes codesign or how it signs the binary? Thanks!

[FiloSottile]

It doesn't look like the linker invocation has anything special, so I assume ld just does that (and @jedisct1 said so on Twitter as well https://twitter.com/jedisct1/status/1328862207715794946).

"/Library/Developer/CommandLineTools/usr/bin/ld" -demangle -lto_library /Library/Developer/CommandLineTools/usr/lib/libLTO.dylib -no_deduplicate -dynamic -arch arm64 -platform_version macos 11.0.0 11.0 -syslibroot /Library/Developer/CommandLineTools/SDKs/MacOSX.sdk -o a.out -L/usr/local/lib /var/folders/jh/3ydm4lxd71s2__g_x4hny6r00000gn/T/hello-580ca7.o -lSystem /Library/Developer/CommandLineTools/usr/lib/clang/12.0.0/lib/darwin/libclang_rt.osx.a

I expect external linking should generate codesigned binaries, but trying `go build -ldflags="-linkmode=external" fails immediately with

# tmp
loadinternal: cannot find runtime/cgo

[cherrymui]

loadinternal: cannot find runtime/cgo

This is not a failure. It prints a message but it should successfully link the binary.

[FiloSottile]

You're right, I do get the binary. Interestingly, it comes out linker-signed but doesn't run, and codesign doesn't work on it.

filippo@Filippos-MacBook-Pro tmp % codesign -d -v tmp                                                       
Executable=/Users/filippo/tmp/tmp
Identifier=a.out
Format=Mach-O thin (arm64)
CodeDirectory v=20400 size=7038 flags=0x20002(adhoc,linker-signed) hashes=217+0 location=embedded
Signature=adhoc
Info.plist=not bound
TeamIdentifier=not set
Sealed Resources=none
Internal requirements=none
filippo@Filippos-MacBook-Pro tmp % ./tmp             
zsh: killed     ./tmp
filippo@Filippos-MacBook-Pro tmp % codesign -s - tmp 
tmp: the codesign_allocate helper tool cannot be found or used

[cherrymui]

Thanks!

Interesting... I'll see if it is possible to generate LC_CODE_SIGNATURE in the linker. If that's not possible, maybe shell out codesign.

Any down side for that? If the user wants to sign with a different identity, would that still be possible?

[ianlancetaylor]

I don't understand this. What is the point of code signing, if every execution of clang or the Go compiler produces a code signed binary?

[FiloSottile]

Any down side for that?

I guess the main problem would be with cross-compilation, but anyway I guess adhoc signatures only work on the same machine where they are generated.

This would be a major pain for projects distributing binaries. I wonder how Homebrew deals with it, since by default they install pre-built "Bottles".

What is the point of code signing, if every execution of clang or the Go compiler produces a code signed binary?

I don't know, but I can imagine 1) to normalize the execution path such that it always check signatures unconditionally, simplifying it and 2) to block binaries generated on other machines (and not properly signed by a trusted Developer ID) which before was sort of the role of the quarantine attributes.

[terinjokes]

You're right, I do get the binary. Interestingly, it comes out linker-signed but doesn't run, and codesign doesn't work on it.

My understanding is this is a bug already reported to Apple, see Homebrew/brew#9082 (comment)

I wonder how Homebrew deals with it, since by default they install pre-built "Bottles".

Per Homebrew/brew#7857 (comment) I don't think they're close to code signing bottles on ARM yet.

[rolandshoemaker]

2. to block binaries generated on other machines (and not properly signed by a trusted Developer ID) which before was sort of the role of the quarantine attributes.

Looks like binaries aren't required to have a signature linked to a Developer ID if they were signed elsewhere. I was able to apply an ad-hoc signature to a binary on my Intel mac and transfer it to my M1 mac and it ran just fine. Appears like binaries just need a signature, doesn't really seem to matter where it came from.

[fxcoudert]

Homebrew maintainer here:

the codesign_allocate helper tool cannot be found or used

That's indeed a bug in codesign, and Apple is aware

My understanding is this is a bug already reported to Apple, see Homebrew/brew#9082 (comment)

Yes. If this occurs, you need to change that file's inode (so copying it someplace and back will work). Once you've done that, you can sign and it will work. See Homebrew/brew#9102 for details.

I don't think they're close to code signing bottles on ARM yet

We do not distribute bottles yet, because our CI is not yet fully operational, but our codebase is otherwise ready. We apply ad-hoc signatures as part of formula installation: Homebrew/brew#9102

[fxcoudert]

As for go, if Go uses its own linker (or anything other than Apple's ld) then it definitely needs to call codesign -s - on the binaries it produces: executables, shared libraries, etc.

[mislav]

@FiloSottile (and others): thank you for the information! 🙇

Do you know if it is possible to use darwin/amd64 architecture to cross-compile binaries that are able to execute on darwin/arm64, providing that the binaries are explicitly signed?

I am interested in what are the options to distribute binaries for Go projects that work on Apple Silicon without having to actually compile them on Apple Silicon.

[bwesterb]

@mislav Cross-compiling for darwin/arm64 works fine from darwin/amd64 and even linux/amd64. Obviously the resulting binaries are not signed. They will run if signed ad hoc on the same machine they will run. Binaries ad hoc signed on one machine didn't work on another machine for me as well.

[jameshartig]

Binaries ad hoc signed on one machine didn't work on another machine for me.

This comment seems to disagree: #42684 (comment)

I was able to apply an ad-hoc signature to a binary on my Intel mac and transfer it to my M1 mac and it ran just fine.

[bwesterb]

Ok, I tested it again and now cross-signing does work. I might've transferred the wrong file earlier. Whoops.

[FiloSottile]

Codesigning from cmd/link should be possible. https://github.com/isignpy/isign by @neilk does that for iOS.

[networkimprov]

The Zig project is also working on this here: ziglang/zig#7103 (thanks @komuw).

[cherrymui]

Thanks for the pointers. I'll look into them.

[fxcoudert]

Binaries ad hoc signed on one machine didn't work on another machine for me

I can confirm that a linker or other ad hoc signature is sufficient to run the binary (or shared library) on any machine.

[jimtut]

You can do that all on the m1 mac by replacing the last line with
arch --x86_64 env GODEBUG=asyncpreemptoff=1 GOOS=darwin GOARCH=arm64 ./bootstrap.bash

This doesn't see true. I'm on an M1 Mac mini, and these commands:

git clone https://go.googlesource.com/go
cd go/src
arch --x86_64 env GODEBUG=asyncpreemptoff=1 GOOS=darwin GOARCH=arm64 ./bootstrap.bash

give an error:

ERROR: Cannot find /Users/jim/go1.4/bin/go.
Set $GOROOT_BOOTSTRAP to a working Go tree >= Go 1.4.

UPDATE: Piecing together various bits and bobs above, this hung the first time, but worked the second:

1. Download go1.15.6.darwin-amd64.pkg from https://golang.org/doc/install
2. Run the PKG file. It's x86, but Rosetta 2 will run the installer, and place an x86 Go install in /usr/local/go
3. git clone https://go.googlesource.com/go
4. cd go/src
5. arch --x86_64 env GOROOT_BOOTSTRAP=/usr/local/go GODEBUG=asyncpreemptoff=1 GOOS=darwin GOARCH=arm64 ./bootstrap.bash

[fayep]

You do need an x86_64 darwin golang to cross build for arm64. It is easiest at this point to get that and then use gotip to build master.

…

On Thu, Dec 17, 2020 at 12:31 PM jimtut ***@***.***> wrote: You can do that all on the m1 mac by replacing the last line with arch --x86_64 env GODEBUG=asyncpreemptoff=1 GOOS=darwin GOARCH=arm64 ./bootstrap.bash This doesn't see true. I'm on an M1 Mac mini, and these commands: git clone https://go.googlesource.com/go cd go/src arch --x86_64 env GODEBUG=asyncpreemptoff=1 GOOS=darwin GOARCH=arm64 ./bootstrap.bash give an error: ERROR: Cannot find /Users/jim/go1.4/bin/go. Set $GOROOT_BOOTSTRAP to a working Go tree >= Go 1.4. It doesn't seem like you can *bootstrap* on an M1 Mac (no Go installed anywhere on this fresh Mac)? — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#42684 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AAEKAP2BPXHPN7K3DUYJ2X3SVJTCHANCNFSM4TZJN5QA> .

[dmitshur]

You can now download a native macOS ARM64 binary release from https://golang.org/dl/#go1.16beta1, as it was released today. Look for go1.16beta1.darwin-arm64.pkg or go1.16beta1.darwin-arm64.tar.gz. Then you can use Go 1.16 Beta 1 itself, or use it to build a newer gotip version.