Skip to content

cmd/gomobile: xcframework and catalyst support #64

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
wants to merge 1,082 commits into from

Conversation

imWildCat
Copy link

@imWildCat imWildCat commented Mar 22, 2021

Talked with @waylybaye offline, I decided to continue #63 and #45.

@waylybaye:

I changed the build process and it will create 3 frameworks for ios, simulator and catalyst. Every framework has a fat library contains both arm64 and x86_64 (except ios that only needs arm64).

Finally create one xcframework from all 3 fat frameworks and this xcframework contains every platform and architecture.

Review comments at https://go-review.googlesource.com/c/mobile/+/288752 should be mostly resolved.

Elias Naur and others added 30 commits November 3, 2016 14:21
Change-Id: Icbec67cbc000e4770ccc82a95a1a9ce1e22bbc18
Reviewed-on: https://go-review.googlesource.com/32592
Reviewed-by: Hyang-Ah Hana Kim <[email protected]>
Update outdated documentation and add documentation and references
for the the new reverse binding support.

Add a new example, reverse, that demonstrates how to use Android
API directly from Go and thereby implement an Android app in pure
Go.

Change-Id: Iac054dbc015d00a37c0cc234931f9bd90172848e
Reviewed-on: https://go-review.googlesource.com/31170
Reviewed-by: David Crawshaw <[email protected]>
REAME shows an example like this:

```
gobind {
  // Package to bind. Separate multiple packages with spaces.
  pkg "github.com/someone/somepackage"

  // GOPATH
  GOPATH "/home/gopher"

  // Optional list of architectures. Defaults to all supported architectures.
  GOARCH="arm amd64"

  // Absolute path to the gomobile binary. Optional.
  GOMOBILE "/mypath/bin/gomobile"

  // Absolute path to the gomobile binary. Optional.
  GOBIND "/mypath/bin/gobind"

  // Absolute path to the go binary. Optional.
  GO "/usr/local/go/bin/go"

  // Pass extra parameters to command line. Optional.
  GOMOBILEFLAGS "-javapkg my.java.package"
}
```

But this example is wrong because each settings need `=` between a key and value.

Fixes golang/go#17996

Change-Id: I0f7acca78aee1032a67c35308b4ce78f97b1aa43
Reviewed-on: https://go-review.googlesource.com/33372
Reviewed-by: David Crawshaw <[email protected]>
…er.language=en

In Japanese environment, `javap` report errors in Japanese.

```
$ /usr/bin/javap -s -protected -constants com.example.ExampleClass
エラー: クラスが見つかりません: com.example.ExampleClass
```

But `java.Importer.Import` handles errors using `javap`'s output in
English.

So `javap` should be called with `-J-Duser.language=en` option.

Fixes golang/go#17987

Change-Id: I63d30cd49446523e54df2beae8d3f09f9de9bca5
Reviewed-on: https://go-review.googlesource.com/33371
Reviewed-by: Elias Naur <[email protected]>
The recent change to specify the Java bootclasspath for the Java
importer wasn't propagated to the test. Fix that.

Change-Id: I740fb14e17b4406fe0ccf97d46f28852e0630b5d
Reviewed-on: https://go-review.googlesource.com/33393
Reviewed-by: David Crawshaw <[email protected]>
The Go runtime doesn't handle SIGIPE signals from writing to closed
sockets or pipes in c-archive and c-shared mode (issue 17393).
Work around it in gomobile by simply ignoring all SIGPIPE signals;
they're not useful in mobile apps anyway.

Change-Id: Ibd7ee41058856c5eddb4a519345a3851a29e9b44
Reviewed-on: https://go-review.googlesource.com/33771
Reviewed-by: David Crawshaw <[email protected]>
The current iOS binding generator only generates returns if the
function being bound does not return an error. If a second error
return type is also present, the binder always generates both the
primary as well as the error as an output parameter.

This is undersirable because most decent functions in Go will
also return errors, so all of those get converted to plain methods
iOS side, each of them requiring allocating the return variable
first and only then execute the call. This gets even more annoying
with the Swift error wrapping protocol which converts errors to
throw statements automatically, but which still needs the ugly pre-
allocs caused by the genrated bindings not returning the result,
just placing it in an output argument.

This CL changes that so that if a nullable result is being returned
by a bound method from Go, then it is generated as a proper return
and not an output argument. This allows erroring functions to still
be called as a function in ObjC, and even more elegantly drop even
the error part in Swift.

Change-Id: I35152d7d2fd2a132eba836fa23be8fd4f317f097
Reviewed-on: https://go-review.googlesource.com/34072
Reviewed-by: Elias Naur <[email protected]>
Since the Go package name is already prefixed to generated ObjC
names, the empty extra prefix is useful. Support that by not reverting
to the default extra prefix, "Go", if -prefix "" is specified.

To avoid file name clashes with the Go header files, add ".objc" to
the ObjC-facing header names.

Change-Id: I559fe60d7474521617f23894af247c6019ff2a21
Reviewed-on: https://go-review.googlesource.com/33954
Reviewed-by: Hyang-Ah Hana Kim <[email protected]>
Since generated names now have their package names prefixed, the
extra prefix, "Go", is both confusing and counter-productive to
making the generated ObjC code look like any other native code.

Change the default to the empty prefix, while preserving support
for an explicit prefix if needed.

This is a backwards incompatible change; to keep the old behaviour,
specify "-prefix Go" to the gobind or gomobile command.

While we're here, fix the Ivy example for the recent change in
error returns.

Change-Id: I7fef4a92a18ddadee972ccf359652e3b31624f33
Reviewed-on: https://go-review.googlesource.com/34643
Reviewed-by: Hyang-Ah Hana Kim <[email protected]>
The return value of a JNI call is undefined when an exception was
raised during the call. To make sure comparisons with NULL works,
clear the value when an exception is raised.

No new tests; some devices, like the Samsung S2, crashes with the
existing tests without this CL.

Change-Id: I85eb983e9444fff1f05e0f83a0640d106280e54d
Reviewed-on: https://go-review.googlesource.com/34631
Reviewed-by: Hyang-Ah Hana Kim <[email protected]>
Change-Id: I1e74a7475ee3af491605e289685cda001ec80bb6
Reviewed-on: https://go-review.googlesource.com/34647
Reviewed-by: David Crawshaw <[email protected]>
Before this CL, using the reverse bindings to access two Java packages
with the same last component would fail with a duplicate package import
error. This CL renames generated import statements to use unique
aliases.

Change-Id: I94696281e58f011f45811445cf81aea02af69c4f
Reviewed-on: https://go-review.googlesource.com/34774
Reviewed-by: David Crawshaw <[email protected]>
Add support for using the Go AST package to find exported Go structs
that embed prefixed types. This is needed for a later CL to generate
wrappers for Go generated Java classes.

Change-Id: Ia304a7924a4e09332b74dc42a572932b7498cdca
Reviewed-on: https://go-review.googlesource.com/34775
Reviewed-by: David Crawshaw <[email protected]>
Before this CL, the type of the implicit "this" parameter to Java methods
implemented in Go could only be a super class of the generated Java
class. For example, the following GoRunnable type is an implementation of
the Java interface java.lang.Runnable with a toString method:

package somepkg

import "Java/java/lang"

type GoRunnable struct {
    lang.Runnable
}

func (r *GoRunnable) ToString(this lang.Runnable) string {
    ...
}

The "this" parameter is implicit in the sense that the reverse generator
automatically fills it with a reference to the Java instance of
GoRunnable.

Note that "this" has the type Java/java/lang.Runnable, not
Java/go/somepkg.GoRunnable, which renders it impossible to call Java
methods and functions that expect GoRunnable. The most practical example
of this is the Android databinding libraries.

This CL changes the implicit this parameter to always match the exact
type. In the example, the toString implementation becomes:

import gopkg "Java/go/somepkg"

func (r *GoRunnable) ToString(this gopkg.GoRunnable) string {
    ...
}

One strategy would be to simply treat the generated Java classes
(GoRunnable in our example) as any other Java class and import it
through javap. However, since the Java classes are generated after
importing, this present a chicken-and-egg problem.

Instead, use the newly added support for structs with embedded prefixed types
and synthesize class descriptors for every exported Go struct type.

Change-Id: Ic5ce4a151312bd89f91798ed4088c9959225b448
Reviewed-on: https://go-review.googlesource.com/34776
Reviewed-by: David Crawshaw <[email protected]>
Before this CL, Java types that were only implicitly referenced
were represented as interface{}. However, if a value of such an
implicit type were passed to Java, a runtime crash would occur
because there would be no wrapper class to unwrap.

Fix this by generating implicit types, fixing the crashes,
gaining type safety, and removing the interface{} special case in
the generator.

While we're here, remove a redundant insert to the clsMap map in
java.go.

Change-Id: Ic50125da3d7cd6075899bf628d419b084c630490
Reviewed-on: https://go-review.googlesource.com/34777
Reviewed-by: David Crawshaw <[email protected]>
The Objective-C bindings was recently changed to support the empty
name prefix and to use that as the default. This CLs changed the Java
generators in the same way, supporting the empty Java package and using
it as the default.

Change-Id: I857affce686c67638a2b6c4e1da5d6a88d7ba560
Reviewed-on: https://go-review.googlesource.com/34778
Reviewed-by: David Crawshaw <[email protected]>
This is the Objective-C equivalent of CL 34776, generating reverse
wrappers for generated ObjC types. The implementation follows the same
strategy as the Java implementation: use the Go ast package to find
exported structs with embedded Objective-C types and synthesize their
types as if they were imported through clang.

In turn, the handling of the implicit "self" parameter changes in the
same way as well: the type of self parameters must be the wrapped type
for the generated type. For example:

func (d *GoNSDate) Description(self Foundation.NSDate) string

becomes

import gopkg "ObjC/Objcpkg"

func (d *GoNSDate) Description(self gopkg.GoNSDate) string

Change-Id: I26f838b06a622864be463f81dbb4dcae76f70f20
Reviewed-on: https://go-review.googlesource.com/34780
Reviewed-by: David Crawshaw <[email protected]>
Change-Id: I681ac02869e137f67880416743352951960f7d8a
Reviewed-on: https://go-review.googlesource.com/33950
Reviewed-by: David Crawshaw <[email protected]>
The gomobile tool mishandled build tags in two ways, first by
ignoring tags for iOS, second by passing multiple tags along to
the go tool incorrectly. This CL fixes both.

Fixes golang/go#18523
Fixes golang/go#18515

Change-Id: I28a49c1e23670adb085617d9f5fb5cd5e22a4b65
Reviewed-on: https://go-review.googlesource.com/34955
Reviewed-by: David Crawshaw <[email protected]>
This is the Objective-C version of a similar change to the Java
generaters, CL 34777.

This CL makes sure the types only implicitly referenced through method
parameters or return values are also reverse generated. In doing so,
the anonymous interface{} is never used by the reverse generator,
gaining type safety, removing a special case and making sure passing
values of implicit types can be passed across the language barrier.

To support implicit types, the Objective-C importer is changed to
extract the module of a type from the clang AST dump instead of
relying on the module implied by the Go reference (e.g.
ObjC/Foundation.NSString).

Change-Id: Ie9305f4cd9a9802decbd93f81cec84dd05af11ab
Reviewed-on: https://go-review.googlesource.com/34991
Reviewed-by: David Crawshaw <[email protected]>
The tests was missing the generated code for the universe package
and some parts of the reverse generated code. Include it.

Change-Id: Id5e2f215c8f6f717c30377965255c4b64f31e923
Reviewed-on: https://go-review.googlesource.com/34992
Reviewed-by: David Crawshaw <[email protected]>
Make the golang.org/x/mobile/bind/java buildable without CGO, and
replace the which tool with exec.LookPath. Both helps on Windows that
often don't have a C compiler available and no which command.

Found while testing external NDK use on Windows.

Change-Id: I6d3311aae3fa97acb61b5ab9bed334e4a608c386
Reviewed-on: https://go-review.googlesource.com/35174
Reviewed-by: Hyang-Ah Hana Kim <[email protected]>
Use Context.getApplicationInfo().nativeLibraryDir to locate
libopenal.so. That path is always correct, even on newer devices.

Found while testing external NDK use on Windows.

Manually tested; there are no existing OpenAL tests or examples.

Change-Id: Ie204a7d7139566f85c9121126722ad597f9d6b19
Reviewed-on: https://go-review.googlesource.com/35175
Reviewed-by: Hyang-Ah Hana Kim <[email protected]>
The filepath package works on native paths, not package paths. Replace
it with the path package. Do it for Objective-C as well, for correctness.

No new tests; the existing reverse tests fails on Windows without
this CL.

Change-Id: I8963db992d4bed30e8828579fb83ecaf8c9d9ade
Reviewed-on: https://go-review.googlesource.com/35176
Reviewed-by: Hyang-Ah Hana Kim <[email protected]>
This CL restructures function generation to match the way methods
are generated, to avoid two different code paths for a coming CL.

Change-Id: I5a4f15e51ea5df101f9aa419ed4170ab36506418
Reviewed-on: https://go-review.googlesource.com/35185
Reviewed-by: David Crawshaw <[email protected]>
Before this CL, calling overloaded methods on reverse bound Java
classes and interfaces involved confusing and ugly name mangling.
If a set of methods with the same name differed only in argument count,
the mangling was simply adding the argument count to the name:

func F()
func F1(int32)

But if two or more methods had the same number of arguments, the type
had to be appended:

func (...) F() int32
func (...) F1(int32) (int32, error)
func (...) F__I(int32, int32)
func (...) F__JLjava_util_concurrent_TimeUnit_2(int64, concurrent.TimeUnit)

This CL sacrifices a bit of type safety and performance to regain the
convenience and simplicity of Go by resolving overloaded method dispatch
at runtime.

Overloaded Java methods are combined to one Go method that, when invoked,
determines the correct Java method variant at runtime.

The signature of the Go method  is compatible with every Java method with
that name. For the example above, the single Go method becomes the most
general

func (...) F(...interface{}) (interface{}, error)

The method is variadic to cover function with a varying number of
arguments, and it returns interface{} to cover int32, int64 and no
argument. Finally, it returns an error to cover the variant that returns
an error. The generator tries to be specific; for example

func G1(int32) int32
func G2(int32, int32) int32

becomes

func G(int32, ...int32) int32

Overriding Java methods in Go is changed to use the Go parameter types to
determine to correct Java method. To avoid name clashes when overriding
multiple overloaded methods, trailing underscores in the method name are
ignored when matching Java methods.  See the Get methods of GoFuture in
bind/testpkg/javapkg for an example.

Change-Id: I6ac3e024141daa8fc2c35187865c5d7a63368094
Reviewed-on: https://go-review.googlesource.com/35186
Reviewed-by: David Crawshaw <[email protected]>
Change-Id: I37bab4e002238f2701d3e52be898a81d1c2f0db0
Reviewed-on: https://go-review.googlesource.com/35187
Reviewed-by: David Crawshaw <[email protected]>
When determining if a Go struct embeds prefixed types, don't consider
unexported fields. This is important to avoid references cycles with the
Android databinding library; see the reverse example for details.

Change-Id: Ia820ca7ba4d1ec11a1f48651fac248eb753aad75
Reviewed-on: https://go-review.googlesource.com/35188
Reviewed-by: David Crawshaw <[email protected]>
When wrapping Java exceptions, their toString() methods are called from
the wrapper's Error() method to satisfy the Go error interface. Make
sure toString() is always included, even if it never directly referenced
from bound packages.

Change-Id: I5653f6ad82afbe4b061e02a69d60453000288a83
Reviewed-on: https://go-review.googlesource.com/35189
Reviewed-by: David Crawshaw <[email protected]>
Running go get golang.org/x/mobile/... results in errors because the
go tool fails to find the reverse generated Java ("Java/...") and
Objective-C ("ObjC/...") packages. Work around the errors by adding
the android and ios tags, respectively, to files importing those
packages.

The gobind gradle plugin is updated to pass along GOOS=android to
ensure the gobind tool continues to build Android reverse packages.

Fixes golang/go#17750

Change-Id: Id66a3c6cdfe249c6ed494192eb12195d6509332f
Reviewed-on: https://go-review.googlesource.com/34956
Reviewed-by: David Crawshaw <[email protected]>
waylybaye and others added 13 commits January 29, 2021 22:42
create one framework for all targets: ios, simulator and catalyst, every
framework contains at least one arch library, and combine all framework
into one xcframework
As of Go1.16, `go build` fails if go.sum does not exist. This change
fixes this issue by forcing to create go.sum by calling `go mod tidy`.

Closes golang/go#44162

Change-Id: Ibba738503d1a58c6533b2427dd4efa6954b8eaa3
Reviewed-on: https://go-review.googlesource.com/c/mobile/+/290409
Run-TryBot: Hajime Hoshi <[email protected]>
Trust: Than McIntosh <[email protected]>
Trust: Hyang-Ah Hana Kim <[email protected]>
TryBot-Result: Go Bot <[email protected]>
Reviewed-by: Hyang-Ah Hana Kim <[email protected]>
For the Android NDK, between r21 and r22 the NDK compiler switched the
default from "-fcommon" to "-fno-common", which causes duplicate
definitions when building the examples given the current C code. To
fix things, rework the C code to insure that we have a single
definition and an "extern" reference to the objects in question.

Updates golang/go#44121.

Change-Id: If198d5cfd977a7cfd1a37652d3a1916d8c8dbcb7
Reviewed-on: https://go-review.googlesource.com/c/mobile/+/290031
Run-TryBot: Than McIntosh <[email protected]>
TryBot-Result: Go Bot <[email protected]>
Reviewed-by: Hyang-Ah Hana Kim <[email protected]>
Trust: Than McIntosh <[email protected]>
Make all our package sources use Go 1.17 gofmt format
(adding //go:build lines).

Not strictly necessary but will avoid spurious changes
as files are edited.

Part of //go:build change (#41184).
See https://golang.org/design/draft-gobuild

Change-Id: I30822eb504168b037ed3ec0f7759da1f41251f52
Reviewed-on: https://go-review.googlesource.com/c/mobile/+/294374
Trust: Russ Cox <[email protected]>
Run-TryBot: Russ Cox <[email protected]>
TryBot-Result: Go Bot <[email protected]>
Reviewed-by: Robert Griesemer <[email protected]>
Reviewed-by: Jason A. Donenfeld <[email protected]>
 into xcframework-catalyst

# Conflicts:
#	cmd/gomobile/bind_iosapp.go
This reverts commit 6ce0cc4.
@google-cla
Copy link

google-cla bot commented Mar 22, 2021

All (the pull request submitter and all commit authors) CLAs are signed, but one or more commits were authored or co-authored by someone other than the pull request submitter.

We need to confirm that all authors are ok with their commits being contributed to this project. Please have them confirm that by leaving a comment that contains only @googlebot I consent. in this pull request.

Note to project maintainer: There may be cases where the author cannot leave a comment, or the comment is not properly detected as consent. In those cases, you can manually confirm consent of the commit author(s), and set the cla label to yes (if enabled on your project).

ℹ️ Googlers: Go here for more info.

@google-cla google-cla bot added the cla: no label Mar 22, 2021
@imWildCat
Copy link
Author

@googlebot I consent.

@google-cla
Copy link

google-cla bot commented Mar 22, 2021

All (the pull request submitter and all commit authors) CLAs are signed, but one or more commits were authored or co-authored by someone other than the pull request submitter.

We need to confirm that all authors are ok with their commits being contributed to this project. Please have them confirm that by leaving a comment that contains only @googlebot I consent. in this pull request.

Note to project maintainer: There may be cases where the author cannot leave a comment, or the comment is not properly detected as consent. In those cases, you can manually confirm consent of the commit author(s), and set the cla label to yes (if enabled on your project).

ℹ️ Googlers: Go here for more info.

@waylybaye
Copy link

@googlebot I consent

@google-cla
Copy link

google-cla bot commented Mar 22, 2021

All (the pull request submitter and all commit authors) CLAs are signed, but one or more commits were authored or co-authored by someone other than the pull request submitter.

We need to confirm that all authors are ok with their commits being contributed to this project. Please have them confirm that by leaving a comment that contains only @googlebot I consent. in this pull request.

Note to project maintainer: There may be cases where the author cannot leave a comment, or the comment is not properly detected as consent. In those cases, you can manually confirm consent of the commit author(s), and set the cla label to yes (if enabled on your project).

ℹ️ Googlers: Go here for more info.

@imWildCat
Copy link
Author

@dpwiese could you please add your consent too? Thanks!

@google-cla
Copy link

google-cla bot commented Mar 22, 2021

All (the pull request submitter and all commit authors) CLAs are signed, but one or more commits were authored or co-authored by someone other than the pull request submitter.

We need to confirm that all authors are ok with their commits being contributed to this project. Please have them confirm that by leaving a comment that contains only @googlebot I consent. in this pull request.

Note to project maintainer: There may be cases where the author cannot leave a comment, or the comment is not properly detected as consent. In those cases, you can manually confirm consent of the commit author(s), and set the cla label to yes (if enabled on your project).

ℹ️ Googlers: Go here for more info.

@dpwiese
Copy link

dpwiese commented Mar 22, 2021

@googlebot I consent

@google-cla google-cla bot added cla: yes and removed cla: no labels Mar 22, 2021
@gopherbot
Copy link
Contributor

This PR (HEAD: 59c6cf4) has been imported to Gerrit for code review.

Please visit https://go-review.googlesource.com/c/mobile/+/303589 to see it.

Tip: You can toggle comments from me using the comments slash command (e.g. /comments off)
See the Wiki page for more info

@gopherbot
Copy link
Contributor

Message from Go Bot:

Patch Set 1:

Congratulations on opening your first change. Thank you for your contribution!

Next steps:
A maintainer will review your change and provide feedback. See
https://golang.org/doc/contribute.html#review for more info and tips to get your
patch through code review.

Most changes in the Go project go through a few rounds of revision. This can be
surprising to people new to the project. The careful, iterative review process
is our way of helping mentor contributors and ensuring that their contributions
have a lasting impact.


Please don’t reply on this GitHub thread. Visit golang.org/cl/303589.
After addressing review feedback, remember to publish your drafts!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

Successfully merging this pull request may close these issues.