Skip to content

ApiXmlAdjuster fails to resolve parent type parameters for nested types #815

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
jpobst opened this issue Mar 11, 2021 · 0 comments · Fixed by #916 or dotnet/android#6533
Closed

ApiXmlAdjuster fails to resolve parent type parameters for nested types #815

jpobst opened this issue Mar 11, 2021 · 0 comments · Fixed by #916 or dotnet/android#6533
Labels
bug Component does not function as intended generator Issues binding a Java library (generator, class-parse, etc.)

Comments

@jpobst
Copy link
Contributor

jpobst commented Mar 11, 2021

Given a type like this:

class MapCollections<K, V> {
  class MapIterator implements Iterator<Map.Entry<K, V>>, Map.Entry<K, V> { 
    public K getKey() { ... }
  }
}

class-parse does not copy the parent's type parameters to the nested type:

<class
  jni-extends="Ljava/lang/Object;"
  extends="java.lang.Object"
  extends-generic-aware="java.lang.Object"
  name="MapCollections.MapIterator"
  jni-signature="Landroidx/collection/MapCollections$MapIterator;"
>
  <implements
    name="java.util.Iterator"
    name-generic-aware="java.util.Iterator&lt;java.util.Map.Entry&lt;K, V&gt;&gt;"
    jni-type="Ljava/util/Iterator&lt;Ljava/util/Map$Entry&lt;TK;TV;&gt;;&gt;;" />
  <implements
    name="java.util.Map.Entry"
    name-generic-aware="java.util.Map.Entry&lt;K, V&gt;"
    jni-type="Ljava/util/Map$Entry&lt;TK;TV;&gt;;" />
  <method
    name="getKey"
    return="K"
    jni-return="TK;"
    jni-signature="()Ljava/lang/Object;" />
...
</class>

ApiXmlAdjuster does not check parent types for type parameters, and thus cannot resolve K or V within this type:

Error while processing type '[Class] androidx.collection.MapCollections.MapIterator': Type 'K' was not found.
Error while processing type '[Class] androidx.collection.MapCollections.MapIterator': Type 'V' was not found.

Because the implements cannot be resolved, the nested type is removed from the binding.

@jpobst jpobst added bug Component does not function as intended generator Issues binding a Java library (generator, class-parse, etc.) labels Mar 11, 2021
jonpryor pushed a commit that referenced this issue Oct 26, 2021
Fixes: #739
Fixes: #852
Fixes: #853

Context: #789
Context: #814
Context: #815

Today, we use a process called `ApiXmlAdjuster` to convert our "new"
`class-parse` format to the old `jar2xml` format.  This process
attempts to resolve every Java type referenced by the library we
intend to bind.  However, performing this as a separate step has
some disadvantages:

  - It duplicates work by reading in `api.xml.class-parse`
    and Cecil'ing all reference assemblies, then writing it back out
    as `api.xml`.
    `generator` then reads in `api.xml` and Cecils all the reference
    assemblies again.

  - This work is performed as a separate step before `generator` and
    thus cannot be influenced by `metadata`.

  - This work is cached, which is "good", but it creates issues for
    users because they only see warnings from it for full Rebuilds.

"Replace" `ApiXmlAdjuster` with `Java.Interop.Tools.JavaTypeSystem`,
which is a much more robust Java model to perform this work, and
contains enough information to be reused by `generator`.
This prevents us from needing to have a second Java and Cecil import
step, and we can run `metadata` directly on `api.xml.class-parse`
before the import.

NOTE: This commit sets the groundwork for the above benefits, but
does not achieve them yet.  Initially we are only focused on
achieving parity with `ApiXmlAdjuster` to make correctness
comparisons possible.

The new `JavaTypeSystem` infrastructure is used by default; the
previous `ApiXmlAdjuster` system can be used instead by using:

	generator --use-legacy-java-resolver=true …

We will provide an MSBuild property to control this when we integrate
with xamarin-android in the future.


~~ Warning Messages ~~

In order to better surface potentially real errors to users, we have
tweaked the warnings displayed.  Errors we encounter during the
initial phase of Java type resolution are emitted as warnings.
These are missing external Java types and generally indicate the user
is missing a reference `.jar` or binding NuGet:

	warning BG8605: The Java type 'androidx.appcompat.widget.AppCompatTextView' could not be found (are you missing a Java reference jar/aar or a Java binding library NuGet?)

The remaining resolution errors are generally the fallout from those
missing external types.  For example, if the above missing type
caused us to remove `com.example.MyTextView`, then that may result in
us removing the method `GetMyTextView()` due to a missing return type.

Today these messages are only displayed in a Diagnostic log and are
often missed.  Instead, we are going to write them to a log file
called `java-resolution-report.log`, and then display a warning
notifying the user that some types could not be bound, which can be
double clicked to view the log file.

	warning BG8606: Some types or members could not be bound because referenced Java types could not be found. See the 'java-resolution-report.log' file for details.

Example `java-resolution-report.log`:

	==== Cycle 1 ====

	'[Class] com.example.MyTextView' was removed because the Java type 'androidx.appcompat.widget.AppCompatTextView' could not be found.

	==== Cycle 2 ====

	'[Method] com.example.MyActivity.getMyTextView' was removed because the Java type 'com.example.MyTextView' could not be found.


"Cycles" represent the passes through the type collection when
resolving the collection.  For example:

  - Cycle 1 removed type `com.example.MyType` because its external
    base type `android.util.List` was missing

  - Cycle 2 removed type `com.example.MyDerivedType` because its base
    type `com.example.MyType` is now missing

  - Cycle 3 removed method `com.example.Foo.getBar()` because it
    returns `com.example.MyDerivedType`, which is now missing

This distinction can be interesting, because Cycle 1 removals are
generally due to missing `.jar` dependencies, whereas the remaining
cycles are just the internal fallout from previous cycles, which will
largely be fixed by fixing the first cycle issues.


~~ Additional Fixes ~~

There are several issues with the `ApiXmlAdjuster` process that *can*
be fixed by this change.  For the initial purpose of being able to
verify that we produce the same output, these have been disabled in
the new `JavaTypeSystem` code.

  * [ApiXmlAdjuster removes required methods from interfaces][0] -
    Currently disabled via flag
    `TypeResolutionOptions.RemoveInterfacesWithUnresolvableMembers`.

  * [ApiXmlAdjuster fails to resolve parent type parameters for nested types][1] -
    Currently commented out in `JavaTypeModel.GetApplicableTypeParameters()`.

  * [ApiXmlAdjuster doesn't remove nested types][2] -
    Currently enabled, compares have been done with a custom
    `ApiXmlAdjuster` where this has been fixed.

  * [ApiXmlAdjuster prefers JavaList over ArrayList][3] -
    Currently enabled, compares have been done with a custom
    `ApiXmlAdjuster` where this has been fixed.


~~ Performance ~~

Although performance isn't really a focus, there are wins here as well,
largely due to only resolving reference types that are actually
referenced.  Example: `ApiXmlAdjuster` fully resolves all 5K+ types in
`Mono.Android.dll`, while `JavaTypeSystem` only resolves the ones
needed by the library being bound.

Time to resolve `androidx.appcompat.appcompat`:

  * ApiXmlAdjuster: 2849ms
  * JavaTypeSystem: 1514ms


~~ Testing ~~

  - Unit tests have been ported from `ApiXmlAdjuster-Tests`
  - Tested for identical output for 136 current AndroidX packages
  - Tested for identical output for 140 current GPS/FB packages

Note `Mono.Android.dll` does not use `ApiXmlAdjuster`, so no testing
was performed with it.

[0]: #814
[1]: #815
[2]: #852
[3]: #853
jonpryor pushed a commit that referenced this issue Nov 30, 2021
)

Fixes: #815

Given Java code such as:

	// Java
	public class MyClass<T> {
	    public class MyNestedClass<U> {
	      public void doT(T value) {}
	      public void doU(U value) {}
	    }
	}

`ApiXmlAdjuster` does not look for generic parameter types declared
on parent types, resulting in the method `doT()` being removed:

	api.xml.class-parse : warning BG8605: The Java type 'T' could not be found (are you missing a Java reference jar/aar or a Java binding library NuGet?)

`java-resolution-report.log` (f658ab2) reports:

	==== Cycle 1 ====

	The method '[Method] void doT(T p0)' was removed because the Java parameter type 'T' could not be found.

`JavaTypeSystem` supports this, but it was initially disabled to help
ensure that it was producing the same output as `ApiXmlAdjuster` for
testing purposes.

We can now enable it.
jonpryor added a commit to jonpryor/xamarin-android that referenced this issue Nov 30, 2021
Fixes: dotnet/java-interop#815
Fixes: dotnet/java-interop#920

Changes: dotnet/java-interop@0293360...7f1a5ab

  * dotnet/java-interop@7f1a5ab1: [JavaTypeSystem] Enable resolving generic types on declaring types (dotnet#916)
  * dotnet/java-interop@8daa0263: [Xamarin.Android.Tools.Bytecode] Fix Nullability warnings (dotnet#922)
  * dotnet/java-interop@77c9c5fa: [class-parse] Import parameter names for unresolvable types (dotnet#921)
  * dotnet/java-interop@a8b444d5: [Java.Interop.Tools.JavaTypeSystem] ctors w/ generic type parameters (dotnet#919)
jonpryor added a commit to dotnet/android that referenced this issue Nov 30, 2021
Fixes: dotnet/java-interop#815
Fixes: dotnet/java-interop#920

Changes: dotnet/java-interop@0293360...7f1a5ab

  * dotnet/java-interop@7f1a5ab1: [JavaTypeSystem] Enable resolving generic types on declaring types (#916)
  * dotnet/java-interop@8daa0263: [Xamarin.Android.Tools.Bytecode] Fix Nullability warnings (#922)
  * dotnet/java-interop@77c9c5fa: [class-parse] Import parameter names for unresolvable types (#921)
  * dotnet/java-interop@a8b444d5: [Java.Interop.Tools.JavaTypeSystem] ctors w/ generic type parameters (#919)
jonathanpeppers pushed a commit to dotnet/android that referenced this issue Nov 30, 2021
Fixes: dotnet/java-interop#815
Fixes: dotnet/java-interop#920

Changes: dotnet/java-interop@0293360...7f1a5ab

  * dotnet/java-interop@7f1a5ab1: [JavaTypeSystem] Enable resolving generic types on declaring types (#916)
  * dotnet/java-interop@8daa0263: [Xamarin.Android.Tools.Bytecode] Fix Nullability warnings (#922)
  * dotnet/java-interop@77c9c5fa: [class-parse] Import parameter names for unresolvable types (#921)
  * dotnet/java-interop@a8b444d5: [Java.Interop.Tools.JavaTypeSystem] ctors w/ generic type parameters (#919)
@github-actions github-actions bot locked and limited conversation to collaborators Apr 12, 2024
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
bug Component does not function as intended generator Issues binding a Java library (generator, class-parse, etc.)
Projects
None yet
1 participant