Skip to content

Commit 684ede6

Browse files
simonrozsivalsimonrozsival
authored
[NativeAOT] Generate optimized type mapping (#9856)
Context: 70bd636 Context: f48b97c Replaces the current managed `Dictionary<string, Type>` type mapping dictionary introduced in 70bd636, which needs to be fully build at app startup by repeatedly calling `Add()`, with a pre-generated static array of class name hashes. The hashes are sorted so that we can binary search to find the index corresponding to a Java class name. This roughly mirrors what is done in `libxamarin-app.so` (f48b97c). Once we know the index of a Java class class, we can use a generated IL switch to look up the corresponding `Type`. I chose to generate this jumptable in code because this is what NativeAOT understands best. Constructing a static array of metadata tokens is AFAIK not an option for NativeAOT. The generated IL is conceptually equivalent to the following C# code: static partial class TypeMapping { internal static bool TryGetType(string javaClassName, [NotNullWhen (true)] out Type? type) { var hash = XxHash3.HashToUInt64(javaClassName); var index = BinarySearch(JavaClassNameHashes, hash); if (index < 0) { type = null; return false; } type = GetTypeByIndex(index); // verify that this is not just a hash collision… return true; } private static Type? GetTypeByIndex(int index) => index switch { 0 => typeof (A), 1 => typeof (B), // … _ => null, }; private static HashesArray s_hashes = …; // rva static field with initial value private static ReadOnlySpan<ulong> JavaClassNameHashes => new ReadOnlySpan<ulong>(ref s_hashes, 54); [StructLayout(LayoutKind.Explicit, Pack = 1, Size = 432)] private struct HashesArray { } } The `TypeMapping.s_hashes` field is stored as a byte buffer in the assembly and it is loaded from the static RVA field of `TypeMapping`. The relevant generated IL looks like this for a simple app such as `samples/NativeAOT.csproj`: .class private auto ansi beforefieldinit Microsoft.Android.Runtime.TypeMapping extends [System.Private.CoreLib]System.Object { .method private hidebysig static class [System.Private.CoreLib]System.Type GetTypeByIndex (int32 index) cil managed { IL_0000: ldarg.0 IL_0001: switch (IL_00e3, IL_00ee, …, IL_032a) IL_00de: br IL_0335 // index 0 IL_00e3: ldtoken [Mono.Android]Java.IO.File IL_00e8: call class [System.Private.CoreLib]System.Type [System.Private.CoreLib]System.Type::GetTypeFromHandle(valuetype [System.Private.CoreLib]System.RuntimeTypeHandle) IL_00ed: ret // index 1 IL_00ee: ldtoken [Mono.Android]Android.Runtime.InputStreamAdapter IL_00f3: call class [System.Private.CoreLib]System.Type [System.Private.CoreLib]System.Type::GetTypeFromHandle(valuetype [System.Private.CoreLib]System.RuntimeTypeHandle) IL_00f8: ret // … // index N IL_032a: ldtoken [Mono.Android]Java.Lang.StackTraceElement IL_032f: call class [System.Private.CoreLib]System.Type [System.Private.CoreLib]System.Type::GetTypeFromHandle(valuetype [System.Private.CoreLib]System.RuntimeTypeHandle) IL_0334: ret IL_0335: ldnull IL_0336: ret } // end of method TypeMapping::GetTypeByIndex .class nested private explicit ansi HashesArray extends [System.Private.CoreLib]System.ValueType { .pack 1 .size 432 } // end of class HashesArray .field assembly static initonly valuetype Microsoft.Android.Runtime.TypeMapping/HashesArray s_hashes at I_00004A33 .data cil I_00004A33 = bytearray ( c5 27 01 ad 86 95 90 00 2e b5 ff b6 eb ef 2c 04 // .'............,. 5d 20 71 9d c0 8f 78 09 24 db 2e 9c f9 cc 8d 0a // ] q...x.$....... bc d1 4e 4c de f1 d9 0b 83 c5 37 e8 d7 c3 b3 0d // ..NL......7..... f9 b1 64 fe 72 d0 d1 0f 8e 99 32 40 ab c2 dd 10 // ..d.r.....2@.... 95 3d a9 89 b0 a9 81 12 da a1 1a e9 62 96 ed 1e // .=..........b... 7c 52 0b 4c 25 28 37 26 6c 89 44 40 86 46 7b 2a // |R.L%(7&[email protected]{* a5 c1 d6 0b 52 8e 84 37 e2 97 e3 1f 8e fa b2 47 // ....R..7.......G 2a 9a c3 8b 8e d0 d8 47 37 b2 3a d6 7d 11 9e 49 // *......G7.:.}..I 34 29 c1 56 77 21 7b 54 8e 7a 2a 5e 62 1c 2d 5b // 4).Vw!{T.z*^b.-[ a0 a4 53 91 42 b7 18 60 51 0d 00 bc 15 cc 4e 60 // ..S.B..`Q.....N` e7 00 00 75 1c cc 4b 68 fb 35 55 50 bf bc e7 68 // ...u..Kh.5UP...h ee 7c 46 a9 1c c5 e8 68 e7 3c 4e 6d 2f b4 b7 6c // .|F....h.<Nm/..l d4 1c 3d 47 2d 2a c0 6c 67 31 a1 c1 cb 5c c2 70 // ..=G-*.lg1...\.p 66 1f cb d6 59 50 76 73 8e 50 c2 47 46 19 8f 76 // f...YPvs.P.GF..v 0c a3 d9 c9 c1 49 bb 76 47 49 52 5a 0c 86 c7 76 // .....I.vGIRZ...v 43 06 de 1a 9c 34 a4 77 07 5a 17 8d 74 58 2c 7c // C....4.w.Z..tX,| d0 05 a6 c5 b5 ff 97 7c f1 36 3a fc 31 bc 8f 86 // .......|.6:.1... 35 45 f1 6e 96 88 2a 87 60 b1 04 f9 af c9 ee 89 // 5E.n..*.`....... 12 08 0d 44 6c a0 32 8c 35 3a b0 71 3c 2c 44 91 // ...Dl.2.5:.q<,D. c8 25 9d d5 9e 5d 1b 93 09 94 5f 54 87 b1 96 a0 // .%...]...._T.... 59 32 69 4d 3f c4 a6 a0 5f 7d 08 b5 e6 2a 9c a8 // Y2iM?..._}...*.. ce 20 91 5b 2a c5 8c af 5f 58 9d 1e 18 3f 65 bf // . .[*..._X...?e. 57 90 af e1 71 13 35 c1 4b b8 2d 6e 9c 25 7a c5 // W...q.5.K.-n.%z. 68 e8 b8 1d 2d 1d 94 d1 5b dc 7a b6 52 f4 cf d4 // h...-...[.z.R... 56 48 36 d6 2b 3f 18 d7 d0 46 bc 99 55 15 bc da // VH6.+?...F..U... c2 5c aa 9f 40 3b 87 ed 68 5c 3c ea 95 3a f2 ef // .\..@;..h\<..:.. b3 c5 64 33 43 f6 0c f1 dc e9 8b df 19 2a 72 f9 // ..d3C........*r. ) } ~~ Notes ~~ I'm using xxhash64 from the System.IO.Hashing NuGet package. This algorithm is used because it's what we're successfully using in the native type maps. Unfortunately, it presented a challenge when used in a custom linker step, as I needed to manually load the assembly into an ALC, because ILLink would not load the package dependency of our custom step assembly automatically. The switch could become too big for the AOT compiler/RyuJIT to compile. This might need to be revisited later and split into several separate methods; see the [macios managed static registrar][0] for a similar approach used in the macios managed static registrar. [0]: https://github.com/dotnet/macios/blob/f14b02010f1e1b59547f609caab35a40f61f5869/docs/managed-static-registrar.md#method-mapping
1 parent 5a2050f commit 684ede6

File tree

7 files changed

+386
-72
lines changed

7 files changed

+386
-72
lines changed

‎build-tools/create-packs/Microsoft.Android.Runtime.proj

Lines changed: 6 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -44,6 +44,11 @@ projects that use the Microsoft.Android framework in .NET 6+.
4444
Include="$(_MonoAndroidNETOutputRoot)$(AndroidLatestStableApiLevel)\Microsoft.Android.Runtime.NativeAOT.dll"
4545
Condition=" '$(AndroidRuntime)' == 'NativeAOT' "
4646
/>
47+
<_AndroidRuntimePackAssemblies
48+
Include="$(_MonoAndroidNETOutputRoot)$(AndroidLatestStableApiLevel)\System.IO.Hashing.dll"
49+
Condition=" '$(AndroidRuntime)' == 'NativeAOT' "
50+
NoSymbols="true"
51+
/>
4752
<_AndroidRuntimePackAssemblies Include="$(_MonoAndroidNETOutputRoot)$(AndroidLatestStableApiLevel)\Mono.Android.Export.dll" />
4853
</ItemGroup>
4954

@@ -77,7 +82,7 @@ projects that use the Microsoft.Android framework in .NET 6+.
7782

7883
<ItemGroup>
7984
<_PackageFiles Include="@(_AndroidRuntimePackAssemblies)" PackagePath="$(_AndroidRuntimePackAssemblyPath)" TargetPath="$(_AndroidRuntimePackAssemblyPath)" />
80-
<_PackageFiles Include="@(_AndroidRuntimePackAssemblies->'%(RelativeDir)%(Filename).pdb')" PackagePath="$(_AndroidRuntimePackAssemblyPath)" />
85+
<_PackageFiles Include="@(_AndroidRuntimePackAssemblies->'%(RelativeDir)%(Filename).pdb')" PackagePath="$(_AndroidRuntimePackAssemblyPath)" Condition=" '%(_AndroidRuntimePackAssemblies.NoSymbols)' != 'true' " />
8186
<_PackageFiles Include="@(_AndroidRuntimePackAssets)" PackagePath="$(_AndroidRuntimePackNativePath)" TargetPath="$(_AndroidRuntimePackNativePath)" IsNative="true" />
8287
</ItemGroup>
8388
</Target>

‎src/Microsoft.Android.Runtime.NativeAOT/Android.Runtime.NativeAOT/NativeAotTypeManager.cs

Lines changed: 6 additions & 21 deletions
Original file line numberDiff line numberDiff line change
@@ -11,20 +11,8 @@ partial class NativeAotTypeManager : JniRuntime.JniTypeManager {
1111
internal const DynamicallyAccessedMemberTypes Methods = DynamicallyAccessedMemberTypes.PublicMethods | DynamicallyAccessedMemberTypes.NonPublicMethods;
1212
internal const DynamicallyAccessedMemberTypes MethodsAndPrivateNested = Methods | DynamicallyAccessedMemberTypes.NonPublicNestedTypes;
1313

14-
readonly IDictionary<string, Type> TypeMappings = new Dictionary<string, Type> (StringComparer.Ordinal);
15-
1614
public NativeAotTypeManager ()
1715
{
18-
var startTicks = global::System.Environment.TickCount;
19-
InitializeTypeMappings ();
20-
var endTicks = global::System.Environment.TickCount;
21-
AndroidLog.Print (AndroidLogLevel.Info, "NativeAotTypeManager", $"InitializeTypeMappings() took {endTicks - startTicks}ms");
22-
}
23-
24-
void InitializeTypeMappings ()
25-
{
26-
// Should be replaced by src/Microsoft.Android.Sdk.ILLink/TypeMappingStep.cs
27-
throw new InvalidOperationException ("TypeMappings should be replaced during trimming!");
2816
}
2917

3018
[return: DynamicallyAccessedMembers (Constructors)]
@@ -139,7 +127,7 @@ public override void RegisterNativeMembers (
139127

140128
protected override IEnumerable<Type> GetTypesForSimpleReference (string jniSimpleReference)
141129
{
142-
if (TypeMappings.TryGetValue (jniSimpleReference, out var target)) {
130+
if (TypeMapping.TryGetType (jniSimpleReference, out var target)) {
143131
yield return target;
144132
}
145133
foreach (var t in base.GetTypesForSimpleReference (jniSimpleReference)) {
@@ -149,15 +137,12 @@ protected override IEnumerable<Type> GetTypesForSimpleReference (string jniSimpl
149137

150138
protected override IEnumerable<string> GetSimpleReferences (Type type)
151139
{
152-
return base.GetSimpleReferences (type)
153-
.Concat (CreateSimpleReferencesEnumerator (type));
154-
}
140+
foreach (var r in base.GetSimpleReferences (type)) {
141+
yield return r;
142+
}
155143

156-
IEnumerable<string> CreateSimpleReferencesEnumerator (Type type)
157-
{
158-
foreach (var e in TypeMappings) {
159-
if (e.Value == type)
160-
yield return e.Key;
144+
if (TypeMapping.TryGetJavaClassName (type, out var javaClassName)) {
145+
yield return javaClassName;
161146
}
162147
}
163148

‎src/Microsoft.Android.Runtime.NativeAOT/Android.Runtime.NativeAOT/TypeMapping.cs

Lines changed: 92 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,92 @@
1+
using System.Buffers.Binary;
2+
using System.Diagnostics;
3+
using System.Diagnostics.CodeAnalysis;
4+
using System.IO.Hashing;
5+
using System.Runtime.InteropServices;
6+
using System.Text;
7+
using Android.Runtime;
8+
9+
namespace Microsoft.Android.Runtime;
10+
11+
internal static class TypeMapping
12+
{
13+
internal static bool TryGetType (string javaClassName, [NotNullWhen (true)] out Type? type)
14+
{
15+
ulong hash = Hash (javaClassName);
16+
17+
// the hashes array is sorted and all the hashes are unique
18+
int typeIndex = MemoryExtensions.BinarySearch (JavaClassNameHashes, hash);
19+
if (typeIndex < 0) {
20+
type = null;
21+
return false;
22+
}
23+
24+
type = GetTypeByIndex (typeIndex);
25+
if (type is null) {
26+
throw new InvalidOperationException ($"Type with hash {hash} not found.");
27+
}
28+
29+
// ensure this is not a hash collision
30+
var resolvedJavaClassName = GetJavaClassNameByIndex (TypeIndexToJavaClassNameIndex [typeIndex]);
31+
if (resolvedJavaClassName != javaClassName) {
32+
type = null;
33+
return false;
34+
}
35+
36+
return true;
37+
}
38+
39+
internal static bool TryGetJavaClassName (Type type, [NotNullWhen (true)] out string? className)
40+
{
41+
string? fullName = type.FullName;
42+
if (fullName is null) {
43+
className = null;
44+
return false;
45+
}
46+
47+
ulong hash = Hash (fullName);
48+
49+
// the hashes array is sorted and all the hashes are unique
50+
int javaClassNameIndex = MemoryExtensions.BinarySearch (TypeNameHashes, hash);
51+
if (javaClassNameIndex < 0) {
52+
className = null;
53+
return false;
54+
}
55+
56+
className = GetJavaClassNameByIndex (javaClassNameIndex);
57+
if (className is null) {
58+
throw new InvalidOperationException ($"Java class name with hash {hash} not found.");
59+
}
60+
61+
// ensure this is not a hash collision
62+
var resolvedType = GetTypeByIndex (JavaClassNameIndexToTypeIndex [javaClassNameIndex]);
63+
if (resolvedType?.FullName != type.FullName) {
64+
className = null;
65+
return false;
66+
}
67+
68+
return true;
69+
}
70+
71+
private static ulong Hash (string javaClassName)
72+
{
73+
ReadOnlySpan<byte> bytes = MemoryMarshal.AsBytes (javaClassName.AsSpan ());
74+
ulong hash = XxHash3.HashToUInt64 (bytes);
75+
76+
// The bytes in the hashes array are stored as little endian. If the target platform is big endian,
77+
// we need to reverse the endianness of the hash.
78+
if (!BitConverter.IsLittleEndian) {
79+
hash = BinaryPrimitives.ReverseEndianness (hash);
80+
}
81+
82+
return hash;
83+
}
84+
85+
// Replaced by src/Microsoft.Android.Sdk.ILLink/TypeMappingStep.cs
86+
private static ReadOnlySpan<ulong> JavaClassNameHashes => throw new NotImplementedException ();
87+
private static ReadOnlySpan<ulong> TypeNameHashes => throw new NotImplementedException ();
88+
private static ReadOnlySpan<int> JavaClassNameIndexToTypeIndex => throw new NotImplementedException ();
89+
private static ReadOnlySpan<int> TypeIndexToJavaClassNameIndex => throw new NotImplementedException ();
90+
private static Type? GetTypeByIndex (int index) => throw new NotImplementedException ();
91+
private static string? GetJavaClassNameByIndex (int index) => throw new NotImplementedException ();
92+
}

‎src/Microsoft.Android.Runtime.NativeAOT/Microsoft.Android.Runtime.NativeAOT.csproj

Lines changed: 5 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -21,6 +21,10 @@
2121
<ProjectReference Include="..\Mono.Android\Mono.Android.csproj" />
2222
</ItemGroup>
2323

24+
<ItemGroup>
25+
<PackageReference Include="System.IO.Hashing" Version="$(SystemIOHashingPackageVersion)" />
26+
</ItemGroup>
27+
2428
<Import Project="Sdk.targets" Sdk="Microsoft.NET.Sdk" />
2529

2630
<!-- Copy runtime assemblies to bin/$(Configuration)/dotnet/packs folder -->
@@ -34,6 +38,7 @@
3438
<Target Name="_CopyToPackDirs">
3539
<ItemGroup>
3640
<_RuntimePackFiles Include="$(OutputPath)Microsoft.Android.Runtime.NativeAOT.dll" AndroidRID="%(AndroidAbiAndRuntimeFlavor.AndroidRID)" AndroidRuntime="%(AndroidAbiAndRuntimeFlavor.AndroidRuntime)" />
41+
<_RuntimePackFiles Include="$(OutputPath)System.IO.Hashing.dll" AndroidRID="%(AndroidAbiAndRuntimeFlavor.AndroidRID)" AndroidRuntime="%(AndroidAbiAndRuntimeFlavor.AndroidRuntime)" />
3742
</ItemGroup>
3843
<Message Importance="high" Text="$(TargetPath) %(AndroidAbiAndRuntimeFlavor.AndroidRID)" />
3944
<Copy

0 commit comments

Comments
 (0)