[Xamarin.Android.Tools.AndroidSdk] Preview Build-Tools are Last (#85)

Context: dotnet/android#4735 >
         https://devdiv.visualstudio.com/DevDiv/_build/results?buildId=3771781&view=ms.vss-test-web.build-test-results-tab&runId=13535824&resultId=100072&paneView=attachments
Context: dotnet/android#4567
Context: https://github.com/xamarin/androidtools/commit/a3965baeea566ba6a1f346c676d9e2d5ecba6167

The Windows Smoke Tests are failing on PR #4735, because the wrong
Android SDK Build-tools version is being used:

	Task "ResolveAndroidTooling" (TaskId:9)
	  Task Parameter:AndroidSdkBuildToolsVersion=29.0.2 (TaskId:9)
	  …
	  Trying build-tools path: C:\Users\dlab14\android-toolchain\sdk\build-tools\30.0.0-rc4 (TaskId:9)
	  …
	  ResolveAndroidTooling Outputs: (TaskId:9)
	    AndroidSdkBuildToolsPath: C:\Users\dlab14\android-toolchain\sdk\build-tools\30.0.0-rc4 (TaskId:9)

Here, Build-tools 30.0.0-rc4 is used.

Why is that a problem?  Because JDK 1.8 is still used!

	C:\Program Files\Android\jdk\microsoft_dist_openjdk_1.8.0.25\bin\java.exe -jar C:\Users\dlab14\android-toolchain\sdk\build-tools\30.0.0-rc4\lib\apksigner.jar sign --ks "C:\Users\dlab14\AppData\Local\Xamarin\Mono for Android\debug.keystore" --ks-pass pass:android --ks-key-alias androiddebugkey --key-pass pass:android --min-sdk-version 21 --max-sdk-version 29 …
	java.lang.UnsupportedClassVersionError: com/android/apksigner/ApkSignerTool has been compiled by a more recent version of the Java Runtime (class file version 53.0), this version of the Java Runtime only recognizes class file versions up to 52.0

JDK11 is required in order to use Build-tools r30+, but we don't
currently support JDK11 use; see PR #4567.

What's happening is that *because of* PR #4567, some of the build
machines are being provisioned with Build-tools 30.0.0-rc4, i.e. e.g.
`$HOME/android-toolchain/sdk/build-tools/30.0.0-rc4` *exists*.
*Because* that directory exists -- and *isn't* parse-able by
`System.Version.TryParse()` -- it's considered to be a "preview"
version, and thus *has priority over non-preview versions*, and thus
is returned *first*.  *Because* it's returned *first*,
`<ResolveAndroidTooling/>` prefers it, causing the
`<AndroidApkSigner/>` task to subsequently fail, because it's
unusable.

The "workaround" is for `$(AndroidSdkBuildToolsVersion)` to be set to
the desired version to use.

In retrospect, however, this is all *backwards*:
`AndroidSdkInfo.GetBuildToolsPaths()` shouldn't prefer preview
versions, it should prefer *non-* preview versions.  If you *really*
want a preview version, *then* you should override
`$(AndroidSdkBuildToolsPath)`.

Previews should be opt-in, not opt-out.

With that in mind, why were preview versions preferred in the first
place?  That was done in xamarin/androidtools@a3965bae to fix
[Bug #30555][0], stating:

> handle[] by searching for these "preview" directories first, so that
> the latest tooling will be used IF the user decides to install it.
> The assumption is that a stable release will NOT inlcude a suffix.

While "a preview directory will only exist if someone explicitly
installs it, and thus should be preferred" sounds reasonable, in
retrospect it's a recipe for pain when using a shared CI environment.
Just because it's there does *not* mean it should be used.

[0]: https://bugzilla.xamarin.com/show_bug.cgi?id=30555

Author: jonpryor

Makefile

@@ -1,5 +1,5 @@
 CONFIGURATION   := Debug
-NUNIT_CONSOLE   := packages/NUnit.ConsoleRunner.3.9.0/tools/nunit3-console.exe
+NUNIT_CONSOLE   := packages/nunit.consolerunner/3.9.0/tools/nunit3-console.exe
 OS              := $(shell uname)
 RUNTIME         := mono --debug=casts
 V               ?= 0
@@ -23,7 +23,7 @@ define RUN_NUNIT_TEST
 	$(RUNTIME) \
 		$(NUNIT_CONSOLE) $(NUNIT_EXTRA) $(1) \
 		$(if $(RUN),-run:$(RUN)) \
-		--result="TestResult-$(basename $(notdir $(1))).xml;format=nunit2" \
+		--result="TestResult-$(basename $(notdir $(1))).xml" \
 		-output=bin/Test$(CONFIGURATION)/TestOutput-$(basename $(notdir $(1))).txt \
 	|| true ; \
 	if [ -f "bin/Test$(CONFIGURATION)/TestOutput-$(basename $(notdir $(1))).txt" ] ; then \

NuGet.config

@@ -0,0 +1,17 @@
+<?xml version="1.0" encoding="utf-8"?>
+<configuration>
+  <packageSources>
+    <clear />
+    <!-- ensure only the sources defined below are used -->
+    <add key="dotnet-eng" value="https://pkgs.dev.azure.com/dnceng/public/_packaging/dotnet-eng/nuget/v3/index.json" protocolVersion="3" />
+    <add key="nuget.org" value="https://api.nuget.org/v3/index.json" protocolVersion="3" />
+    <add key="dotnet internal feed" value="https://dotnetfeed.blob.core.windows.net/dotnet-core/index.json" protocolVersion="3" />
+
+     <!-- This is needed (currently) for the Xamarin.Android.Deploy.Installer dependency, getting the installer -->
+     <!-- Android binary, to support delta APK install -->
+     <add key="xamarin.android util" value="https://pkgs.dev.azure.com/xamarin/public/_packaging/Xamarin.Android/nuget/v3/index.json" />
+  </packageSources>
+  <config>
+    <add key="globalPackagesFolder" value="packages" />
+  </config>
+</configuration>

src/Xamarin.Android.Tools.AndroidSdk/AndroidSdkInfo.cs

@@ -45,17 +45,17 @@ public IEnumerable<string> GetBuildToolsPaths ()
 		{
 			var buildTools  = Path.Combine (AndroidSdkPath, "build-tools");
 			if (Directory.Exists (buildTools)) {
+				var sorted = SortedSubdirectoriesByVersion (buildTools);
+
+				foreach (var d in sorted)
+					yield return d;
+
 				var preview = Directory.EnumerateDirectories (buildTools)
 					.Where(x => TryParseVersion (Path.GetFileName (x)) == null)
 					.Select(x => x);
 
 				foreach (var d in preview)
 					yield return d;
-
-				var sorted = SortedSubdirectoriesByVersion (buildTools);
-
-				foreach (var d in sorted)
-					yield return d;
 			}
 			var ptPath  = Path.Combine (AndroidSdkPath, "platform-tools");
 			if (Directory.Exists (ptPath))

tests/Xamarin.Android.Tools.AndroidSdk-Tests/AndroidSdkInfoTests.cs

@@ -448,5 +448,30 @@ string UnixConfigPath {
 				return Path.Combine (UnixConfigDirOverridePath, "monodroid-config.xml");
 			}
 		}
+
+		[Test]
+		public void GetBuildToolsPaths_StableVersionsFirst ()
+		{
+			CreateSdks (out string root, out string jdk, out string ndk, out string sdk);
+			CreateFauxAndroidSdkDirectory (sdk, "27.0.0-rc4");
+
+			var logs    = new StringWriter ();
+			Action<TraceLevel, string> logger = (level, message) => {
+				logs.WriteLine ($"[{level}] {message}");
+			};
+
+			try {
+				var info    = new AndroidSdkInfo (logger, androidSdkPath: sdk, androidNdkPath: ndk, javaSdkPath: jdk);
+
+				var buildToolsPaths = info.GetBuildToolsPaths ().ToList ();
+				Assert.AreEqual (3, buildToolsPaths.Count);
+				Assert.AreEqual (Path.Combine (sdk, "build-tools", "26.0.0"),       buildToolsPaths [0]);
+				Assert.AreEqual (Path.Combine (sdk, "build-tools", "27.0.0-rc4"),   buildToolsPaths [1]);
+				Assert.AreEqual (Path.Combine (sdk, "platform-tools"),              buildToolsPaths [2]);
+			}
+			finally {
+				Directory.Delete (root, recursive: true);
+			}
+		}
 	}
 }