-
Notifications
You must be signed in to change notification settings - Fork 545
[tests] Check for emulator issues and error early #660
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Merged
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
The Android emulator is a "wonderful" beast. Case in point: [PR Build dotnet#1074][0], which *hung* during emulator startup, for *5 hours*: 09:13:49 Hax is enabled 09:13:49 Hax ram_size 0x0 09:13:49 HAX is working and emulator runs in fast virt mode. ... 09:13:56 adb I 06-21 09:13:56 47148 21294922 adb_io.cpp:75] readx: fd=3 wanted=4 ... # 5.5 hours later... 14:47:17 Tool /Users/builder/android-toolchain/sdk/platform-tools/adb execution finished. It "finished" because I manually killed the Job, as it had take over 6 *hours*, cumulative, to do nothing. (*Arguably* we shouldn't have a 10 hour timeout on Jenkins jobs, but Mono bumps can take upwards of 6 hours to build, so a 6 hour time isn't necessarily unusual...) This also isn't our first attempt at improving emulator reliability. See also commits c089267, bc6440b, 3fa9e9e, b54f8cd, 3b893cd, 7450efc, and 6358a64. (Is that enough? Likely not. Is that all of them? Probably not.) Take yet another stab at improving things: in this case, improving error checking, so that *when* an error occurs we can *fail early*, as opposed to waiting for hours on end for a Jenkins timeout to occur *or* for me to get annoyed at general Jenkins build slowness enough to track down the hung job and manually kill it... Update the `emulator` process launch so that we have event handlers for stdout and stderr messages, so that we can look for currently known error conditions, which fall into two categories: 1. HAXM needs reinstallation, or 2. Another VM is in use. HAXM appears to need reinstallation when no memory is allocated: Hax ram_size 0x0 I don't know *why* HAXM would require *re*-installation in these circumstances, but that generally appears to fix the problem. When another VM is in use, e.g. when I'm running a Veertu VM on macOS, `emulator` also fails to launch, writing the following to stderr: Failed to sync vcpu reg Failed to sync HAX vcpu contextInternal error: Initial hax sync failed If either of these conditions occur, the `<StartAndroidEmulator/>` task should *fail*, with a (hopefully) relevant error message. With these changes in place, remove the `<Sleep/>` task invocations from the `AcquireAndroidTarget` target, as the new 20 *second* timeout in `<StartAndroidEmulator/>` removes the need for the first Sleep, and the 2nd sleep doesn't appear to do any good. Followup to c089267! > in case the added sleep doesn't help, we might at least get some > more useful information We do get more information. It's not entirely *useful*: `adb` is waiting for `adbd` to provide it information, and that information never arrives: adb I 06-21 09:13:56 47148 21294922 adb_io.cpp:75] readx: fd=3 wanted=4 ...and there it waits, seemingly forever, until Job timeout or Job death occurs. [0]: https://jenkins.mono-project.com/view/Xamarin.Android/job/xamarin-android-pr-builder/1074/
The build took very long time. It looks like Azure related though, so hopefully next build will be OK.
|
jonpryor
added a commit
to jonpryor/xamarin-android
that referenced
this pull request
Jun 15, 2020
Changes: dotnet/java-interop@b00e644...1de5501 * dotnet/java-interop@1de5501: [CI] Add an "macOS .NET Core" build (dotnet#655) * dotnet/java-interop@9a56465: [generator] Use proper syntax for nested classes for DIM invokers (dotnet#662) * dotnet/java-interop@267c3f3: [generator] Support XML defined enums with no JNI info (dotnet#659) * dotnet/java-interop@5dcf896: [generator] Don't invalidate interface if static method is invalidated (dotnet#660) * dotnet/java-interop@1b59dcc: [java-interop] Update to SDK style project (dotnet#657) * dotnet/java-interop@b136ac9: [param-name-importer] Bump to Microsoft.Xml.SgmlReader 1.8.16 (dotnet#656) * dotnet/java-interop@eb39a3a: Bump to xamarin/xamarin-android-tools/master@3974fc38 (dotnet#658) * dotnet/java-interop@a99b451: [logcat-parse] Update to Mono.Terminal 5.4.0 (dotnet#654) * dotnet/java-interop@425f79d: [Java.Interop] Fix C# warnings (dotnet#652) * dotnet/java-interop@2e0f55d: [Java.Interop.Tools.Generator] Specify $(OutputPath) (dotnet#650) * dotnet/java-interop@4b266fa: [Xamarin.Android.Tools.Bytecode] Support @jvmoverloads (dotnet#651)
jonpryor
added a commit
to jonpryor/xamarin-android
that referenced
this pull request
Jun 15, 2020
Changes: dotnet/java-interop@76d1ac7...1f3388a * dotnet/java-interop@1f3388a: [generator] Use proper syntax for nested classes for DIM invokers (dotnet#662) * dotnet/java-interop@5e23163: [generator] Support XML defined enums with no JNI info (dotnet#659) * dotnet/java-interop@5c4581d: [generator] Don't invalidate interface if static method is invalidated (dotnet#660) * dotnet/java-interop@9687bb5: Bump to xamarin/xamarin-android-tools/d16-7@017078f2 (dotnet#658) * dotnet/java-interop@abfade5: [Java.Interop] Fix C# warnings (dotnet#652) * dotnet/java-interop@fb6d5f9: [Java.Interop.Tools.Generator] Specify $(OutputPath) (dotnet#650) * dotnet/java-interop@a3f148c: [Xamarin.Android.Tools.Bytecode] Support @jvmoverloads (dotnet#651)
jonpryor
added a commit
to jonpryor/xamarin-android
that referenced
this pull request
Jun 15, 2020
Changes: dotnet/java-interop@76d1ac7...1f3388a * dotnet/java-interop@1f3388a: [generator] Use proper syntax for nested classes for DIM invokers (dotnet#662) * dotnet/java-interop@5e23163: [generator] Support XML defined enums with no JNI info (dotnet#659) * dotnet/java-interop@5c4581d: [generator] Don't invalidate interface if static method is invalidated (dotnet#660) * dotnet/java-interop@9687bb5: Bump to xamarin/xamarin-android-tools/d16-7@017078f2 (dotnet#658) * dotnet/java-interop@abfade5: [Java.Interop] Fix C# warnings (dotnet#652) * dotnet/java-interop@fb6d5f9: [Java.Interop.Tools.Generator] Specify $(OutputPath) (dotnet#650) * dotnet/java-interop@a3f148c: [Xamarin.Android.Tools.Bytecode] Support @jvmoverloads (dotnet#651)
jonpryor
added a commit
to jonpryor/xamarin-android
that referenced
this pull request
Jun 15, 2020
Changes: dotnet/java-interop@b00e644...1de5501 * dotnet/java-interop@1de5501: [CI] Add an "macOS .NET Core" build (dotnet#655) * dotnet/java-interop@9a56465: [generator] Use proper syntax for nested classes for DIM invokers (dotnet#662) * dotnet/java-interop@267c3f3: [generator] Support XML defined enums with no JNI info (dotnet#659) * dotnet/java-interop@5dcf896: [generator] Don't invalidate interface if static method is invalidated (dotnet#660) * dotnet/java-interop@1b59dcc: [java-interop] Update to SDK style project (dotnet#657) * dotnet/java-interop@b136ac9: [param-name-importer] Bump to Microsoft.Xml.SgmlReader 1.8.16 (dotnet#656) * dotnet/java-interop@eb39a3a: Bump to xamarin/xamarin-android-tools/master@3974fc38 (dotnet#658) * dotnet/java-interop@a99b451: [logcat-parse] Update to Mono.Terminal 5.4.0 (dotnet#654) * dotnet/java-interop@425f79d: [Java.Interop] Fix C# warnings (dotnet#652) * dotnet/java-interop@2e0f55d: [Java.Interop.Tools.Generator] Specify $(OutputPath) (dotnet#650) * dotnet/java-interop@4b266fa: [Xamarin.Android.Tools.Bytecode] Support @jvmoverloads (dotnet#651)
jonpryor
added a commit
to jonpryor/xamarin-android
that referenced
this pull request
Jun 15, 2020
Changes: dotnet/java-interop@b00e644...1de5501 * dotnet/java-interop@1de5501: [CI] Add an "macOS .NET Core" build (dotnet#655) * dotnet/java-interop@9a56465: [generator] Use proper syntax for nested classes for DIM invokers (dotnet#662) * dotnet/java-interop@267c3f3: [generator] Support XML defined enums with no JNI info (dotnet#659) * dotnet/java-interop@5dcf896: [generator] Don't invalidate interface if static method is invalidated (dotnet#660) * dotnet/java-interop@1b59dcc: [java-interop] Update to SDK style project (dotnet#657) * dotnet/java-interop@b136ac9: [param-name-importer] Bump to Microsoft.Xml.SgmlReader 1.8.16 (dotnet#656) * dotnet/java-interop@eb39a3a: Bump to xamarin/xamarin-android-tools/master@3974fc38 (dotnet#658) * dotnet/java-interop@a99b451: [logcat-parse] Update to Mono.Terminal 5.4.0 (dotnet#654) * dotnet/java-interop@425f79d: [Java.Interop] Fix C# warnings (dotnet#652) * dotnet/java-interop@2e0f55d: [Java.Interop.Tools.Generator] Specify $(OutputPath) (dotnet#650) * dotnet/java-interop@4b266fa: [Xamarin.Android.Tools.Bytecode] Support @jvmoverloads (dotnet#651)
jonpryor
added a commit
that referenced
this pull request
Jun 15, 2020
Changes: dotnet/java-interop@76d1ac7...1f3388a * dotnet/java-interop@1f3388a: [generator] Use proper syntax for nested classes for DIM invokers (#662) * dotnet/java-interop@5e23163: [generator] Support XML defined enums with no JNI info (#659) * dotnet/java-interop@5c4581d: [generator] Don't invalidate interface if static method is invalidated (#660) * dotnet/java-interop@9687bb5: Bump to xamarin/xamarin-android-tools/d16-7@017078f2 (#658) * dotnet/java-interop@abfade5: [Java.Interop] Fix C# warnings (#652) * dotnet/java-interop@fb6d5f9: [Java.Interop.Tools.Generator] Specify $(OutputPath) (#650) * dotnet/java-interop@a3f148c: [Xamarin.Android.Tools.Bytecode] Support @jvmoverloads (#651)
jonpryor
added a commit
that referenced
this pull request
Jun 16, 2020
Changes: dotnet/java-interop@b00e644...1de5501 * dotnet/java-interop@1de5501: [CI] Add an "macOS .NET Core" build (#655) * dotnet/java-interop@9a56465: [generator] Use proper syntax for nested classes for DIM invokers (#662) * dotnet/java-interop@267c3f3: [generator] Support XML defined enums with no JNI info (#659) * dotnet/java-interop@5dcf896: [generator] Don't invalidate interface if static method is invalidated (#660) * dotnet/java-interop@1b59dcc: [java-interop] Update to SDK style project (#657) * dotnet/java-interop@b136ac9: [param-name-importer] Bump to Microsoft.Xml.SgmlReader 1.8.16 (#656) * dotnet/java-interop@eb39a3a: Bump to xamarin/xamarin-android-tools/master@3974fc38 (#658) * dotnet/java-interop@a99b451: [logcat-parse] Update to Mono.Terminal 5.4.0 (#654) * dotnet/java-interop@425f79d: [Java.Interop] Fix C# warnings (#652) * dotnet/java-interop@2e0f55d: [Java.Interop.Tools.Generator] Specify $(OutputPath) (#650) * dotnet/java-interop@4b266fa: [Xamarin.Android.Tools.Bytecode] Support @jvmoverloads (#651)
Sign up for free
to subscribe to this conversation on GitHub.
Already have an account?
Sign in.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
The Android emulator is a "wonderful" beast. Case in point:
PR Build #1074, which hung during emulator startup, for
5 hours:
It "finished" because I manually killed the Job, as it had take over
6 hours, cumulative, to do nothing.
(Arguably we shouldn't have a 10 hour timeout on Jenkins jobs, but
Mono bumps can take upwards of 6 hours to build, so a 6 hour time
isn't necessarily unusual...)
This also isn't our first attempt at improving emulator reliability.
See also commits c089267, bc6440b, 3fa9e9e, b54f8cd, 3b893cd,
7450efc, and 6358a64. (Is that enough? Likely not. Is that all of
them? Probably not.)
Take yet another stab at improving things: in this case, improving
error checking, so that when an error occurs we can fail early, as
opposed to waiting for hours on end for a Jenkins timeout to occur
or for me to get annoyed at general Jenkins build slowness enough to
track down the hung job and manually kill it...
Update the
emulator
process launch so that we have event handlersfor stdout and stderr messages, so that we can look for currently
known error conditions, which fall into two categories:
HAXM appears to need reinstallation when no memory is allocated:
I don't know why HAXM would require re-installation in these
circumstances, but that generally appears to fix the problem.
When another VM is in use, e.g. when I'm running a Veertu VM on macOS,
emulator
also fails to launch, writing the following to stderr:If either of these conditions occur, the
<StartAndroidEmulator/>
task should fail, with a (hopefully) relevant error message.
With these changes in place, remove the
<Sleep/>
task invocationsfrom the
AcquireAndroidTarget
target, as the new 20 second timeoutin
<StartAndroidEmulator/>
removes the need for the first Sleep, andthe 2nd sleep doesn't appear to do any good.
Followup to c089267!
We do get more information. It's not entirely useful:
adb
iswaiting for
adbd
to provide it information, and that informationnever arrives:
...and there it waits, seemingly forever, until Job timeout or Job
death occurs.