Skip to content

firebase_release_smoke_test crashing consistently on Engine roll attempts #74502

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Closed
zanderso opened this issue Jan 22, 2021 · 16 comments
Closed

firebase_release_smoke_test crashing consistently on Engine roll attempts #74502

zanderso opened this issue Jan 22, 2021 · 16 comments
Assignees
@gaaclarke
Labels
c: fatal crash Crashes that terminate the process engine flutter/engine repository. See also e: labels. P0 Critical issues such as a build break or regression

Comments

@zanderso
Copy link
Member

zanderso commented Jan 22, 2021 •
edited
Loading

Starting with #74457

For example: https://ci.chromium.org/ui/p/flutter/builders/try/Linux%20firebase_release_smoke_test/6528/overview

On "Nexus 5, Virtual, API Level 19"

Some logs from the run:

01-22 00:26:43.102: E/flutter(5604): [ERROR:flutter/shell/gpu/gpu_surface_gl.cc(67)] Failed to setup Skia Gr context.
01-22 00:26:43.182: I/flutter(5604): Observatory listening on http://127.0.0.1:37964/-JtntZl5k4g=/
01-22 00:26:43.362: I/flutter(5604): Text
01-22 00:26:43.392: I/flutter(5604): Text("Hello, world!", textDirection: ltr)
01-22 00:26:43.542: I/ActivityManager(1720): Displayed com.example.release_smoke_test/.MainActivity: +848ms
01-22 00:26:43.682: I/flutter(5604): [116, 101, 115, 116]
01-22 00:26:43.882: I/Remoter(1345): Saved screenshot to '/data/screenshots/__pmv__-0142.jpg'

Crash log from the roll to 7c19824

 Fatal signal 11 (SIGSEGV) at 0x00000008 (code=1), thread 5623 (1.raster)
 *** *** *** *** *** *** *** *** *** *** *** *** *** *** *** ***
 Build fingerprint: 'generic/gce_x86_phone/gce_x86:4.4.4/KGM2.180904.001/4990979:userdebug/test-keys'
 Revision: '0'
 pid: 5604, tid: 5623, name: 1.raster  >>> com.example.release_smoke_test <<<
 signal 11 (SIGSEGV), code 1 (SEGV_MAPERR), fault addr 00000008
     eax 00000000  ebx 7efad7c0  ecx fffffff9  edx 00000000
     esi 00000004  edi 7d1ac458
     xcs 00000073  xds 0000007b  xes 0000007b  xfs 00000000  xss 0000007b
     eip 7e90288d  ebp 7d1ac1b8  esp 7d1ac1b8  flags 00210206
 backtrace:
     #00  pc 0025c88d  /data/app-lib/com.example.release_smoke_test-1/libflutter.so
     #01  pc 00276015  /data/app-lib/com.example.release_smoke_test-1/libflutter.so
     #02  pc 003032dc  /data/app-lib/com.example.release_smoke_test-1/libflutter.so
     #03  pc 004f0979  /data/app-lib/com.example.release_smoke_test-1/libflutter.so
     #04  pc 004f0845  /data/app-lib/com.example.release_smoke_test-1/libflutter.so
     #05  pc 004f0d33  /data/app-lib/com.example.release_smoke_test-1/libflutter.so
     #06  pc 004f0b1e  /data/app-lib/com.example.release_smoke_test-1/libflutter.so
     #07  pc 00077277  /data/app-lib/com.example.release_smoke_test-1/libflutter.so
     #08  pc 00077f63  /data/app-lib/com.example.release_smoke_test-1/libflutter.so
     #09  pc 00078dfa  /data/app-lib/com.example.release_smoke_test-1/libflutter.so
     #10  pc 0002908b  /data/app-lib/com.example.release_smoke_test-1/libflutter.so
     #11  pc 00077862  /data/app-lib/com.example.release_smoke_test-1/libflutter.so
     #12  pc 000775de  /data/app-lib/com.example.release_smoke_test-1/libflutter.so
     #13  pc 000868d2  /data/app-lib/com.example.release_smoke_test-1/libflutter.so
     #14  pc 0002df9b  /data/app-lib/com.example.release_smoke_test-1/libflutter.so
     #15  pc 00030d93  /data/app-lib/com.example.release_smoke_test-1/libflutter.so
     #16  pc 00030ca4  /data/app-lib/com.example.release_smoke_test-1/libflutter.so
     #17  pc 00037430  /data/app-lib/com.example.release_smoke_test-1/libflutter.so
     #18  pc 0003745e  /data/app-lib/com.example.release_smoke_test-1/libflutter.so
     #19  pc 0001cd61  /system/lib/libutils.so (android::SimpleLooperCallback::handleEvent(int, int, void*)+33)
     #20  pc 0001ef5a  /system/lib/libutils.so (android::Looper::pollInner(int)+890)
     #21  pc 0001f2c3  /system/lib/libutils.so (android::Looper::pollOnce(int, int*, int*, void**)+51)
     #22  pc 0000b6ab  /system/lib/libandroid.so (ALooper_pollOnce+107)
     #23  pc 00037373  /data/app-lib/com.example.release_smoke_test-1/libflutter.so
     #24  pc 00030c52  /data/app-lib/com.example.release_smoke_test-1/libflutter.so
     #25  pc 00030782  /data/app-lib/com.example.release_smoke_test-1/libflutter.so
     #26  pc 0003485f  /data/app-lib/com.example.release_smoke_test-1/libflutter.so
     #27  pc 0000bc0c  /system/lib/libc.so (__thread_entry+236)
     #28  pc 0003e075  /system/lib/libc.so (__pthread_clone+69)
     #29  pc 000347ce  /data/app-lib/com.example.release_smoke_test-1/libflutter.so
     #30  pc 0003043f  [heap]
     #31  pc 0002e61f  [heap]
@zanderso zanderso added engine flutter/engine repository. See also e: labels. P0 Critical issues such as a build break or regression c: fatal crash Crashes that terminate the process labels Jan 22, 2021
@zanderso

This comment has been minimized.

Sign in to view
@flutter-symbolizer-bot

This comment has been minimized.

Sign in to view
@zanderso

This comment has been minimized.

Sign in to view
@flutter-symbolizer-bot

This comment has been minimized.

Sign in to view
@zanderso

This comment has been minimized.

Sign in to view
@flutter-symbolizer-bot

This comment has been minimized.

Sign in to view
@zanderso

This comment has been minimized.

Sign in to view
@zanderso
Copy link
Member Author

@flutter-symbolizer-bot #74502 (comment) engine#7c19824c6d08bb8e3fc4217058178d42d451e26b android debug x86

@flutter-symbolizer-bot
Copy link

crash from #74502 (comment) symbolized using symbols for 7c19824c6d08bb8e3fc4217058178d42d451e26b android-x86-debug

#00 0025c88d /data/app-lib/com.example.release_smoke_test-1/libflutter.so 
                                                                          sk_sp<GrContextThreadSafeProxy>::operator->() const
                                                                          third_party/skia/include/core/SkRefCnt.h:299:36
                                                                          GrContext_Base::caps() const
                                                                          third_party/skia/src/gpu/GrContext_Base.cpp:35:53
#01 00276015 /data/app-lib/com.example.release_smoke_test-1/libflutter.so 
                                                                          GrRecordingContext::maxSurfaceSampleCountForColorType(SkColorType) const
                                                                          third_party/skia/src/gpu/GrRecordingContext.cpp:48:19
#02 003032dc /data/app-lib/com.example.release_smoke_test-1/libflutter.so 
                                                                          GrRecordingContext::colorTypeSupportedAsSurface(SkColorType) const
                                                                          third_party/skia/include/gpu/GrRecordingContext.h:64:22
#03 004f0979 /data/app-lib/com.example.release_smoke_test-1/libflutter.so 
                                                                          flutter::FirstSupportedColorType(GrDirectContext*, unsigned int*)
                                                                          flutter/shell/gpu/gpu_surface_gl.cc:144:3
                                                                          flutter::WrapOnscreenSurface(GrDirectContext*, SkISize const&, int)
                                                                          flutter/shell/gpu/gpu_surface_gl.cc:154:34
#04 004f0845 /data/app-lib/com.example.release_smoke_test-1/libflutter.so 
                                                                          flutter::GPUSurfaceGL::CreateOrUpdateSurfaces(SkISize const&)
                                                                          flutter/shell/gpu/gpu_surface_gl.cc:205:22
#05 004f0d33 /data/app-lib/com.example.release_smoke_test-1/libflutter.so 
                                                                          flutter::GPUSurfaceGL::AcquireRenderSurface(SkISize const&, SkMatrix const&)
                                                                          flutter/shell/gpu/gpu_surface_gl.cc:320:8
#06 004f0b1e /data/app-lib/com.example.release_smoke_test-1/libflutter.so 
                                                                          flutter::GPUSurfaceGL::AcquireFrame(SkISize const&)
                                                                          flutter/shell/gpu/gpu_surface_gl.cc:252:7
#07 00077277 /data/app-lib/com.example.release_smoke_test-1/libflutter.so 
                                                                          flutter::Rasterizer::DrawToSurface(flutter::LayerTree&)
                                                                          flutter/shell/common/rasterizer.cc:438:26
#08 00077f63 /data/app-lib/com.example.release_smoke_test-1/libflutter.so 
                                                                          flutter::Rasterizer::DoDraw(std::__1::unique_ptr<flutter::LayerTree, std::__1::default_delete<flutter::LayerTree> >)
                                                                          flutter/shell/common/rasterizer.cc:338:32
#09 00078dfa /data/app-lib/com.example.release_smoke_test-1/libflutter.so 
                                                                          flutter::Rasterizer::Draw(fml::RefPtr<flutter::Pipeline<flutter::LayerTree> >, std::__1::function<bool (flutter::LayerTree&)>)::$_1::operator()(std::__1::unique_ptr<flutter::LayerTree, std::__1::default_delete<flutter::LayerTree> >) const
                                                                          flutter/shell/common/rasterizer.cc:172:27
                                                                          decltype(std::__1::forward<flutter::Rasterizer::Draw(fml::RefPtr<flutter::Pipeline<flutter::LayerTree> >, std::__1::function<bool (flutter::LayerTree&)>)::$_1&>(fp)(std::__1::forward<std::__1::unique_ptr<flutter::LayerTree, std::__1::default_delete<flutter::LayerTree> > >(fp0))) std::__1::__invoke<flutter::Rasterizer::Draw(fml::RefPtr<flutter::Pipeline<flutter::LayerTree> >, std::__1::function<bool (flutter::LayerTree&)>)::$_1&, std::__1::unique_ptr<flutter::LayerTree, std::__1::default_delete<flutter::LayerTree> > >(flutter::Rasterizer::Draw(fml::RefPtr<flutter::Pipeline<flutter::LayerTree> >, std::__1::function<bool (flutter::LayerTree&)>)::$_1&, std::__1::unique_ptr<flutter::LayerTree, std::__1::default_delete<flutter::LayerTree> >&&)
                                                                          third_party/libcxx/include/type_traits:3530:1
                                                                          void std::__1::__invoke_void_return_wrapper<void>::__call<flutter::Rasterizer::Draw(fml::RefPtr<flutter::Pipeline<flutter::LayerTree> >, std::__1::function<bool (flutter::LayerTree&)>)::$_1&, std::__1::unique_ptr<flutter::LayerTree, std::__1::default_delete<flutter::LayerTree> > >(flutter::Rasterizer::Draw(fml::RefPtr<flutter::Pipeline<flutter::LayerTree> >, std::__1::function<bool (flutter::LayerTree&)>)::$_1&, std::__1::unique_ptr<flutter::LayerTree, std::__1::default_delete<flutter::LayerTree> >&&)
                                                                          third_party/libcxx/include/__functional_base:348:9
                                                                          std::__1::__function::__alloc_func<flutter::Rasterizer::Draw(fml::RefPtr<flutter::Pipeline<flutter::LayerTree> >, std::__1::function<bool (flutter::LayerTree&)>)::$_1, std::__1::allocator<flutter::Rasterizer::Draw(fml::RefPtr<flutter::Pipeline<flutter::LayerTree> >, std::__1::function<bool (flutter::LayerTree&)>)::$_1>, void (std::__1::unique_ptr<flutter::LayerTree, std::__1::default_delete<flutter::LayerTree> >)>::operator()(std::__1::unique_ptr<flutter::LayerTree, std::__1::default_delete<flutter::LayerTree> >&&)
                                                                          third_party/libcxx/include/functional:1533:16
                                                                          std::__1::__function::__func<flutter::Rasterizer::Draw(fml::RefPtr<flutter::Pipeline<flutter::LayerTree> >, std::__1::function<bool (flutter::LayerTree&)>)::$_1, std::__1::allocator<flutter::Rasterizer::Draw(fml::RefPtr<flutter::Pipeline<flutter::LayerTree> >, std::__1::function<bool (flutter::LayerTree&)>)::$_1>, void (std::__1::unique_ptr<flutter::LayerTree, std::__1::default_delete<flutter::LayerTree> >)>::operator()(std::__1::unique_ptr<flutter::LayerTree, std::__1::default_delete<flutter::LayerTree> >&&)
                                                                          third_party/libcxx/include/functional:1707:12
#10 0002908b /data/app-lib/com.example.release_smoke_test-1/libflutter.so 
                                                                          std::__1::__function::__value_func<void (long long)>::operator()(long long&&) const
                                                                          third_party/libcxx/include/functional:1860:16
#11 00077862 /data/app-lib/com.example.release_smoke_test-1/libflutter.so 
                                                                          std::__1::function<void (std::__1::unique_ptr<flutter::LayerTree, std::__1::default_delete<flutter::LayerTree> >)>::operator()(std::__1::unique_ptr<flutter::LayerTree, std::__1::default_delete<flutter::LayerTree> >) const
                                                                          third_party/libcxx/include/functional:2419:12
                                                                          flutter::Pipeline<flutter::LayerTree>::Consume(std::__1::function<void (std::__1::unique_ptr<flutter::LayerTree, std::__1::default_delete<flutter::LayerTree> >)> const&)
                                                                          flutter/shell/common/pipeline.h:161:7
#12 000775de /data/app-lib/com.example.release_smoke_test-1/libflutter.so 
                                                                          flutter::Rasterizer::Draw(fml::RefPtr<flutter::Pipeline<flutter::LayerTree> >, std::__1::function<bool (flutter::LayerTree&)>)
                                                                          flutter/shell/common/rasterizer.cc:176:52
#13 000868d2 /data/app-lib/com.example.release_smoke_test-1/libflutter.so 
                                                                          flutter::Shell::OnAnimatorDraw(fml::RefPtr<flutter::Pipeline<flutter::LayerTree> >, fml::TimePoint)::$_33::operator()() const
                                                                          flutter/shell/common/shell.cc:1121:23
                                                                          decltype(std::__1::forward<flutter::Shell::OnAnimatorDraw(fml::RefPtr<flutter::Pipeline<flutter::LayerTree> >, fml::TimePoint)::$_33&>(fp)()) std::__1::__invoke<flutter::Shell::OnAnimatorDraw(fml::RefPtr<flutter::Pipeline<flutter::LayerTree> >, fml::TimePoint)::$_33&>(flutter::Shell::OnAnimatorDraw(fml::RefPtr<flutter::Pipeline<flutter::LayerTree> >, fml::TimePoint)::$_33&)
                                                                          third_party/libcxx/include/type_traits:3530:1
                                                                          void std::__1::__invoke_void_return_wrapper<void>::__call<flutter::Shell::OnAnimatorDraw(fml::RefPtr<flutter::Pipeline<flutter::LayerTree> >, fml::TimePoint)::$_33&>(flutter::Shell::OnAnimatorDraw(fml::RefPtr<flutter::Pipeline<flutter::LayerTree> >, fml::TimePoint)::$_33&)
                                                                          third_party/libcxx/include/__functional_base:348:9
                                                                          std::__1::__function::__alloc_func<flutter::Shell::OnAnimatorDraw(fml::RefPtr<flutter::Pipeline<flutter::LayerTree> >, fml::TimePoint)::$_33, std::__1::allocator<flutter::Shell::OnAnimatorDraw(fml::RefPtr<flutter::Pipeline<flutter::LayerTree> >, fml::TimePoint)::$_33>, void ()>::operator()()
                                                                          third_party/libcxx/include/functional:1533:16
                                                                          std::__1::__function::__func<flutter::Shell::OnAnimatorDraw(fml::RefPtr<flutter::Pipeline<flutter::LayerTree> >, fml::TimePoint)::$_33, std::__1::allocator<flutter::Shell::OnAnimatorDraw(fml::RefPtr<flutter::Pipeline<flutter::LayerTree> >, fml::TimePoint)::$_33>, void ()>::operator()()
                                                                          third_party/libcxx/include/functional:1707:12
#14 0002df9b /data/app-lib/com.example.release_smoke_test-1/libflutter.so 
                                                                          std::__1::__function::__value_func<void ()>::operator()() const
                                                                          third_party/libcxx/include/functional:1860:16
#15 00030d93 /data/app-lib/com.example.release_smoke_test-1/libflutter.so 
                                                                          std::__1::function<void ()>::operator()() const
                                                                          third_party/libcxx/include/functional:2419:12
                                                                          fml::MessageLoopImpl::FlushTasks(fml::FlushType)
                                                                          flutter/fml/message_loop_impl.cc:130:5
#16 00030ca4 /data/app-lib/com.example.release_smoke_test-1/libflutter.so 
                                                                          fml::MessageLoopImpl::RunExpiredTasksNow()
                                                                          flutter/fml/message_loop_impl.cc:143:3
#17 00037430 /data/app-lib/com.example.release_smoke_test-1/libflutter.so 
                                                                          fml::MessageLoopAndroid::OnEventFired()
                                                                          flutter/fml/platform/android/message_loop_android.cc:92:5
#18 0003745e /data/app-lib/com.example.release_smoke_test-1/libflutter.so 
                                                                          fml::MessageLoopAndroid::MessageLoopAndroid()::$_0::operator()(int, int, void*) const
                                                                          flutter/fml/platform/android/message_loop_android.cc:42:52
                                                                          fml::MessageLoopAndroid::MessageLoopAndroid()::$_0::__invoke(int, int, void*)
                                                                          flutter/fml/platform/android/message_loop_android.cc:40:40
#19 0001cd61 /system/lib/libutils.so (android::SimpleLooperCallback::handleEvent(int, int, void*)+33)
#20 0001ef5a /system/lib/libutils.so (android::Looper::pollInner(int)+890)
#21 0001f2c3 /system/lib/libutils.so (android::Looper::pollOnce(int, int*, int*, void**)+51)
#22 0000b6ab /system/lib/libandroid.so (ALooper_pollOnce+107)
#23 00037373 /data/app-lib/com.example.release_smoke_test-1/libflutter.so 
                                                                          fml::MessageLoopAndroid::Run()
                                                                          flutter/fml/platform/android/message_loop_android.cc:68:18
#24 00030c52 /data/app-lib/com.example.release_smoke_test-1/libflutter.so 
                                                                          fml::MessageLoopImpl::DoRun()
                                                                          flutter/fml/message_loop_impl.cc:96:3
#25 00030782 /data/app-lib/com.example.release_smoke_test-1/libflutter.so 
                                                                          fml::MessageLoop::Run()
                                                                          flutter/fml/message_loop.cc:49:10
#26 0003485f /data/app-lib/com.example.release_smoke_test-1/libflutter.so 
                                                                          fml::Thread::Thread(std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&)::$_0::operator()() const
                                                                          flutter/fml/thread.cc:35:10
                                                                          decltype(std::__1::forward<fml::Thread::Thread(std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&)::$_0>(fp)()) std::__1::__invoke<fml::Thread::Thread(std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&)::$_0>(fml::Thread::Thread(std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&)::$_0&&)
                                                                          third_party/libcxx/include/type_traits:3530:1
                                                                          void std::__1::__thread_execute<std::__1::unique_ptr<std::__1::__thread_struct, std::__1::default_delete<std::__1::__thread_struct> >, fml::Thread::Thread(std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&)::$_0>(std::__1::tuple<std::__1::unique_ptr<std::__1::__thread_struct, std::__1::default_delete<std::__1::__thread_struct> >, fml::Thread::Thread(std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&)::$_0>&, std::__1::__tuple_indices<>)
                                                                          third_party/libcxx/include/thread:341:5
                                                                          void* std::__1::__thread_proxy<std::__1::tuple<std::__1::unique_ptr<std::__1::__thread_struct, std::__1::default_delete<std::__1::__thread_struct> >, fml::Thread::Thread(std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&)::$_0> >(void*)
                                                                          third_party/libcxx/include/thread:351:5
#27 0000bc0c /system/lib/libc.so (__thread_entry+236)
#28 0003e075 /system/lib/libc.so (__pthread_clone+69)
#29 000347ce /data/app-lib/com.example.release_smoke_test-1/libflutter.so 
                                                                          std::__1::default_delete<std::__1::thread>::operator()(std::__1::thread*) const
                                                                          third_party/libcxx/include/memory:2339:3
#30 0003043f [heap] 
#31 0002e61f [heap]

@zanderso
Copy link
Member Author

Based on the back trace it looks like the error message is relevant:

[ERROR:flutter/shell/gpu/gpu_surface_gl.cc(67)] Failed to setup Skia Gr context.

@zanderso
Copy link
Member Author

Based on the commits in that roll I will guess that this might be related to flutter/engine#23634 and assign @gaaclarke

@zanderso zanderso mentioned this issue Jan 22, 2021
Merged
@gaaclarke gaaclarke mentioned this issue Jan 22, 2021
Merged
9 tasks
@xster
Copy link
Member

xster commented Jan 22, 2021

ummm, not easy to find. Is that test in https://github.com/FirebaseExtended/flutterfire?

@jason-simmons
Copy link
Member

The easiest way to reproduce this is running an app on an Android API level 16 emulator. Apps previously would be unable to create graphics contexts and would fail to render on old emulators. But with the latest engine the app process is now segfaulting.

The failing test is being run on a Firebase test lab by a LUCI recipe:
https://flutter.googlesource.com/recipes/+/refs/heads/master/recipes/firebaselab/firebaselab.py

The test runs the https://github.com/flutter/flutter/tree/master/dev/integration_tests/release_smoke_test app on an emulator.

@jason-simmons
Copy link
Member

This can be fixed by patching flutter/engine@296902b#diff-dde92a82824b129292aacba507c9807c5614fa6529e581d36531877c0d8eba80R108 to:

valid_ = gr_context != nullptr;

@zanderso
Copy link
Member Author

This is no longer P0 since the revert landed and the engine rolled into the framework. I will also close this as it seems the root cause of the issue has been found and can be integrated into the re-land.

@github-actions
Copy link

github-actions bot commented Aug 6, 2021

This thread has been automatically locked since there has not been any recent activity after it was closed. If you are still experiencing a similar issue, please open a new bug, including the output of flutter doctor -v and a minimal reproduction of the issue.

@github-actions github-actions bot locked as resolved and limited conversation to collaborators Aug 6, 2021
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Assignees

@gaaclarke gaaclarke

Labels
c: fatal crash Crashes that terminate the process engine flutter/engine repository. See also e: labels. P0 Critical issues such as a build break or regression
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
5 participants
@xster @zanderso @jason-simmons @gaaclarke @flutter-symbolizer-bot