runtime/vm/os_thread_linux.cc: 264: error: pthread error: 16 (Device or resource busy) #28549

zoechi · 2017-01-28T12:42:30Z

root@5af831c338c0:/app/fhir_client# echo "main() => print('hello');" > test.dart && dart test.dart
hello
runtime/vm/os_thread_linux.cc: 264: error: pthread error: 16 (Device or resource busy)
Aborting re-entrant request for stack trace.
Aborted (core dumped)

Whenever I run any Dart program on my local machine within Docker.
When I deploy the same Docker image to GCloud I don't get this error.

I updated to the latest Docker version

Docker version 1.13.0, build 49bf474f9

but no change.

See also #24222

The text was updated successfully, but these errors were encountered:

zoechi · 2017-01-28T12:44:41Z

Before I updated my Docker version was 1.11.2

Linux zdev 4.9.0-1-amd64 #1 SMP Debian 4.9.2-2 (2017-01-12) x86_64 GNU/Linux

mraleph · 2017-01-30T07:07:06Z

@zoechi could you get us a core dump or at least run it under the gdb and send us output of thread apply all bt

zoechi · 2017-01-30T08:53:09Z

How do I create a core dump? Some basic instructions about how to use gdb?
Never really used gdb.

fsc8000 · 2017-01-31T19:31:02Z

For gdb, you can use gdb --args dart hello.dart

The commands you need are run and after the crash thread apply all bt to print stack traces of all threads.

To get a core file from the command line, the core file size limit should be set to unlimited for the current shell you're running in: ulimit -c unlimited will do that.

zoechi · 2017-02-05T17:36:14Z

Installing gdb fixes the problem 😬

zoechi · 2017-02-05T17:37:27Z

I'm using the Docker image FROM google/dart

zoechi · 2017-02-07T12:27:01Z

I was able to get this core file. It was created automatically when I run the dart script.
Hope this helps.

core.zip

stijnvanbael · 2017-02-08T12:51:52Z

I have a similar issue running pub in a Docker container.

Precompiling dependencies...
Loading source assets...
Loading di/module_transformer transformers...
runtime/vm/os_thread_linux.cc: 265: error: pthread error: 16 (Device or resource busy)
Aborting re-entrant request for stack trace.
Aborted

I'm using Docker 1.13.0 on MacOS Sierra FROM google/dart-runtime-base

I tried using versions 1.21.1 down to 1.19.1

stijnvanbael · 2017-02-15T07:26:58Z

I managed to get pub complete successfully using FROM google/dart instead. Precompilation for some dependencies still fails, but the application starts and seems to be working correctly.

zoechi · 2017-02-15T07:30:01Z

I use ; exit 0 to prevent docker build to fail

RUN pub upgrade --packages-dir; exit 0

pub upgrade and other Dart scripts, run fine, just when it exits it throws the exception.

mraleph · 2017-02-15T14:51:32Z

We tried reproducing it locally without success.

Here are thread stacks @zoechi got for me from his core running gdb inside the container:

(gdb) thread apply all bt

Thread 5 (LWP 10):
#0  0x00007ffa557fa438 in pthread_cond_timedwait@@GLIBC_2.3.2 () at ../nptl/sysdeps/unix/sysv/linux/x86_64/pthread_cond_timedwait.S:287
#1  0x00000000007f2df0 in dart::Monitor::Wait(long) ()
#2  0x000000000090d6a8 in dart::ThreadPool::Worker::Loop() ()
#3  0x000000000090d79f in dart::ThreadPool::Worker::Main(unsigned long) ()
#4  0x00000000007f2106 in ?? ()
#5  0x00007ffa557f60a4 in start_thread (arg=0x7ffa54615700) at pthread_create.c:403
#6  0x00007ffa548fd04d in __libc_ifunc_impl_list (name=<optimized out>, array=0x7ffa54615700, max=<optimized out>) at ../sysdeps/x86_64/multiarch/ifunc-impl-list.c:172
#7  0x0000000000000000 in ?? ()

Thread 4 (LWP 8):
#0  0x00007ffa548fd623 in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:104

Thread 3 (LWP 11):
---Type <return> to continue, or q <return> to quit---
#0  0x00007ffa557ff201 in __lll_lock_elision (futex=0x2667350, adapt_count=0x2667366, private=0) at ../nptl/sysdeps/unix/sysv/linux/x86/elision-lock.c:74
#1  0x00000000006ec51e in dart::FreeList::TryAllocate(long, bool) ()
#2  0x00000000007f4055 in dart::PageSpace::TryAllocateInternal(long, dart::HeapPage::PageType, dart::PageSpace::GrowthPolicy, bool, bool) ()
#3  0x00000000006f06c1 in dart::Heap::AllocateOld(long, dart::HeapPage::PageType) ()
#4  0x000000000078ba67 in dart::Object::Allocate(long, long, dart::Heap::Space) ()
#5  0x000000000078bca8 in ?? ()
#6  0x00000000007ae241 in dart::Array::Grow(dart::Array const&, long, dart::Heap::Space) ()
#7  0x00000000007ae4af in dart::ICData::FindFreeIndex(long*) const ()
#8  0x00000000007b9a44 in dart::ICData::AddReceiverCheck(long, dart::Function const&, long) const ()
#9  0x000000000094e22f in ?? ()
#10 0x0000000000952e3c in dart::DRT_InlineCacheMissHandlerOneArg(dart::NativeArguments) ()
#11 0x00007ffa55a9f47b in ?? ()
#12 0x00000000039cdb00 in ?? ()
#13 0x0000000000000002 in ?? ()
#14 0x00007ffa53cdf048 in ?? ()
---Type <return> to continue, or q <return> to quit---
#15 0x00007ffa53cdf050 in ?? ()
#16 0x00007ffa55ae1551 in ?? ()
#17 0x00007ffa55ae2511 in ?? ()
#18 0x00007ffa53cdf078 in ?? ()
#19 0x00007ffa55aa0768 in ?? ()
#20 0x00007ffa539a98a1 in ?? ()
#21 0x00007ffa540ebec9 in ?? ()
#22 0x0000000000000000 in ?? ()

Thread 2 (LWP 9):
#0  0x00007ffa557fa438 in pthread_cond_timedwait@@GLIBC_2.3.2 () at ../nptl/sysdeps/unix/sysv/linux/x86_64/pthread_cond_timedwait.S:287
#1  0x00000000007f2ec0 in dart::Monitor::WaitMicros(long) ()
#2  0x000000000090c74c in dart::ThreadInterrupter::ThreadMain(unsigned long) ()
#3  0x00000000007f2106 in ?? ()
#4  0x00007ffa557f60a4 in start_thread (arg=0x7ffa54816700) at pthread_create.c:403
---Type <return> to continue, or q <return> to quit---
#5  0x00007ffa548fd04d in __libc_ifunc_impl_list (name=<optimized out>, array=0x7ffa54816700, max=<optimized out>) at ../sysdeps/x86_64/multiarch/ifunc-impl-list.c:172
#6  0x0000000000000000 in ?? ()

Thread 1 (LWP 7):
#0  0x00007ffa5484c107 in __GI_qsort (b=<optimized out>, n=<optimized out>, s=<optimized out>, cmp=<optimized out>) at msort.c:307
#1  0x00007ffa5484d4e8 in __initstate (seed=<optimized out>, arg_state=<optimized out>, n=<optimized out>) at random.c:245
#2  0x0000000000000000 in ?? ()
(gdb)

Notice that Thread 1 stack trace (one that crashed) makes 0 sense.

amoghe · 2017-09-01T03:31:17Z

Still seeing this in the google/dart container.

Setup

I've mounted a volume from the host->container so that I can edit files using an IDE, but run them in the container (so I don't have to install dart on my host)
docker (server) version 17.05.0-ce
dart version (that came with the container) D1.24.2 (Thu Jun 22 15:42:21 2017) on "linux_x64"

Program that causes this

simple program:

root@eeebbc1209df:/# cat /dart/hello/hello.dart 
void main() {
  print('Hello, World!');
}

Output from executing this program

root@eeebbc1209df:/# dart /dart/hello/hello.dart 
Hello, World!
../../runtime/vm/os_thread_linux.cc: 284: error: pthread error: 16 (Device or resource busy)
Aborting re-entrant request for stack trace.
Aborted (core dumped)

Backtrace

(gdb) thread apply all bt

Thread 5 (Thread 0x7f3b14562700 (LWP 10)):
#0  0x00007f3b15342438 in pthread_cond_timedwait@@GLIBC_2.3.2 () from /lib/x86_64-linux-gnu/libpthread.so.0
#1  0x0000000000ab8f3f in dart::Monitor::WaitMicros(long) ()
#2  0x0000000000c06f7f in dart::ThreadInterrupter::ThreadMain(unsigned long) ()
#3  0x0000000000ab8619 in ?? ()
#4  0x00007f3b1533e0a4 in start_thread () from /lib/x86_64-linux-gnu/libpthread.so.0
#5  0x00007f3b1464904d in clone () from /lib/x86_64-linux-gnu/libc.so.6

Thread 4 (Thread 0x7f3b143a1700 (LWP 11)):
#0  0x00007f3b15342438 in pthread_cond_timedwait@@GLIBC_2.3.2 () from /lib/x86_64-linux-gnu/libpthread.so.0
#1  0x0000000000ab8f3f in dart::Monitor::WaitMicros(long) ()
#2  0x0000000000c07d3d in dart::ThreadPool::Worker::Loop() ()
#3  0x0000000000c07b5a in dart::ThreadPool::Worker::Main(unsigned long) ()
#4  0x0000000000ab8619 in ?? ()
#5  0x00007f3b1533e0a4 in start_thread () from /lib/x86_64-linux-gnu/libpthread.so.0
#6  0x00007f3b1464904d in clone () from /lib/x86_64-linux-gnu/libc.so.6

Thread 3 (Thread 0x7f3b1596a700 (LWP 9)):
#0  0x00007f3b14649623 in epoll_wait () from /lib/x86_64-linux-gnu/libc.so.6
#1  0x00000000006fce45 in dart::bin::EventHandlerImplementation::Poll(unsigned long) ()
#2  0x0000000000718f3e in ?? ()
#3  0x00007f3b1533e0a4 in start_thread () from /lib/x86_64-linux-gnu/libpthread.so.0
#4  0x00007f3b1464904d in clone () from /lib/x86_64-linux-gnu/libc.so.6

Thread 2 (Thread 0x7f3b13a30700 (LWP 12)):
#0  0x00000000009c7820 in ?? ()
#1  0x0000000000965b39 in dart::EffectGraphVisitor::VisitArrayNode(dart::ArrayNode*) ()
#2  0x0000000000966133 in dart::EffectGraphVisitor::VisitStringInterpolateNode(dart::StringInterpolateNode*) ()
#3  0x0000000000967be7 in dart::EffectGraphVisitor::BuildPushArguments(dart::ArgumentListNode const&, dart::ZoneGrowableArray<dart::PushArgumentInstr*>*) ()
#4  0x00000000009687e1 in dart::EffectGraphVisitor::VisitStaticCallNode(dart::StaticCallNode*) ()
#5  0x000000000097379b in dart::EffectGraphVisitor::VisitSequenceNode(dart::SequenceNode*) ()
#6  0x000000000097379b in dart::EffectGraphVisitor::VisitSequenceNode(dart::SequenceNode*) ()
#7  0x00000000009763d2 in dart::FlowGraphBuilder::BuildGraph() ()
#8  0x00000000008d41f7 in dart::DartCompilationPipeline::BuildFlowGraph(dart::Zone*, dart::ParsedFunction*, dart::ZoneGrowableArray<dart::ICData const*> const&, long) ()
#9  0x00000000008d685f in dart::CompileParsedFunctionHelper::Compile(dart::CompilationPipeline*) ()
#10 0x00000000008d81ce in ?? ()
#11 0x00000000008d7c08 in dart::Compiler::CompileFunction(dart::Thread*, dart::Function const&) ()
#12 0x00000000008d4487 in dart::DRT_CompileFunction(dart::NativeArguments) ()
#13 0x00007f3b157ea61b in ?? ()
#14 0x0000000003291b00 in ?? ()
#15 0x0000000000000001 in ?? ()
#16 0x00007f3b13a2f0b0 in ?? ()
#17 0x00007f3b13a2f0b8 in ?? ()
#18 0x00007f3b1582c661 in ?? ()
#19 0x00007f3b1582d901 in ?? ()
#20 0x00007f3b13a2f0d8 in ?? ()
#21 0x00007f3b157ea690 in ?? ()
#22 0x00007f3b13b980f1 in ?? ()
#23 0x00007f3b13768941 in ?? ()
#24 0x00007f3b1582cc81 in ?? ()
#25 0x00007f3b137689b1 in ?? ()
#26 0x00007f3b1582d9b1 in ?? ()
#27 0x00007f3b13a2f118 in ?? ()
#28 0x00007f3b137c7aa0 in ?? ()
#29 0x00007f3b13e48dd9 in ?? ()
#30 0x00007f3b13e3be49 in ?? ()
#31 0x00007f3b13e3f2e9 in ?? ()
#32 0x00007f3b1582a021 in ?? ()
---Type <return> to continue, or q <return> to quit---
#33 0x00007f3b13a34621 in ?? ()
#34 0x00007f3b13768a21 in ?? ()
#35 0x00007f3b13a2f150 in ?? ()
#36 0x00007f3b13b459f8 in ?? ()
#37 0x00007f3b13e48dd9 in ?? ()
#38 0x00007f3b13e3f2b9 in ?? ()
#39 0x00007f3b1582c631 in ?? ()
#40 0x00007f3b13a34211 in ?? ()
#41 0x00007f3b13a34681 in ?? ()
#42 0x00007f3b13a2f190 in ?? ()
#43 0x00007f3b13b45907 in ?? ()
#44 0x00007f3b13e48dd9 in ?? ()
#45 0x00007f3b13e3f2b9 in ?? ()
#46 0x00007f3b1582c631 in ?? ()
#47 0x00007f3b1582a021 in ?? ()
#48 0x0000000000000000 in ?? ()

Thread 1 (Thread 0x7f3b1596c780 (LWP 8)):
#0  0x00007f3b14598107 in raise () from /lib/x86_64-linux-gnu/libc.so.6
#1  0x00007f3b145994e8 in abort () from /lib/x86_64-linux-gnu/libc.so.6
#2  0x0000000000ab7d79 in dart::OS::Abort() ()
#3  0x000000000084772b in dart::DynamicAssertionHelper::Fail(char const*, ...) ()
#4  0x0000000000ab89e9 in dart::Mutex::~Mutex() ()
#5  0x00000000009f8cf9 in dart::Isolate::~Isolate() ()
#6  0x000000000090e401 in dart::Dart::ShutdownIsolate() ()
#7  0x000000000082328f in Dart_ShutdownIsolate ()
#8  0x00000000006f65e7 in dart::bin::RunMainIsolate(char const*, dart::bin::CommandLineOptions*) ()
#9  0x00000000006f7aaf in dart::bin::main(int, char**) ()
#10 0x00000000006f7ef9 in main ()
(gdb) q

metalmatze · 2017-09-09T19:57:14Z

I still have a similar problem when running pub get && pub build with google/dart... 😕

mraleph · 2017-09-11T07:26:50Z

We need to understand which mutex is being destroyed. Unfortunately stack traces don't really help - because line information is stripped.

I added some debug information in this CL: https://dart-review.googlesource.com/c/sdk/+/4721

If somebody could build custom dart binary with this patch and try it out that would help a lot.

Use this name when reporting errors. We need this sort of debug information to better understand Isolate shutdown races like one in #28549 Bug: #28549 Change-Id: I24f27b4eef4faf4b2f80f82b269d88a6e5c3880b Reviewed-on: https://dart-review.googlesource.com/4721 Commit-Queue: Vyacheslav Egorov <[email protected]> Reviewed-by: Zach Anderson <[email protected]>

itsjoeconway · 2017-09-26T17:34:10Z

Currently working around this by adding --no-precompile flag to pub get.

muzuro · 2017-11-27T11:58:38Z

I have similar error when executing pub run test in gitlab ci using google/dart image

pub run test

00:00 +0: loading test/util_test.dart                                                                                                                                                                  
00:00 +0: loading test/util_test.dart                                                                                                                                                                  
time is 11:54:AM
1970-01-01 11:54:00.000
1970-01-01 11:54:00.000
No tests ran.
../../runtime/vm/os_thread_linux.cc: 284: error: pthread error: 16 (Device or resource busy)

ircecho · 2017-11-28T09:09:31Z

@muzuro The problem occurs when a dart program is running, and since pub is written in dart, it is affected as well. Try to use the 1.25-dev version. It might fix the problem.

zoechi · 2020-05-15T10:52:49Z

I guess this can be closed. If someone sees this. Please add a comment to have it reopened or create a new one an link here.

mit-mit added the area-vm Use area-vm for VM related issues, including code coverage, and the AOT and JIT backends. label Jan 29, 2017

metalmatze mentioned this issue Sep 9, 2017

Building angular.dart inside of google's dart docker container crashes dart-archive/angular.dart#1773

Closed

zoechi closed this as completed May 15, 2020

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

runtime/vm/os_thread_linux.cc: 264: error: pthread error: 16 (Device or resource busy) #28549

runtime/vm/os_thread_linux.cc: 264: error: pthread error: 16 (Device or resource busy) #28549

zoechi commented Jan 28, 2017

zoechi commented Jan 28, 2017 •

edited

Loading

mraleph commented Jan 30, 2017

zoechi commented Jan 30, 2017

fsc8000 commented Jan 31, 2017

zoechi commented Feb 5, 2017

zoechi commented Feb 5, 2017

zoechi commented Feb 7, 2017

stijnvanbael commented Feb 8, 2017

stijnvanbael commented Feb 15, 2017

zoechi commented Feb 15, 2017

mraleph commented Feb 15, 2017

amoghe commented Sep 1, 2017

metalmatze commented Sep 9, 2017

mraleph commented Sep 11, 2017

itsjoeconway commented Sep 26, 2017

muzuro commented Nov 27, 2017 •

edited

Loading

ircecho commented Nov 28, 2017

zoechi commented May 15, 2020

runtime/vm/os_thread_linux.cc: 264: error: pthread error: 16 (Device or resource busy) #28549

runtime/vm/os_thread_linux.cc: 264: error: pthread error: 16 (Device or resource busy) #28549

Comments

zoechi commented Jan 28, 2017

zoechi commented Jan 28, 2017 • edited Loading

mraleph commented Jan 30, 2017

zoechi commented Jan 30, 2017

fsc8000 commented Jan 31, 2017

zoechi commented Feb 5, 2017

zoechi commented Feb 5, 2017

zoechi commented Feb 7, 2017

stijnvanbael commented Feb 8, 2017

stijnvanbael commented Feb 15, 2017

zoechi commented Feb 15, 2017

mraleph commented Feb 15, 2017

amoghe commented Sep 1, 2017

Setup

Program that causes this

Output from executing this program

Backtrace

metalmatze commented Sep 9, 2017

mraleph commented Sep 11, 2017

itsjoeconway commented Sep 26, 2017

muzuro commented Nov 27, 2017 • edited Loading

ircecho commented Nov 28, 2017

zoechi commented May 15, 2020

zoechi commented Jan 28, 2017 •

edited

Loading

muzuro commented Nov 27, 2017 •

edited

Loading