[ocaml5-issue] s390x thread_createtree failures

[jmid]

On s390x we are seeing failures of Thread 6995 killed on uncaught exception Sys_error("Thread.create: Resource temporarily unavailable")

For 5.2 (still bytecode due to a misconfigured opam switch) s390x this is triggering failures:
https://ocaml-multicoretests.ci.dev:8100/job/2023-07-10/154501-ci-ocluster-build-649336

random seed: 435876653
generated error fail pass / total     time test name

[ ]    0    0    0    0 / 1000     0.0s thread_createtree - with Atomic
[ ]    0    0    0    0 / 1000     0.0s thread_createtree - with Atomic (generating)Thread 6995 killed on uncaught exception Sys_error("Thread.create: Resource temporarily unavailable")
Thread 7000 killed on uncaught exception Sys_error("Thread.create: Resource temporarily unavailable")
Thread 70863 killed on uncaught exception Sys_error("Thread.create: Resource temporarily unavailable")
Thread 71047 killed on uncaught exception Sys_error("Thread.create: Resource temporarily unavailable")
Thread 71289 killed on uncaught exception Sys_error("Thread.create: Resource temporarily unavailable")
Thread 341741 killed on uncaught exception Sys_error("Thread.create: Resource temporarily unavailable")
Thread 341742 killed on uncaught exception Sys_error("Thread.create: Resource temporarily unavailable")
Thread 438647 killed on uncaught exception Sys_error("Thread.create: Resource temporarily unavailable")
Thread 582731 killed on uncaught exception Sys_error("Thread.create: Resource temporarily unavailable")
Thread 656832 killed on uncaught exception Sys_error("Thread.create: Resource temporarily unavailable")
Thread 656674 killed on uncaught exception Sys_error("Thread.create: Resource temporarily unavailable")
Thread 656850 killed on uncaught exception Sys_error("Thread.create: Resource temporarily unavailable")
Thread 833969 killed on uncaught exception Sys_error("Thread.create: Resource temporarily unavailable")
Thread 1013487 killed on uncaught exception Sys_error("Thread.create: Resource temporarily unavailable")
Thread 1013488 killed on uncaught exception Sys_error("Thread.create: Resource temporarily unavailable")
Thread 1013492 killed on uncaught exception Sys_error("Thread.create: Resource temporarily unavailable")
Thread 1013496 killed on uncaught exception Sys_error("Thread.create: Resource temporarily unavailable")
Thread 1013491 killed on uncaught exception Sys_error("Thread.create: Resource temporarily unavailable")
Thread 1013506 killed on uncaught exception Sys_error("Thread.create: Resource temporarily unavailable")
Thread 1013508 killed on uncaught exception Sys_error("Thread.create: Resource temporarily unavailable")
Thread 1013361 killed on uncaught exception Sys_error("Thread.create: Resource temporarily unavailable")
Thread 1013368 killed on uncaught exception Sys_error("Thread.create: Resource temporarily unavailable")
Thread 1179011 killed on uncaught exception Sys_error("Thread.create: Resource temporarily unavailable")
Thread 1179012 killed on uncaught exception Sys_error("Thread.create: Resource temporarily unavailable")
Thread 1179016 killed on uncaught exception Sys_error("Thread.create: Resource temporarily unavailable")
Thread 1179020 killed on uncaught exception Sys_error("Thread.create: Resource temporarily unavailable")
Thread 1179030 killed on uncaught exception Sys_error("Thread.create: Resource temporarily unavailable")
Thread 1179031 killed on uncaught exception Sys_error("Thread.create: Resource temporarily unavailable")
Thread 1178981 killed on uncaught exception Sys_error("Thread.create: Resource temporarily unavailable")
Thread 1179034 killed on uncaught exception Sys_error("Thread.create: Resource temporarily unavailable")
Thread 1179038 killed on uncaught exception Sys_error("Thread.create: Resource temporarily unavailable")
Thread 1179049 killed on uncaught exception Sys_error("Thread.create: Resource temporarily unavailable")
Thread 1179055 killed on uncaught exception Sys_error("Thread.create: Resource temporarily unavailable")
Thread 1492327 killed on uncaught exception Sys_error("Thread.create: Resource temporarily unavailable")
Thread 1492330 killed on uncaught exception Sys_error("Thread.create: Resource temporarily unavailable")
Thread 1492331 killed on uncaught exception Sys_error("Thread.create: Resource temporarily unavailable")
Thread 1492332 killed on uncaught exception Sys_error("Thread.create: Resource temporarily unavailable")
Thread 1588743 killed on uncaught exception Sys_error("Thread.create: Resource temporarily unavailable")
Thread 1588570 killed on uncaught exception Sys_error("Thread.create: Resource temporarily unavailable")
Thread 1588767 killed on uncaught exception Sys_error("Thread.create: Resource temporarily unavailable")
Thread 1588635 killed on uncaught exception Sys_error("Thread.create: Resource temporarily unavailable")
Thread 1588772 killed on uncaught exception Sys_error("Thread.create: Resource temporarily unavailable")
Thread 1588776 killed on uncaught exception Sys_error("Thread.create: Resource temporarily unavailable")
Thread 1588777 killed on uncaught exception Sys_error("Thread.create: Resource temporarily unavailable")
Thread 1588782 killed on uncaught exception Sys_error("Thread.create: Resource temporarily unavailable")
Thread 1660571 killed on uncaught exception Sys_error("Thread.create: Resource temporarily unavailable")
Thread 1660714 killed on uncaught exception Sys_error("Thread.create: Resource temporarily unavailable")

[ ]  222    0    0  222 / 1000    61.2s thread_createtree - with Atomic (shrinking:   10.0006)Thread 1931339 killed on uncaught exception Sys_error("Thread.create: Resource temporarily unavailable")
Thread 1931210 killed on uncaught exception Sys_error("Thread.create: Resource temporarily unavailable")
Thread 1931352 killed on uncaught exception Sys_error("Thread.create: Resource temporarily unavailable")
Thread 2394358 killed on uncaught exception Sys_error("Thread.create: Resource temporarily unavailable")
Thread 2454627 killed on uncaught exception Sys_error("Thread.create: Resource temporarily unavailable")
...
Thread 15623009 killed on uncaught exception Sys_error("Thread.create: Resource temporarily unavailable")
Thread 15741116 killed on uncaught exception Sys_error("Thread.create: Resource temporarily unavailable")
Thread 15776928 killed on uncaught exception Sys_error("Thread.create: Resource temporarily unavailable")
Thread 15828124 killed on uncaught exception Sys_error("Thread.create: Resource temporarily unavailable")

[ ]  222    0    0  222 / 1000   545.3s thread_createtree - with Atomic (shrinking:  114.0006)Thread 15937931 killed on uncaught exception Sys_error("Thread.create: Resource temporarily unavailable")
Thread 15988336 killed on uncaught exception Sys_error("Thread.create: Resource temporarily unavailable")
Thread 16012902 killed on uncaught exception Sys_error("Thread.create: Resource temporarily unavailable")
Thread 16024751 killed on uncaught exception Sys_error("Thread.create: Resource temporarily unavailable")
Thread 16024757 killed on uncaught exception Sys_error("Thread.create: Resource temporarily unavailable")
Thread 16054847 killed on uncaught exception Sys_error("Thread.create: Resource temporarily unavailable")
Thread 16058083 killed on uncaught exception Sys_error("Thread.create: Resource temporarily unavailable")
Thread 16074139 killed on uncaught exception Sys_error("Thread.create: Resource temporarily unavailable")
Thread 16074162 killed on uncaught exception Sys_error("Thread.create: Resource temporarily unavailable")
Thread 16117778 killed on uncaught exception Sys_error("Thread.create: Resource temporarily unavailable")
Thread 16161213 killed on uncaught exception Sys_error("Thread.create: Resource temporarily unavailable")
Thread 16177150 killed on uncaught exception Sys_error("Thread.create: Resource temporarily unavailable")
Thread 16177148 killed on uncaught exception Sys_error("Thread.create: Resource temporarily unavailable")
Thread 16177149 killed on uncaught exception Sys_error("Thread.create: Resource temporarily unavailable")
Thread 16223585 killed on uncaught exception Sys_error("Thread.create: Resource temporarily unavailable")
Thread 16288414 killed on uncaught exception Sys_error("Thread.create: Resource temporarily unavailable")
Thread 16329149 killed on uncaught exception Sys_error("Thread.create: Resource temporarily unavailable")
Thread 16342189 killed on uncaught exception Sys_error("Thread.create: Resource temporarily unavailable")
Thread 16438406 killed on uncaught exception Sys_error("Thread.create: Resource temporarily unavailable")
Thread 16445608 killed on uncaught exception Sys_error("Thread.create: Resource temporarily unavailable")
Thread 16447161 killed on uncaught exception Sys_error("Thread.create: Resource temporarily unavailable")
Thread 16506497 killed on uncaught exception Sys_error("Thread.create: Resource temporarily unavailable")
Thread 16519258 killed on uncaught exception Sys_error("Thread.create: Resource temporarily unavailable")
Thread 16537179 killed on uncaught exception Sys_error("Thread.create: Resource temporarily unavailable")
Thread 16550489 killed on uncaught exception Sys_error("Thread.create: Resource temporarily unavailable")
Thread 16598640 killed on uncaught exception Sys_error("Thread.create: Resource temporarily unavailable")

[✗]  223    0    1  222 / 1000   585.4s thread_createtree - with Atomic

--- Failure --------------------------------------------------------------------

Test thread_createtree - with Atomic failed (136 shrink steps):

Create [Decr; Decr; Incr; Incr; Incr; Incr; Create [Incr; Incr; Incr; Incr;
Incr; Create [Create [Incr; Decr; Incr; Decr]; Decr; Create [Decr]; Decr;
Create [Create [Incr; Incr; Incr]]; Create [Create [Decr]; Decr;
Create [Decr; Incr; Incr]; Decr; Decr; Decr; Create [Incr; Decr; Decr]]]];
Create [Incr;
Incr]]
================================================================================
failure (1 tests failed, 0 tests errored, ran 1 tests)
File "src/thread/dune", line 14, characters 7-24:
14 |  (name thread_createtree)
            ^^^^^^^^^^^^^^^^^
(cd _build/default/src/thread && ./thread_createtree.exe --verbose)

[jmid]

I also observed this a month back on a CI run after rebasing #304.
The symptoms were the same: s390x running trunk/5.2 (bytecode back too then I suspect), also failing on thread_createtree:
https://ocaml-multicoretests.ci.dev:8100/job/2023-06-07/124748-ci-ocluster-build-33c2c3

random seed: 486091516
generated error fail pass / total     time test name

[ ]    0    0    0    0 / 1000     0.0s thread_createtree - with Atomic
[ ]    0    0    0    0 / 1000     0.0s thread_createtree - with Atomic (generating)Thread 14618 killed on uncaught exception Sys_error("Thread.create: Resource temporarily unavailable")
Thread 14657 killed on uncaught exception Sys_error("Thread.create: Resource temporarily unavailable")
Thread 14659 killed on uncaught exception Out_of_memory
Thread 14463 killed on uncaught exception Sys_error("Thread.create: Resource temporarily unavailable")
Thread 14465 killed on uncaught exception Sys_error("Thread.create: Resource temporarily unavailable")
Thread 62925 killed on uncaught exception Sys_error("Thread.create: Resource temporarily unavailable")
Thread 62923 killed on uncaught exception Sys_error("Thread.create: Resource temporarily unavailable")
Thread 62805 killed on uncaught exception Sys_error("Thread.create: Resource temporarily unavailable")
Thread 62926 killed on uncaught exception Sys_error("Thread.create: Resource temporarily unavailable")
Thread 62930 killed on uncaught exception Sys_error("Thread.create: Resource temporarily unavailable")
Thread 71263 killed on uncaught exception Sys_error("Thread.create: Resource temporarily unavailable")
Thread 71267 killed on uncaught exception Sys_error("Thread.create: Resource temporarily unavailable")
Thread 119680 killed on uncaught exception Sys_error("Thread.create: Resource temporarily unavailable")
Thread 119430 killed on uncaught exception Sys_error("Thread.create: Resource temporarily unavailable")
Thread 119889 killed on uncaught exception Sys_error("Thread.create: Resource temporarily unavailable")
Thread 142068 killed on uncaught exception Sys_error("Thread.create: Resource temporarily unavailable")
Thread 142070 killed on uncaught exception Sys_error("Thread.create: Resource temporarily unavailable")
Thread 142073 killed on uncaught exception Sys_error("Thread.create: Resource temporarily unavailable")
Thread 142085 killed on uncaught exception Sys_error("Thread.create: Resource temporarily unavailable")
Thread 142086 killed on uncaught exception Sys_error("Thread.create: Resource temporarily unavailable")
Thread 141998 killed on uncaught exception Sys_error("Thread.create: Resource temporarily unavailable")
Thread 142089 killed on uncaught exception Sys_error("Thread.create: Resource temporarily unavailable")
Thread 141965 killed on uncaught exception Sys_error("Thread.create: Resource temporarily unavailable")

[ ]  391    0    0  391 / 1000    61.5s thread_createtree - with Atomic (shrinking:    4.0011)Thread 955418 killed on uncaught exception Sys_error("Thread.create: Resource temporarily unavailable")
Thread 955553 killed on uncaught exception Sys_error("Thread.create: Resource temporarily unavailable")
Thread 955548 killed on uncaught exception Sys_error("Thread.create: Resource temporarily unavailable")
Thread 970015 killed on uncaught exception Sys_error("Thread.create: Resource temporarily unavailable")
Thread 1030022 killed on uncaught exception Sys_error("Thread.create: Resource temporarily unavailable")
Thread 1255346 killed on uncaught exception Sys_error("Thread.create: Resource temporarily unavailable")
Thread 1255347 killed on uncaught exception Sys_error("Thread.create: Resource temporarily unavailable")
Thread 1255349 killed on uncaught exception Sys_error("Thread.create: Resource temporarily unavailable")
Thread 1255351 killed on uncaught exception Sys_error("Thread.create: Resource temporarily unavailable")

[ ]  391    0    0  391 / 1000   122.0s thread_createtree - with Atomic (shrinking:    8.0005)Thread 1388072 killed on uncaught exception Sys_error("Thread.create: Resource temporarily unavailable")
Thread 1388152 killed on uncaught exception Sys_error("Thread.create: Resource temporarily unavailable")
Thread 1388106 killed on uncaught exception Sys_error("Thread.create: Resource temporarily unavailable")
Thread 1388166 killed on uncaught exception Sys_error("Thread.create: Resource temporarily unavailable")
Thread 1553246 killed on uncaught exception Sys_error("Thread.create: Resource temporarily unavailable")
Thread 1553254 killed on uncaught exception Sys_error("Thread.create: Resource temporarily unavailable")
Thread 1553255 killed on uncaught exception Sys_error("Thread.create: Resource temporarily unavailable")
Thread 1553268 killed on uncaught exception Sys_error("Thread.create: Resource temporarily unavailable")
Thread 1577694 killed on uncaught exception Sys_error("Thread.create: Resource temporarily unavailable")
Thread 1577801 killed on uncaught exception Sys_error("Thread.create: Resource temporarily unavailable")
Thread 1654427 killed on uncaught exception Sys_error("Thread.create: Resource temporarily unavailable")
Thread 1798024 killed on uncaught exception Sys_error("Thread.create: Resource temporarily unavailable")
Thread 1798028 killed on uncaught exception Sys_error("Thread.create: Resource temporarily unavailable")
Thread 1798000 killed on uncaught exception Sys_error("Thread.create: Resource temporarily unavailable")

[ ]  391    0    0  391 / 1000   183.0s thread_createtree - with Atomic (shrinking:   13.0005)Thread 1887774 killed on uncaught exception Sys_error("Thread.create: Resource temporarily unavailable")
Thread 2081799 killed on uncaught exception Sys_error("Thread.create: Resource temporarily unavailable")
Thread 2081802 killed on uncaught exception Sys_error("Thread.create: Resource temporarily unavailable")
Thread 2081803 killed on uncaught exception Sys_error("Thread.create: Resource temporarily unavailable")
Thread 2081816 killed on uncaught exception Sys_error("Thread.create: Resource temporarily unavailable")
Thread 2164012 killed on uncaught exception Sys_error("Thread.create: Resource temporarily unavailable")

[ ]  391    0    0  391 / 1000   245.3s thread_createtree - with Atomic (shrinking:   16.0014)Thread 2612295 killed on uncaught exception Sys_error("Thread.create: Resource temporarily unavailable")
Thread 2612251 killed on uncaught exception Sys_error("Thread.create: Resource temporarily unavailable")

....

[ ]  391    0    0  391 / 1000   670.6s thread_createtree - with Atomic (shrinking:   38.0005)Thread 6520508 killed on uncaught exception Sys_error("Thread.create: Resource temporarily unavailable")
Thread 6520533 killed on uncaught exception Sys_error("Thread.create: Resource temporarily unavailable")
Thread 6554873 killed on uncaught exception Sys_error("Thread.create: Resource temporarily unavailable")
Thread 6569032 killed on uncaught exception Sys_error("Thread.create: Resource temporarily unavailable")
Thread 6569070 killed on uncaught exception Sys_error("Thread.create: Resource temporarily unavailable")
Thread 6837251 killed on uncaught exception Sys_error("Thread.create: Resource temporarily unavailable")
Thread 6837250 killed on uncaught exception Sys_error("Thread.create: Resource temporarily unavailable")
Thread 6923848 killed on uncaught exception Sys_error("Thread.create: Resource temporarily unavailable")
Thread 6944672 killed on uncaught exception Sys_error("Thread.create: Resource temporarily unavailable")
Thread 6977521 killed on uncaught exception Sys_error("Thread.create: Resource temporarily unavailable")
Thread 6977507 killed on uncaught exception Sys_error("Thread.create: Resource temporarily unavailable")
Thread 6977519 killed on uncaught exception Sys_error("Thread.create: Resource temporarily unavailable")
Thread 6977558 killed on uncaught exception Sys_error("Thread.create: Resource temporarily unavailable")
Thread 6977532 killed on uncaught exception Sys_error("Thread.create: Resource temporarily unavailable")
Thread 6977535 killed on uncaught exception Sys_error("Thread.create: Resource temporarily unavailable")
Thread 7028464 killed on uncaught exception Sys_error("Thread.create: Resource temporarily unavailable")
Thread 7028467 killed on uncaught exception Sys_error("Thread.create: Resource temporarily unavailable")
Thread 7028468 killed on uncaught exception Sys_error("Thread.create: Resource temporarily unavailable")
Thread 7076874 killed on uncaught exception Sys_error("Thread.create: Resource temporarily unavailable")
Thread 7076876 killed on uncaught exception Sys_error("Thread.create: Resource temporarily unavailable")
Thread 7076881 killed on uncaught exception Sys_error("Thread.create: Resource temporarily unavailable")

[ ]  391    0    0  391 / 1000   731.2s thread_createtree - with Atomic (shrinking:   47.0005)Thread 7131357 killed on uncaught exception Sys_error("Thread.create: Resource temporarily unavailable")
Thread 7131377 killed on uncaught exception Sys_error("Thread.create: Resource temporarily unavailable")
Thread 7131354 killed on uncaught exception Sys_error("Thread.create: Resource temporarily unavailable")
Thread 7131364 killed on uncaught exception Sys_error("Thread.create: Resource temporarily unavailable")
Thread 7176810 killed on uncaught exception Sys_error("Thread.create: Resource temporarily unavailable")
Thread 7277754 killed on uncaught exception Sys_error("Thread.create: Resource temporarily unavailable")
Thread 7277742 killed on uncaught exception Sys_error("Thread.create: Resource temporarily unavailable")
Thread 7353568 killed on uncaught exception Sys_error("Thread.create: Resource temporarily unavailable")
Thread 7384930 killed on uncaught exception Sys_error("Thread.create: Resource temporarily unavailable")
Thread 7384928 killed on uncaught exception Sys_error("Thread.create: Resource temporarily unavailable")
Thread 7479172 killed on uncaught exception Sys_error("Thread.create: Resource temporarily unavailable")
Thread 7532438 killed on uncaught exception Sys_error("Thread.create: Resource temporarily unavailable")
Thread 7576586 killed on uncaught exception Sys_error("Thread.create: Resource temporarily unavailable")
Thread 7587878 killed on uncaught exception Sys_error("Thread.create: Resource temporarily unavailable")
Thread 7665669 killed on uncaught exception Sys_error("Thread.create: Resource temporarily unavailable")
Thread 7665673 killed on uncaught exception Sys_error("Thread.create: Resource temporarily unavailable")
Thread 7670562 killed on uncaught exception Sys_error("Thread.create: Resource temporarily unavailable")

[ ]  391    0    0  391 / 1000   791.8s thread_createtree - with Atomic (shrinking:   57.0004)Thread 7756211 killed on uncaught exception Sys_error("Thread.create: Resource temporarily unavailable")
Thread 7766333 killed on uncaught exception Sys_error("Thread.create: Resource temporarily unavailable")
Thread 7812863 killed on uncaught exception Sys_error("Thread.create: Resource temporarily unavailable")
Thread 7812867 killed on uncaught exception Sys_error("Thread.create: Resource temporarily unavailable")
Thread 7853416 killed on uncaught exception Sys_error("Thread.create: Resource temporarily unavailable")
Thread 8055347 killed on uncaught exception Sys_error("Thread.create: Resource temporarily unavailable")
Thread 8111048 killed on uncaught exception Sys_error("Thread.create: Resource temporarily unavailable")
Thread 8113984 killed on uncaught exception Sys_error("Thread.create: Resource temporarily unavailable")
Thread 8201229 killed on uncaught exception Sys_error("Thread.create: Resource temporarily unavailable")
Thread 8201206 killed on uncaught exception Sys_error("Thread.create: Resource temporarily unavailable")
Thread 8265601 killed on uncaught exception Sys_error("Thread.create: Resource temporarily unavailable")
Thread 8265603 killed on uncaught exception Sys_error("Thread.create: Resource temporarily unavailable")
Thread 8265605 killed on uncaught exception Sys_error("Thread.create: Resource temporarily unavailable")
Thread 8292716 killed on uncaught exception Sys_error("Thread.create: Resource temporarily unavailable")
Thread 8292715 killed on uncaught exception Sys_error("Thread.create: Resource temporarily unavailable")

[ ]  391    0    0  391 / 1000   852.3s thread_createtree - with Atomic (shrinking:   65)Thread 8315799 killed on uncaught exception Sys_error("Thread.create: Resource temporarily unavailable")
Thread 8315804 killed on uncaught exception Sys_error("Thread.create: Resource temporarily unavailable")
Thread 8325638 killed on uncaught exception Sys_error("Thread.create: Resource temporarily unavailable")

[✗]  392    0    1  391 / 1000   897.5s thread_createtree - with Atomic

--- Failure --------------------------------------------------------------------

Test thread_createtree - with Atomic failed (67 shrink steps):

Create [Incr; Incr; Incr; Incr; Incr; Decr; Decr; Create [Incr; Incr; Decr;
Incr; Create [Incr; Incr; Incr]; Create [Create [Incr; Incr; Create [Decr;
Incr; Decr; Decr; Incr; Decr; Decr]; Create [Incr; Decr; Decr; Decr; Decr;
Decr; Incr; Decr; Decr]; Create [Decr]; Decr; Incr; Create [Incr; Decr; Incr;
Decr]]]; Incr; Create [Incr; Create [Decr; Create [Incr; Decr]]; Create [];
Incr; Create [Decr; Create [Incr; Incr; Decr; Decr; Incr]]]]; Create [Decr];
Decr]
================================================================================
failure (1 tests failed, 0 tests errored, ran 1 tests)
File "src/thread/dune", line 14, characters 7-24:
14 |  (name thread_createtree)
            ^^^^^^^^^^^^^^^^^
(cd _build/default/src/thread && ./thread_createtree.exe --verbose)
Command exited with code 1.

[jmid]

With a repetition count of 20:
https://github.com/ocaml-multicore/multicoretests/tree/thread-createtree-focus
I was able to trigger this on 2/20 test runs:
https://github.com/ocaml-multicore/multicoretests/runs/14952327534

It is puzzling that it is triggered on s390x and 5.2/trunk only 🤔

[jmid]

Observed this again on s390x 5.2 when merging #376 into main:
https://ocaml-multicoretests.ci.dev:8100/job/2023-07-14/221157-ci-ocluster-build-52b7b3

random seed: 362338500
generated error fail pass / total     time test name

[ ]    0    0    0    0 / 1000     0.0s thread_createtree - with Atomic
[ ]    0    0    0    0 / 1000     0.0s thread_createtree - with Atomic (generating)Thread 16216 killed on uncaught exception Sys_error("Thread.create: Resource temporarily unavailable")
Thread 16218 killed on uncaught exception Sys_error("Thread.create: Resource temporarily unavailable")
Thread 16219 killed on uncaught exception Sys_error("Thread.create: Resource temporarily unavailable")
Thread 168684 killed on uncaught exception Sys_error("Thread.create: Resource temporarily unavailable")
Thread 168689 killed on uncaught exception Sys_error("Thread.create: Resource temporarily unavailable")
Thread 168691 killed on uncaught exception Sys_error("Thread.create: Resource temporarily unavailable")
Thread 168692 killed on uncaught exception Sys_error("Thread.create: Resource temporarily unavailable")
Thread 168698 killed on uncaught exception Sys_error("Thread.create: Resource temporarily unavailable")
Thread 168726 killed on uncaught exception Sys_error("Thread.create: Resource temporarily unavailable")
Thread 168731 killed on uncaught exception Sys_error("Thread.create: Resource temporarily unavailable")
Thread 168734 killed on uncaught exception Sys_error("Thread.create: Resource temporarily unavailable")
Thread 168735 killed on uncaught exception Sys_error("Thread.create: Resource temporarily unavailable")
...
Thread 35103029 killed on uncaught exception Sys_error("Thread.create: Resource temporarily unavailable")
Thread 35179609 killed on uncaught exception Sys_error("Thread.create: Resource temporarily unavailable")
Thread 35207059 killed on uncaught exception Sys_error("Thread.create: Resource temporarily unavailable")
Thread 35281640 killed on uncaught exception Sys_error("Thread.create: Resource temporarily unavailable")
Thread 35288013 killed on uncaught exception Sys_error("Thread.create: Resource temporarily unavailable")
Thread 35300159 killed on uncaught exception Sys_error("Thread.create: Resource temporarily unavailable")
Thread 35365424 killed on uncaught exception Sys_error("Thread.create: Resource temporarily unavailable")
Thread 35382127 killed on uncaught exception Sys_error("Thread.create: Resource temporarily unavailable")
Thread 35464898 killed on uncaught exception Sys_error("Thread.create: Resource temporarily unavailable")
Thread 35533394 killed on uncaught exception Sys_error("Thread.create: Resource temporarily unavailable")
Thread 35562296 killed on uncaught exception Sys_error("Thread.create: Resource temporarily unavailable")
Thread 35578194 killed on uncaught exception Sys_error("Thread.create: Resource temporarily unavailable")
Thread 35685741 killed on uncaught exception Sys_error("Thread.create: Resource temporarily unavailable")
Thread 35739951 killed on uncaught exception Sys_error("Thread.create: Resource temporarily unavailable")
Thread 35748511 killed on uncaught exception Sys_error("Thread.create: Resource temporarily unavailable")
Thread 35765594 killed on uncaught exception Sys_error("Thread.create: Resource temporarily unavailable")
Thread 35772051 killed on uncaught exception Sys_error("Thread.create: Resource temporarily unavailable")

[✗]  355    1    0  354 / 1000   889.5s thread_createtree - with Atomic

=== Error ======================================================================

Test thread_createtree - with Atomic errored on (10 shrink steps):

Create [Incr]

exception Sys_error("Thread.create: Resource temporarily unavailable")

================================================================================
failure (0 tests failed, 1 tests errored, ran 1 tests)
File "src/thread/dune", line 14, characters 7-24:
14 |  (name thread_createtree)
            ^^^^^^^^^^^^^^^^^
(cd _build/default/src/thread && ./thread_createtree.exe --verbose)
Command exited with code 1.