test_multiprocessing_spawn.test_manager: _TestCondition hung (20 min timeout) on AMD64 RHEL8 3.x #110206
Labels
Comments
|
Similar error on PPC64LE RHEL8 Refleaks 3.x: https://buildbot.python.org/all/#/builders/384/builds/892 |
|
I failed to reproduce the issue on my Fedora 38 by stressing my laptop (12 logical CPUs) with:
I ran the test for 3 min 30 sec. |
|
I didn't see this failure recently, I close the issue. |
|
I've seen this a few times recently. For example on the GH ubuntu-24.04-arm runner: https://github.com/python/cpython/actions/runs/13642688178/job/38135745891?pr=130811 |
colesbury
added a commit
to colesbury/cpython
that referenced
this issue
Mar 6, 2025
The test could deadlock trying join on the worker processes due to a combination of behaviors: * The use of `assertReachesEventually` did not ensure that workers actually woken.release() because the SyncManager's Semaphore does not implement get_value. * This mean that the test could finish and the variable "sleeping" would got out of scope and be collected. This unregisters the proxy leading to failures in the worker or possibly the manager. * The subsequent call to `p.join()` during cleanUp therefore never finished. This takes two approaches to fix this: 1) Use woken.acquire() to ensure that the workers actually finish calling woken.release() 2) Wait until the workers finish during the test, while cond, sleeping, and woken are still valid.
colesbury
added a commit
that referenced
this issue
Mar 7, 2025
The test could deadlock trying join on the worker processes due to a combination of behaviors: * The use of `assertReachesEventually` did not ensure that workers actually woken.release() because the SyncManager's Semaphore does not implement get_value. * This mean that the test could finish and the variable "sleeping" would got out of scope and be collected. This unregisters the proxy leading to failures in the worker or possibly the manager. * The subsequent call to `p.join()` during cleanUp therefore never finished. This takes two approaches to fix this: 1) Use woken.acquire() to ensure that the workers actually finish calling woken.release() 2) At the end of the test, wait until the workers are finished, while `cond`, `sleeping`, and `woken` are still valid.
miss-islington
pushed a commit
to miss-islington/cpython
that referenced
this issue
Mar 7, 2025
The test could deadlock trying join on the worker processes due to a combination of behaviors: * The use of `assertReachesEventually` did not ensure that workers actually woken.release() because the SyncManager's Semaphore does not implement get_value. * This mean that the test could finish and the variable "sleeping" would got out of scope and be collected. This unregisters the proxy leading to failures in the worker or possibly the manager. * The subsequent call to `p.join()` during cleanUp therefore never finished. This takes two approaches to fix this: 1) Use woken.acquire() to ensure that the workers actually finish calling woken.release() 2) At the end of the test, wait until the workers are finished, while `cond`, `sleeping`, and `woken` are still valid. (cherry picked from commit c476410) Co-authored-by: Sam Gross <[email protected]>
miss-islington
pushed a commit
to miss-islington/cpython
that referenced
this issue
Mar 7, 2025
The test could deadlock trying join on the worker processes due to a combination of behaviors: * The use of `assertReachesEventually` did not ensure that workers actually woken.release() because the SyncManager's Semaphore does not implement get_value. * This mean that the test could finish and the variable "sleeping" would got out of scope and be collected. This unregisters the proxy leading to failures in the worker or possibly the manager. * The subsequent call to `p.join()` during cleanUp therefore never finished. This takes two approaches to fix this: 1) Use woken.acquire() to ensure that the workers actually finish calling woken.release() 2) At the end of the test, wait until the workers are finished, while `cond`, `sleeping`, and `woken` are still valid. (cherry picked from commit c476410) Co-authored-by: Sam Gross <[email protected]>
This was referenced Mar 7, 2025
colesbury
added a commit
that referenced
this issue
Mar 7, 2025
…30951) The test could deadlock trying join on the worker processes due to a combination of behaviors: * The use of `assertReachesEventually` did not ensure that workers actually woken.release() because the SyncManager's Semaphore does not implement get_value. * This mean that the test could finish and the variable "sleeping" would got out of scope and be collected. This unregisters the proxy leading to failures in the worker or possibly the manager. * The subsequent call to `p.join()` during cleanUp therefore never finished. This takes two approaches to fix this: 1) Use woken.acquire() to ensure that the workers actually finish calling woken.release() 2) At the end of the test, wait until the workers are finished, while `cond`, `sleeping`, and `woken` are still valid. (cherry picked from commit c476410) Co-authored-by: Sam Gross <[email protected]>
colesbury
added a commit
that referenced
this issue
Mar 7, 2025
…30950) The test could deadlock trying join on the worker processes due to a combination of behaviors: * The use of `assertReachesEventually` did not ensure that workers actually woken.release() because the SyncManager's Semaphore does not implement get_value. * This mean that the test could finish and the variable "sleeping" would got out of scope and be collected. This unregisters the proxy leading to failures in the worker or possibly the manager. * The subsequent call to `p.join()` during cleanUp therefore never finished. This takes two approaches to fix this: 1) Use woken.acquire() to ensure that the workers actually finish calling woken.release() 2) At the end of the test, wait until the workers are finished, while `cond`, `sleeping`, and `woken` are still valid. (cherry picked from commit c476410) Co-authored-by: Sam Gross <[email protected]>
|
|
seehwan
pushed a commit
to seehwan/cpython
that referenced
this issue
Apr 16, 2025
The test could deadlock trying join on the worker processes due to a combination of behaviors: * The use of `assertReachesEventually` did not ensure that workers actually woken.release() because the SyncManager's Semaphore does not implement get_value. * This mean that the test could finish and the variable "sleeping" would got out of scope and be collected. This unregisters the proxy leading to failures in the worker or possibly the manager. * The subsequent call to `p.join()` during cleanUp therefore never finished. This takes two approaches to fix this: 1) Use woken.acquire() to ensure that the workers actually finish calling woken.release() 2) At the end of the test, wait until the workers are finished, while `cond`, `sleeping`, and `woken` are still valid.
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
This frame comes from
_TestCondition.AMD64 RHEL8 3.x:
test_multiprocessing_spawn.test_managerpassed when re-run.build: https://buildbot.python.org/all/#/builders/185/builds/5160
Linked PRs
The text was updated successfully, but these errors were encountered: