Skip to content

test_multiprocessing_spawn.test_manager: test_notify_n deadlock #130954

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Closed
colesbury opened this issue Mar 7, 2025 · 0 comments
Closed

test_multiprocessing_spawn.test_manager: test_notify_n deadlock #130954

colesbury opened this issue Mar 7, 2025 · 0 comments
Labels
tests Tests in the Lib/test dir topic-multiprocessing type-bug An unexpected behavior, bug, or error

Comments

@colesbury
Copy link
Contributor

colesbury commented Mar 7, 2025 •
edited by bedevere-app bot
Loading

Bug report

Seen in https://buildbot.python.org/all/#/builders/1606/builds/220/steps/6/logs/stdio.

This looks like the same type of failures as we had with test_notify_all. It would have been nice to fix it in #130933 together with test_notify_all, but in my limited local testing of test_notify_n, I didn't see any failures.

test_notify_n (test.test_multiprocessing_spawn.test_manager.WithManagerTestCondition.test_notify_n) ... Process Process-48:
Traceback (most recent call last):
  File "/home/buildbot/buildarea/pull_request.itamaro-centos-aws.refleak.nogil/build/Lib/multiprocessing/process.py", line 313, in _bootstrap
    self.run()
    ~~~~~~~~^^
  File "/home/buildbot/buildarea/pull_request.itamaro-centos-aws.refleak.nogil/build/Lib/multiprocessing/process.py", line 108, in run
    self._target(*self._args, **self._kwargs)
    ~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/buildbot/buildarea/pull_request.itamaro-centos-aws.refleak.nogil/build/Lib/test/_test_multiprocessing.py", line 1581, in f
    cond.release()
    ~~~~~~~~~~~~^^
  File "/home/buildbot/buildarea/pull_request.itamaro-centos-aws.refleak.nogil/build/Lib/multiprocessing/managers.py", line 1066, in release
    return self._callmethod('release')
           ~~~~~~~~~~~~~~~~^^^^^^^^^^^
  File "/home/buildbot/buildarea/pull_request.itamaro-centos-aws.refleak.nogil/build/Lib/multiprocessing/managers.py", line 847, in _callmethod
    raise convert_to_error(kind, result)
multiprocessing.managers.RemoteError: 
---------------------------------------------------------------------------
Traceback (most recent call last):
  File "/home/buildbot/buildarea/pull_request.itamaro-centos-aws.refleak.nogil/build/Lib/multiprocessing/managers.py", line 263, in serve_client
    self.id_to_local_proxy_obj[ident]
    ~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^
KeyError: '200081100c0'

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/home/buildbot/buildarea/pull_request.itamaro-centos-aws.refleak.nogil/build/Lib/multiprocessing/managers.py", line 265, in serve_client
    raise ke
  File "/home/buildbot/buildarea/pull_request.itamaro-centos-aws.refleak.nogil/build/Lib/multiprocessing/managers.py", line 259, in serve_client
    obj, exposed, gettypeid = id_to_obj[ident]
                              ~~~~~~~~~^^^^^^^
KeyError: '200081100c0'
---------------------------------------------------------------------------
Timeout (0:45:00)!
Thread 0x00007fce16a26740 (most recent call first):

Linked PRs
@colesbury colesbury added tests Tests in the Lib/test dir topic-multiprocessing type-bug An unexpected behavior, bug, or error labels Mar 7, 2025
colesbury added a commit to colesbury/cpython that referenced this issue Mar 7, 2025
@colesbury
pythongh-130954: Fix multiprocessing test_notify_n
1864a7f 
The test could deadlock trying join on the worker processes.
Apply the same technique as pythongh-130933.

Join the process before the test ends in `test_notify` as well.
@bedevere-app bedevere-app bot mentioned this issue Mar 7, 2025
Merged
@colesbury colesbury mentioned this issue Mar 7, 2025
Closed
colesbury added a commit that referenced this issue Mar 8, 2025
@colesbury
gh-130954: Fix multiprocessing test_notify_n (#130955)
edd1eca 
The test could deadlock trying join on the worker processes.
Apply the same technique as gh-130933.

Join the process before the test ends in `test_notify` as well.
miss-islington pushed a commit to miss-islington/cpython that referenced this issue Mar 8, 2025
@colesbury @miss-islington
pythongh-130954: Fix multiprocessing test_notify_n (pythonGH-130955)
f5f6b01 
The test could deadlock trying join on the worker processes.
Apply the same technique as pythongh-130933.

Join the process before the test ends in `test_notify` as well.
(cherry picked from commit edd1eca)

Co-authored-by: Sam Gross <[email protected]>
miss-islington pushed a commit to miss-islington/cpython that referenced this issue Mar 8, 2025
@colesbury @miss-islington
pythongh-130954: Fix multiprocessing test_notify_n (pythonGH-130955)
ae3573f 
The test could deadlock trying join on the worker processes.
Apply the same technique as pythongh-130933.

Join the process before the test ends in `test_notify` as well.
(cherry picked from commit edd1eca)

Co-authored-by: Sam Gross <[email protected]>
This was referenced Mar 8, 2025
Merged
Merged
colesbury added a commit that referenced this issue Mar 8, 2025
@miss-islington @colesbury
[3.12] gh-130954: Fix multiprocessing test_notify_n (GH-130955) (#130982
4eb4785 
)

The test could deadlock trying join on the worker processes.
Apply the same technique as gh-130933.

Join the process before the test ends in `test_notify` as well.
(cherry picked from commit edd1eca)

Co-authored-by: Sam Gross <[email protected]>
colesbury added a commit that referenced this issue Mar 8, 2025
@miss-islington @colesbury
[3.13] gh-130954: Fix multiprocessing test_notify_n (GH-130955) (#130981
fead51d 
)

The test could deadlock trying join on the worker processes.
Apply the same technique as gh-130933.

Join the process before the test ends in `test_notify` as well.
(cherry picked from commit edd1eca)

Co-authored-by: Sam Gross <[email protected]>
seehwan pushed a commit to seehwan/cpython that referenced this issue Apr 16, 2025
@colesbury @seehwan80
pythongh-130954: Fix multiprocessing test_notify_n (python#130955)
3cbf05d 
The test could deadlock trying join on the worker processes.
Apply the same technique as pythongh-130933.

Join the process before the test ends in `test_notify` as well.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
tests Tests in the Lib/test dir topic-multiprocessing type-bug An unexpected behavior, bug, or error
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
1 participant
@colesbury