gh-111358: Fix timeout behaviour in BaseEventLoop.shutdown_default_executor

[ordinary-jamie]

Fixes the timeout behaviour in asyncio.BaseEventLoop.shutdown_default_executor to ensure that the coroutine correctly times-out when awaiting the shutdown thread.

If the timeout is reaching, the thread is left to run, and is not joined (since waiting for the thread to join in this case would exceed the timeout). The Executor is shutdown without waiting.

Minor updates to _do_shutdown to ensure future isn't cancelled. This is necessary since the thread can still run and attempt to set the future result/exception

• Issue: shutdown_default_executor / wait_for_tstate_lock deadlock (?) after Ctrl+C #111358

[gvanrossum]

@ordinary-jamie Can you look into the test failures? They all look related to shutdown_default_executor().

[gvanrossum]

This can cleanly be backported to 3.12, but not to 3.11 -- that doesn't have the timeout argument.

Either way, while this PR fixes the timeout logic, it doesn't really help the OP of gh-111358 -- it'll still wait 300 seconds for the shutdown upon receiving the first ^C, so it still won't print anything right away, and it'll take three ^C to exit.

But I'll note that this isn't entirely asyncio's fault -- if you use an executor to sleep 1000 seconds and print the result, it'll take two ^C to exit. (Admittedly, if you don't wait for the future returned by submit() it only takes one ^C.) In either case, the final traceback printed is from threading._shutdown() handling atexit calls, trying to quit the idle threads in all executors. Note that this traceback is being logged during shutdown (it says "Exception ignored in ...") -- it is not printed by the regular exception handling if __main__ is interrupted.

Also, I might as well ping @kumaraditya303, to see if he agrees with our analysis. We made the mistake in gh-97561 (which he cleaned up and committed but didn't write) not to add a test. :-(

[ordinary-jamie]

@ordinary-jamie Can you look into the test failures?

Sorry! Didn't notice these yesterday. Took a look at it appears our issue it because the Lib/test/ssltests.py script flags -Werror

cpython/Lib/test/ssltests.py

Lines 15 to 19 in ded5f1f

	def run_regrtests(*extra_args):
	print(ssl.OPENSSL_VERSION)
	args = [
	sys.executable,
	'-Werror', '-bb', # turn warnings into exceptions

Pushed a fix to temporarily fix the warning filter for this case

[gvanrossum]

I see. It seems that in tests, using catch_warnings() requires simplefilter(), otherwise the test fails when using -Werror. Is there any particular reason you're not using assertWarns() though? It takes care of this.

[ordinary-jamie]

No reason (besides me not knowing about that method, sorry). Updated!

[gvanrossum]

Now you know. :-)

Looks good, I'll merge.

[miss-islington-app]

Thanks @ordinary-jamie for the PR, and @gvanrossum for merging it 🌮🎉.. I'm working now to backport this PR to: 3.12.
🐍🍒⛏🤖

[bedevere-app]

GH-115641 is a backport of this pull request to the 3.12 branch.