Skip to content

bpo-41710: PyThread_acquire_lock_timed() uses sem_clockwait() #28662

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 3 commits into from
Oct 1, 2021
+160 −72
Merged

bpo-41710: PyThread_acquire_lock_timed() uses sem_clockwait() #28662

Show file tree
Hide file tree
Changes from 1 commit
Commits
Show all changes
3 commits
Select commit Hold shift + click to select a range
058de0e
bpo-41710: PyThread_acquire_lock_timed() uses sem_clockwait()
vstinner Sep 30, 2021
ced6f76
Compute the absolute timeout once outside the loop
vstinner Sep 30, 2021
6c4774e
Fix typo
vstinner Oct 1, 2021
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
10 changes: 10 additions & 0 deletions Doc/whatsnew/3.11.rst
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -239,6 +239,16 @@ sqlite3
(Contributed by Aviv Palivoda, Daniel Shahaf, and Erlend E. Aasland in
:issue:`16379`.)

threading
---------

* On Unix, if the ``sem_clockwait()`` function is available in the C library
(glibc 2.30 and newer), the :meth:`threading.Lock.acquire` method now uses
the monotonic clock (:data:`time.CLOCK_MONOTONIC`) for the timeout, rather
than using the system clock (:data:`time.CLOCK_REALTIME`), to not be affected
by system clock changes.
(Contributed by Livius and Victor Stinner in :issue:`41710`.)

time
----

Expand Down
5 changes: 5 additions & 0 deletions Misc/NEWS.d/next/Library/2021-09-30-23-00-18.bpo-41710.svuloZ.rst
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,5 @@
On Unix, if the ``sem_clockwait()`` function is available in the C library
(glibc 2.30 and newer), the :meth:`threading.Lock.acquire` method now uses the
monotonic clock (:data:`time.CLOCK_MONOTONIC`) for the timeout, rather than
using the system clock (:data:`time.CLOCK_REALTIME`), to not be affected by
system clock changes. Patch by Victor Stinner.
29 changes: 25 additions & 4 deletions Python/thread_pthread.h
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -92,7 +92,7 @@
* mutexes and condition variables:
*/
#if (defined(_POSIX_SEMAPHORES) && !defined(HAVE_BROKEN_POSIX_SEMAPHORES) && \
defined(HAVE_SEM_TIMEDWAIT))
(defined(HAVE_SEM_TIMEDWAIT) || defined(HAVE_SEM_CLOCKWAIT)))
# define USE_SEMAPHORES
#else
# undef USE_SEMAPHORES
Expand Down Expand Up @@ -462,16 +462,27 @@ PyThread_acquire_lock_timed(PyThread_type_lock lock, PY_TIMEOUT_T microseconds,
}

_PyTime_t deadline = 0;
if (timeout > 0 && !intr_flag) {
if (timeout > 0
#ifndef HAVE_SEM_CLOCKWAIT
&& !intr_flag
#endif
)
{
deadline = _PyTime_GetMonotonicClock() + timeout;
}

while (1) {
if (timeout > 0) {
#ifdef HAVE_SEM_CLOCKWAIT
struct timespec abs_timeout;
_PyTime_AsTimespec_clamp(deadline, &abs_timeout);
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't know if sem_clockwait() timeout argument can be modified by the function, for example if the wait is interrupted by a signal. In case of doubt, I prefer to compute abs_timeout before each call.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

In the same file, pthread_cond_timedwait() absolute timeout is only computed once outside the loop. I understand that we can also compute abs_timeout once outside the loop.

The new implementation of time.sleep(), pysleep() function, now also uses an absolute timeout (timespec) computed once outside the loop. It uses clock_nanosleep() or nanosleep(). These functions have an argument for the remaining time, it's not used by pysleep().

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

sem_clockwait() documentation doesn't say that the absolute timeout is modified, which confirms that it can be only computed once:
https://www.gnu.org/software/libc/manual/html_node/Waiting-with-Explicit-Clocks.html

status = fix_status(sem_clockwait(thelock, CLOCK_MONOTONIC, &abs_timeout));
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't know if sem_clockwait() can fail with ENOSYS if Python is built with a recent glibc and then run with an older glibc or a different kernel version.

Should we attempt to fallback to sem_timedwait() if sem_clockwait() "doesn't work"? How can we determine if sem_clockwait() "doesn't work"? So far, I failed to find a sem_clockwait() documentation or manual page.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

In case of doubt, I prefer to leave the code as it is, and only implement a fallback once we can test it on a real system.

#else
_PyTime_t t = _PyTime_GetSystemClock() + timeout;
struct timespec ts;
_PyTime_AsTimespec_clamp(t, &ts);
status = fix_status(sem_timedwait(thelock, &ts));
#endif
}
else if (timeout == 0) {
status = fix_status(sem_trywait(thelock));
Expand All @@ -486,6 +497,9 @@ PyThread_acquire_lock_timed(PyThread_type_lock lock, PY_TIMEOUT_T microseconds,
break;
}

// sem_clockwait() uses an absolution timeout, there is no need
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

s/absolution/absolute

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nobody expects the Spanish Absolution!

// to recompute the relative timeout.
#ifndef HAVE_SEM_CLOCKWAIT
if (timeout > 0) {
/* wait interrupted by a signal (EINTR): recompute the timeout */
_PyTime_t timeout = deadline - _PyTime_GetMonotonicClock();
Expand All @@ -494,17 +508,24 @@ PyThread_acquire_lock_timed(PyThread_type_lock lock, PY_TIMEOUT_T microseconds,
break;
}
}
#endif
}

/* Don't check the status if we're stopping because of an interrupt. */
if (!(intr_flag && status == EINTR)) {
if (timeout > 0) {
if (status != ETIMEDOUT)
if (status != ETIMEDOUT) {
#ifdef HAVE_SEM_CLOCKWAIT
CHECK_STATUS("sem_clockwait");
#else
CHECK_STATUS("sem_timedwait");
#endif
}
}
else if (timeout == 0) {
if (status != EAGAIN)
if (status != EAGAIN) {
CHECK_STATUS("sem_trywait");
}
}
else {
CHECK_STATUS("sem_wait");
Expand Down
Loading