Skip to content

HeartbeatManager logs normal shutdown as thread crash #110

Description

@kyle-li-dev

Summary

HeartbeatManager.stop() can log a normal worker shutdown as an error:

HeartbeatManager thread crashed
Traceback (most recent call last):
  File ".../site-packages/litestar_saq/heartbeat.py", line 180, in _run
    self._loop.run_until_complete(self._manager_loop())
  File ".../asyncio/base_events.py", line 723, in run_until_complete
    raise RuntimeError('Event loop stopped before Future completed.')
RuntimeError: Event loop stopped before Future completed.

This happens during process shutdown / Ctrl-C, after the worker is being stopped normally.

Environment

  • litestar-saq: 0.8.0
  • saq: 0.26.4
  • litestar: 2.23.0
  • Python: 3.13.13
  • Platform: macOS arm64

What appears to be happening

HeartbeatManager.stop() sets the stop flag and then stops the event loop from another thread:

self._stop_event.set()
self._loop.call_soon_threadsafe(self._loop.stop)

But the heartbeat thread is running:

self._loop.run_until_complete(self._manager_loop())

If the loop is stopped while _manager_loop() is sleeping / still pending, run_until_complete() raises:

RuntimeError: Event loop stopped before Future completed.

The generic exception handler then logs this expected shutdown path as:

logger.exception("HeartbeatManager thread crashed")

Expected behavior

A normal worker shutdown should not emit an error-level "thread crashed" log.

Possible approaches:

  • Let _manager_loop() wake up and return naturally instead of calling loop.stop() directly.
  • Track/cancel the manager task explicitly and treat cancellation during shutdown as expected.
  • At minimum, suppress or downgrade RuntimeError("Event loop stopped before Future completed") when _stop_event.is_set().

Actual behavior

The worker appears to stop, but shutdown logs an error traceback, which makes normal Ctrl-C / graceful process termination look like a runtime crash.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions