listener code execution aborts with potential db connection error #435
Description
What I am doing ?
Implemented a listener using plugins for failed tasks. There is some code there to do some monitoring related actions.
I have added debug log lines to figure out the abrupt end to the execution.
It appears that after some time when the internal db connection issue is detected the task just terminates the execution of the listener.
Following is the error line that shows up consistently when this issue occurs. After this line it is pretty much end of the task/worker log.
[2025-05-14, 20:22:04 UTC] {{standard_task_runner.py:100}} ERROR - Failed to execute job 20480 for task failure_task (This transaction is closed; 6582)
PS: I have not yet tried to reproduce in cloud.
What i have tried ?
- I tried allocating more resources to the docker container that runs postgres
- I tried to tweak sql alchemy configs from the airflow.cfg file. There were multiple config items that i changed.