[qgallouedec]

What does this PR do?

Fixes the secondary problem reported in #48298 (silent timeouts after the background thread dies). Part of #48294.

Change: add_request raises immediately when fatal_error is set, and get_result raises once the remaining results are drained, both chaining from self.fatal_error so the root cause is in the traceback. get_result's blocking wait is chunked into 1s slices so a thread dying mid-wait is noticed within a second rather than after the caller's full timeout (the death usually happens after the caller has already started waiting).

Before/after, submitting to a manager whose background thread died (here: a model unsupported by CB).

Before: add_request accepts the requests and get_result returns None until the caller's timeout. Nothing surfaces the error, it is only in the logs:

ERROR - Error in generation loop: Invalid group type: linear_attention
# ... then, 120s later, whatever timeout handling the caller wrote:
TimeoutError: Only 0/8 eval results after 120s

After: the caller gets the error 1.2s after startup, with the root cause chained:

ValueError: Invalid group type: linear_attention

The above exception was the direct cause of the following exception:
RuntimeError: The continuous batching background thread died with a fatal error.

Who can review?

@remi-or (continuous batching)

[HuggingFaceDocBuilderDev]

The docs for this PR live here. All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.

[github-actions]

---

CI recap

Dashboard: View test results in Grafana
Latest run: 32874867852:1
Result: failure | Jobs: 2 | Tests: 25 | Failures: 0 | Duration: 2m 41s

[remi-or]

Thanks for raising the issue! It's true the failing is a bit slow, but this touches some pretty core part of the code, so I opened this PR #48334 instead with a bit more fixes. LMK if that does the trick for you.