[Serve] Fix replicas hanging forever when requests are stuck draining in direct ingress mode

[abrarsheikh]

• In direct ingress mode, replicas waiting for requests to drain were never force-killed, causing them to hang indefinitely if requests got stuck
• Replicas are now force-killed after max(graceful_shutdown_timeout_s, RAY_SERVE_DIRECT_INGRESS_MIN_DRAINING_PERIOD_S)

Before

Scenario	Behavior
Requests drain normally	Wait forever for 30s min drain period
Requests stuck	Hang forever

After

Scenario	Behavior
Requests drain normally	Force-kill after max(timeout, 30s)
Requests stuck	Force-kill after max(timeout, 30s)

Test Plan

• python/ray/serve/tests/unit/test_deployment_state.py
• python/ray/serve/tests/test_direct_ingress.py

^{Cursor Bugbot reviewed your changes and found no issues for commit 6460607}

[gemini-code-assist]

Code Review

This pull request addresses an issue where replicas in direct ingress mode could hang indefinitely if requests were stuck during the draining process. The fix ensures that replicas are now forcefully killed after a timeout, which is calculated as the maximum of the deployment's graceful_shutdown_timeout_s and a new minimum draining period constant for direct ingress. This change moves the timeout enforcement logic to the controller, simplifying the replica's shutdown process. Additionally, the RAY_SERVE_DISABLE_SHUTTING_DOWN_INGRESS_REPLICAS_FORCEFULLY feature flag has been removed, which further simplifies the code and makes the force-kill behavior more robust. The changes are well-structured and improve the overall code quality.

[cursor]

Cursor Bugbot has reviewed your changes and found 1 potential issue.

[harshit-anyscale]

we should also document the behavior (like who will take priority) in the scenario when MIN_DRANING_PERIOD and RAY_SERVE_FORCE_STOP_UNHEALTHY_REPLICAS, both are set by the users.

[abrarsheikh]

we should also document the behavior (like who will take priority) in the scenario when MIN_DRANING_PERIOD and RAY_SERVE_FORCE_STOP_UNHEALTHY_REPLICAS, both are set by the users.

good point, for now I am going to add a comment in code since DI is not public

[eicherseiji]

My understanding of this PR: Makes force_stop unconditional and updates _shutdown_deadline to enforce RAY_SERVE_DIRECT_INGRESS_MIN_DRAINING_PERIOD_S within the check_and_update_replicas's check_stopped function to guarantee replicas are stopped, even while draining.