[sahilsGit]

Purpose

Adds abort() by external (user-supplied) request id to the Rust frontend's
Llm, TextLlm, and ChatLlm facades, mirroring Python AsyncLLM.abort().

This PR introduces InflightRequests, a
refcounted index of external_id → {internal_id → live_guard_count} that Llm
maintains across active streams.

Changes:

• vllm-llm: new inflight.rs module (InflightRequests, RequestGuard);
  Llm::generate() tracks the edge after client.call() succeeds; new
  Llm::abort().
• vllm-text: thin TextLlm::abort() delegating to Llm::abort().
• vllm-chat: thin ChatLlm::abort() delegating to TextLlm::abort().

This PR complements #44382 ([Rust Frontend] Add /abort_requests endpoint) (relates to #44280),
which adds the POST /abort_requests HTTP route. This PR provides the
external→internal resolution layer, further abort_request related changes would be done in #44382.

Test Plan

cargo fmt -p vllm-llm -p vllm-chat -p vllm-text --check
cargo clippy -p vllm-llm -p vllm-chat -p vllm-text --all-targets -- -D warnings
cargo test -p vllm-llm                          # unit tests + integration tests
cargo test -p vllm-llm inflight                 # inflight module unit tests only
cargo test -p vllm-llm abort                    # abort integration tests only

New tests in rust/src/llm/src/inflight.rs (unit):

• resolves_external_to_internal — basic resolve + unknown id is empty
• one_external_maps_to_many_internal — duplicate external ids each get their own internal id
• dropping_guard_removes_only_its_own_edge_then_cleans_empty_key — per-edge removal + map compaction
• identical_edges_are_refcounted_across_guards — two guards on the same (external, internal) pair; first drop keeps the edge live for the second
• guard_drop_is_a_noop_after_inflight_is_gone — Weak upgrade fails gracefully

New tests in rust/src/llm/tests/generate.rs (integration, against mock engine):

• abort_resolves_external_request_id_to_internal_before_reaching_engine — abort frame carries the internal id, not the external one
• abort_by_external_id_aborts_all_internal_requests — single external id mapped to two internal ids; both appear in the abort frame

Test Result

cargo fmt -p vllm-llm -p vllm-chat -p vllm-text --check
  → clean

cargo clippy -p vllm-llm -p vllm-chat -p vllm-text --all-targets -- -D warnings
  → Finished, 0 warnings

cargo test -p vllm-llm
  running 10 tests
  test inflight::tests::one_external_maps_to_many_internal ... ok
  test inflight::tests::resolves_external_to_internal ... ok
  test inflight::tests::guard_drop_is_a_noop_after_inflight_is_gone ... ok
  test inflight::tests::dropping_guard_removes_only_its_own_edge_then_cleans_empty_key ... ok
  test inflight::tests::identical_edges_are_refcounted_across_guards ... ok
  test request::tests::prepare_builds_engine_core_request ... ok
  test request::tests::prepare_forwards_multimodal_features ... ok
  test request::tests::prepare_rejects_empty_prompt_tokens ... ok
  test request::tests::prepare_can_preserve_external_request_id ... ok
  test request_metrics::tests::tracker_updates_timing_state_across_prefill_decode_and_finish ... ok
  test result: ok. 10 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out

  running 9 tests
  test collect_output_aggregates_raw_tokens_logprobs_and_terminal_metadata ... ok
  test generate_propagates_unexpected_close_errors ... ok
  test abort_resolves_external_request_id_to_internal_before_reaching_engine ... ok
  test abort_by_external_id_aborts_all_internal_requests ... ok
  test dropping_a_live_generate_stream_triggers_abort ... ok
  test duplicate_external_request_ids_are_randomized_before_reaching_engine_core_client ... ok
  test generate_streams_outputs ... ok
  test dropping_stream_records_abort_terminal_request_metrics ... ok
  test generate_records_request_metrics_in_prometheus_output ... ok
  test result: ok. 9 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out

---

Essential Elements of an Effective PR Description Checklist

• The purpose of the PR, such as "Fix some issue (link existing issues this PR will resolve)".
• The test plan, such as providing test command.
• The test results, such as pasting the results comparison before and after, or e2e results
• (Optional) The necessary documentation update, such as updating supported_models.md and examples for a new model. Not needed — Rust-frontend API wiring with no model/docs surface.

[tahsintunan]

The map and resolution logic look right, but I think this covers only half of Python's abort contract. AsyncLLM.abort() also finishes the aborted stream locally. output_processor.abort_requests puts a final output with FinishReason.ABORT on the queue. That matters because the engine sends nothing back for client-initiated aborts, so after client.abort, the request's GenerateOutputStream never yields again, and a caller awaiting it hangs until disconnect.

Python finishes it promptly with finish_reason "abort". Here, with #44382's /abort_requests, the endpoint returns 200, and the engine stops, but the original request never ends.

Fixing this also makes abort idempotent for free: a final output marks the stream terminated, the guard drops, and the stream's Drop skips the extra auto-abort frame it currently sends. Just dropping the registry senders isn't enough since a close without a final output surfaces as Err(RequestStreamClosed) rather than a clean finish, so the abort path probably needs to build that final abort output itself and push it down the request's stream as Python does.

If you'd rather land the map first and do the finalization as a follow-up, that works too, but then the "makes #44382 correct end-to-end" claim in the description should be softened until that lands.

[sahilsGit]

Appreciate the in-depth review, yes, my bad thought AI could at least do one thing good was PR description, I was wrong.

Your concerns are ligit and for the same reason the abort_requests pr is still in draft stage, I will try to address them there.

[BugenZhao]

That matters because the engine sends nothing back for client-initiated aborts, so after client.abort, the request's GenerateOutputStream never yields again, and a caller awaiting it hangs until disconnect.

This might not be the case. It looks like the engine will eventually put the aborted requests inside the finished_requests field in an output batch, so we'll still drop the sender and caller should get Err(RequestStreamClosed) instead of hanging forever.

However this is still a good catch. I guess the easiest way to fix this could be like calling ClientInner.finish_requests, collecting the senders, and manually sending Abort outputs before dropping them.

If the changes are minimal it would be better to include it directly within this PR :)

[tahsintunan]

This might not be the case. It looks like the engine will eventually put the aborted requests inside the finished_requests field in an output batch, so we'll still drop the sender and caller should get Err(RequestStreamClosed) instead of hanging forever.

I think that only kicks in with internal DP balancing on. In the default single-engine or external-LB setup, the batch goes out with finished_requests unset, the sender never drops, and a caller awaiting the stream just hangs till disconnect.

I guess the easiest way to fix this could be like calling ClientInner.finish_requests, collecting the senders, and manually sending Abort outputs before dropping them.

Agreed, and it should be a small addition, so good to fold into this PR. One nit: the synthesized Abort would also run through observe_output, so a request aborted before its first token logs a stray TTFT sample. Edge case, though.

Alternatively, we could synthesize the abort terminal one layer up in the frontend output layer, where Python does it, rather than faking an engine output deeper down, but that's a bigger change that would widen this PR's scope.

[tahsintunan]

Not for this PR, but worth a TODO note here: Python's abort_requests also expands parent request ids into their child requests for parallel sampling (n > 1). The Rust facade is n = 1 only today, so nothing to do now, but when n > 1 lands, resolution here will need that parent-to-children expansion too.

[sahilsGit]

Right

[BugenZhao]

Rest LGTM. Thanks!

[BugenZhao]

That matters because the engine sends nothing back for client-initiated aborts, so after client.abort, the request's GenerateOutputStream never yields again, and a caller awaiting it hangs until disconnect.

This might not be the case. It looks like the engine will eventually put the aborted requests inside the finished_requests field in an output batch, so we'll still drop the sender and caller should get Err(RequestStreamClosed) instead of hanging forever.

However this is still a good catch. I guess the easiest way to fix this could be like calling ClientInner.finish_requests, collecting the senders, and manually sending Abort outputs before dropping them.

If the changes are minimal it would be better to include it directly within this PR :)

[sahilsGit]

Cool, I will make the change

[mergify]

This pull request has merge conflicts that must be resolved before it can be
merged. Please rebase the PR, @sahilsGit.

https://docs.github.com/en/pull-requests/collaborating-with-pull-requests/working-with-forks/syncing-a-fork

[BugenZhao]

LGTM. Thanks for the work!

Let's now work on the user-facing follow-up PR.