[Sunt-ing]

Purpose

Follow-up to #44680, which added out-of-vocab token-id validation to the Rust frontend's /v1/completions and /v1/chat/completions routes. The native token-in/token-out route POST /inference/v1/generate feeds the same token-id fields to the engine but was left unvalidated.

A /inference/v1/generate request whose token_ids prompt holds an id >= the vocab size reaches the engine, indexes the embedding table out of bounds, and aborts EngineCore, taking the whole server down (single-request DoS). The Python serving stack rejects this with a 400 before it reaches the engine; the Rust frontend bypasses the Python Processor, so the check is missing.

This extends the shared utils::token_ids validators to the generate route, with the same bounds the OpenAI routes use and the Python input processor follows (max(tokenizer.max_token_id, model_vocab_size - 1)): the token_ids prompt by max(tokenizer, model) vocab (the engine embeds ids beyond either alone, e.g. added LM tokens or multimodal placeholders), allowed_token_ids by the tokenizer vocab, and logit_bias keys by the model vocab. allowed_token_ids reuses the existing validator directly; the generate route's bare Vec<u32> prompt and u32-keyed logit_bias get two thin sibling validators next to the existing ones.

This is complementary to #45286 (which maps lowering-stage PromptTooLong / EmptyPromptTokenIds submit errors to 400): out-of-vocab token ids pass lowering and abort the engine instead, so they have to be rejected before submit, which this PR does.

Test Plan

Build the Rust frontend from this branch and serve any model through it, then send a /inference/v1/generate request with an out-of-vocab token_ids prompt, with and without this change, and check whether the server survives.

Test Result

cargo test -p vllm-server (217 passed), cargo fmt --check, and cargo clippy -p vllm-server are clean.

Differential on the real /inference/v1/generate route (Rust frontend serving Qwen/Qwen3-0.6B, vocab 151936), same source with and without the fix:

OOB token_ids: [9000000]	server afterwards
without fix: HTTP 500 engine core reported fatal failure	DEAD, later requests refused
with fix: HTTP 400 token_ids contains out-of-vocab token id 9000000; vocabulary size is 151936.	healthy, keeps serving

repro (serve command + client + full before/after output)

# build the Rust frontend from this branch
cd rust && cargo build --release --bin vllm-rs

# serve any model through the Rust frontend
VLLM_USE_RUST_FRONTEND=1 \
VLLM_RUST_FRONTEND_PATH=$PWD/target/release/vllm-rs \
vllm serve Qwen/Qwen3-0.6B --port 8744 --enforce-eager

# send an out-of-vocab token-id prompt, then check the server is still up
curl -s -X POST http://127.0.0.1:8744/inference/v1/generate \
  -H 'Content-Type: application/json' \
  -d '{"token_ids":[9000000],"sampling_params":{"max_tokens":4}}'
curl -s -o /dev/null -w '%{http_code}\n' http://127.0.0.1:8744/health

Without this change:

HTTP 500
{"error":{"message":"failed to collect raw generate response: engine core reported fatal failure","type":"server_error","code":"server_error"}}
# server log: EngineCore encountered a fatal error
#   (run_busy_loop -> _process_engine_step -> step_with_batch_queue -> execute_model)
/health -> 000   # connection refused, server is down

With this change:

HTTP 400
{"error":{"message":"token_ids contains out-of-vocab token id 9000000; vocabulary size is 151936.","type":"invalid_request_error","param":"token_ids","code":"invalid_request_error"}}
/health -> 200   # server still serving

The same holds for allowed_token_ids and logit_bias: with the fix they return 400 at the correct bound (tokenizer vocab 151669 and model vocab 151936 respectively). On Qwen/Qwen3-0.6B the engine tolerated those two fields out of range without aborting; they are validated for parity with the Python serving layer, which rejects them regardless.

AI assistance was used to investigate, reproduce, and draft this change; the author reviewed the diff and validation output.

cc @BugenZhao

[BugenZhao]

@codex review

[chatgpt-codex-connector]

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: f7e751a6d7

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

• Open a pull request for review
• Mark a draft as ready
• Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

[chatgpt-codex-connector]

Reject OOB logprob_token_ids before forwarding

Returning here still leaves sampling_params.logprob_token_ids unchecked on the raw generate route. When a /inference/v1/generate client sends an ID >= model_vocab_size, it bypasses this validator and is forwarded to the engine, where gather_specific_token_logprobs indexes the [batch, vocab] logprobs tensor with those IDs; Python's SamplingParams._validate_logprobs rejects the same case before submission. Please bound logprob_token_ids by the model vocab here as well so this route does not keep a separate single-request OOB engine-failure path.

Useful? React with 👍 / 👎.

[Sunt-ing]

Good catch.

logprob_token_ids is the one token-id field I deliberately left out of the initial scope (it sits in the logprob family alongside the max_logprobs discussion), but the out-of-vocab bound is independent of that and belongs here.

Added in a follow-up commit: the generate route forwards sampling_params.logprob_token_ids straight to the engine, which gathers them from the [batch, vocab] logprobs tensor in gather_specific_token_logprobs, so an id >= the model vocab indexes out of bounds and aborts EngineCore; it is now bounded by the model vocab, matching Python's SamplingParams._validate_logprobs. Worth noting it is stronger than the other two fields: where allowed_token_ids/logit_bias happened to be tolerated out of range by the engine on Qwen3-0.6B, an out-of-vocab logprob_token_ids actually aborts the engine on that same model, so this is a confirmed single-request DoS rather than a parity-only check.

repro (serve command + client + full before/after output)

Built the Rust frontend twice from this branch, once without the new commit (stock) and once with it (fixed), and served Qwen/Qwen3-0.6B (vocab 151936) through each. RTX 4090, vLLM editable at this branch's commit, --enforce-eager.

# build vllm-rs from this branch (stock = HEAD~1, fixed = HEAD)
cd rust && cargo build --release --bin vllm-rs

# serve Qwen3-0.6B through the Rust frontend
VLLM_USE_RUST_FRONTEND=1 VLLM_RUST_FRONTEND_PATH=$PWD/target/release/vllm-rs \
  VLLM_USE_FLASHINFER_SAMPLER=0 VLLM_ATTENTION_BACKEND=FLASH_ATTN \
  vllm serve Qwen/Qwen3-0.6B --port 8744 --enforce-eager

# out-of-vocab logprob_token_ids, then check the server is still up
curl -s -X POST http://127.0.0.1:8744/inference/v1/generate \
  -H 'Content-Type: application/json' \
  -d '{"token_ids":[9707],"sampling_params":{"max_tokens":4,"logprob_token_ids":[9000000]}}'
curl -s -o /dev/null -w '%{http_code}\n' http://127.0.0.1:8744/health

Without this commit:

HTTP 500
{"error":{"message":"failed to collect raw generate response: engine-core error: engine core reported fatal failure","type":"server_error","code":"server_error"}}
/health -> 000   # connection refused, server is down

With this commit:

HTTP 400
{"error":{"message":"logprob_token_ids contains out-of-vocab token id 9000000; vocabulary size is 151936.","type":"invalid_request_error","param":"logprob_token_ids","code":"invalid_request_error"}}
/health -> 200   # server still serving

A valid {"token_ids":[9707],"sampling_params":{"max_tokens":4}} returns 200 and generates on both builds. cargo test -p vllm-server (217 passed), cargo fmt --check, and cargo clippy -p vllm-server are clean.

[mergify]

This pull request has merge conflicts that must be resolved before it can be
merged. Please rebase the PR, @Sunt-ing.

https://docs.github.com/en/pull-requests/collaborating-with-pull-requests/working-with-forks/syncing-a-fork

[BugenZhao]

Thanks for the work. After careful consideration, we believe it would be better to move the validation down to the text layer so that all routes, including upcoming ones, are automatically covered. As a result, the changes in this PR are already included in #45685.

[Sunt-ing]

It's OK. Thanks for your insightful thoughts.