[Bugfix] RoBERTa position_id accumulation in CUDA graph padding region

[yanghui1-arch]

Fix a crash that affected all RoBERTa-based embedding models (BAAI/bge-m3, XLM-RoBERTa, stsb-roberta, bge-reranker-v2-m3) when CUDA graphs are enabled. After approximately max_position_embeddings / 2 requests the server crashes with: Assertion 'index out of bounds: 0 <= tmp25 < 8194' failed.

• gpu_model_runner keeps a persistent GPU buffer self.positions that is reused across every request. Each request refreshes only the first num_scheduled_tokens entries via copy_to_gpu; the remaining padding slots [num_scheduled_tokens : num_input_tokens_padded] are not reset.
• RoBERTa models call replace_roberta_positions outside the CUDA graph (before BertModel.forward), which does an in-place position_ids += padding_idx + 1 on the full padded tensor, including the stale padding slots. Because those slots are never reset by copy_to_gpu, each request adds another +(padding_idx + 1) to them.

For BAAI/bge-m3 (max_position_embeddings = 8194, padding_idx = 1, offset = 2) with short sentences (6 tokens) padded to 8:

padding slot value after K requests  =  V_init + 2K
overflow when                         >= 8194
K                                     = (8194 - V_init) / 2  ≈  3999

Fixes #37868

Related #37648 #37868

Purpose

Test Plan

Reproduce the bug

CUDA_LAUNCH_BLOCKING=1 vllm serve BAAI/bge-m3 --port 9001 \
  --hf-overrides '{"architectures": ["BgeM3EmbeddingModel"]}' \
  --runner pooling

# Send 10000 sequential requests
for i in $(seq 1 10000); do
  response=$(curl -s -o /dev/null -w "%{http_code}" http://localhost:9001/pooling \
    -H "Content-Type: application/json" \
    -d '{"model": "BAAI/bge-m3", "task": "embed",
         "input": ["test sentence number '$i'"]}')
  if [ "$response" != "200" ]; then
    echo "FAILED at index $i (HTTP $response)"; break
  fi
done

Existing tests

pytest tests/models/language/pooling/test_bge_m3.py -v -s
pytest tests/models/language/pooling/test_embedding.py -v -s -k "stsb-roberta"
pytest tests/models/language/pooling/test_multi_vector_retrieval.py -v -s
pytest tests/models/language/pooling/test_scoring.py -v -s

---

Test Result

Tested on RTX 5090 (Blackwell, sm_120), vLLM 0.18.1rc1.dev38+ga16133a0f:

	Before fix	After fix
10000 sequential BGE-M3 /pooling requests	Crash at index ~3999	All pass
test_bge_m3.py (dense / sparse / ColBERT scores)	Pass	Pass
test_embedding.py[stsb-roberta-base-v2]	Pass	Pass
test_multi_vector_retrieval.py	Pass	Pass
test_scoring.py (bge-reranker-v2-m3)	Pass	Pass

---

Essential Elements of an Effective PR Description Checklist

• The purpose of the PR, such as "Fix some issue (link existing issues this PR will resolve)".
• The test plan, such as providing test command.
• The test results, such as pasting the results comparison before and after, or e2e results
• (Optional) The necessary documentation update, such as updating supported_models.md and examples for a new model.
• (Optional) Release notes update. If your change is user facing, please update the release notes draft in the Google Doc.

[gemini-code-assist]

Code Review

The pull request fixes a critical bug causing crashes in RoBERTa-based models when CUDA graphs are enabled. The root cause is an accumulation of values in the padding region of the positions buffer across requests, leading to an out-of-bounds access. The fix correctly addresses this by explicitly zeroing out the padding region of the buffer for each request. This change is logical, well-targeted, and effectively prevents the bug as described. The provided context and test plan are thorough, confirming the correctness of the solution.

[yanghui1-arch]

cc @noooop

[noooop]

cc @Isotr0py

[Isotr0py]

Thanks! Look reasonable to me.

[Isotr0py]

But seems #37884 would be a better fix which doesn't touch model runner?

[yanghui1-arch]

But seems #37884 would be a better fix which doesn't touch model runner?

#37873 (This PR) protects against this class of bug for models which use positions += offset inner. #37884 can't solve the bug which will be caused by future models which use positions += offset. It's possible that the same issue will appear again in the future due to new models.