Repository navigation
[Rust Frontend] Support chat message names - #17
richard-for-bugen[bot] wants to merge 1 commit into
Conversation
Python forwards a message's `name` to the chat template for any role (`_parse_chat_message_content` in `chat_message.py`). The Rust frontend parsed `name` on most roles but dropped it when lowering to `vllm-chat`, and ignored it on tool and custom-role messages. Carry an optional `name` on every `vllm-chat` message variant and expose it to HF templates; the model-specific encoders ignore it, as in Python. Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com> Signed-off-by: richard-for-bugen[bot] <342439978+richard-for-bugen[bot]@users.noreply.github.com>
BugenZhao
left a comment
There was a problem hiding this comment.
What's the real purpose? When is it introduced?
Purpose. When. vllm-project#5649 (2024-09, the OpenAI tools API for Hermes and Mistral) added Who reads it. Only templates that render it, so for most mainstream templates this PR changes nothing. The cases that change are:
Position. Keep it as is. The cost is one optional field per |
Motivation
name("an optional name for the participant") reaches the chat template for any role:_parse_chat_message_content(vllm/renderers/chat_utils/chat_message.py) copiesmessage["name"]onto every conversation message it produces, and HF templates read it asmessage.name. The model-specific encoders (DeepSeek V3.2/V4, Harmony) ignore it.nameon system, user, assistant, and developer messages, thenconvert_messagedrops it, becausevllm_chat::ChatMessagehas no field for it. On tool messages and custom-role messages it is not even parsed. A template that prints participant names, as multi-agent clients rely on, renders a different prompt than Python.namefrom the request throughvllm-chatto HF templates for every role.A 400 placeholder was not an option here: clients send
nameroutinely, so rejecting it would break working requests.Ownership and layering
nameis a message attribute, so it lives on thevllm-chatmessage type: everyChatMessagevariant gainsname: Option<String>(omitted from serialization whenNone), withwith_nameandname()helpers. The HTTP DTO keeps owning the wire shape:Tooland custom-role messages now parsenametoo, andconvert_messagepasses it through. The HF renderer addsnameto itsTemplateMessagefor every role. The model-specific renderers only gain..in their patterns, since they ignorenamein Python too. Most of the diff is that mechanical...Validation
Commands from
.buildkite/scripts/run-rust-frontend-cargo-ci.sh(style-clippyandtest), run from the worktree root:Tests:
chat_template_passes_message_names_for_every_role(vllm-chat): an HF template printsrole:namefor root, system, developer, user, assistant, and tool messages, with-for a message without a name.prepare_chat_request_keeps_message_names(vllm-server): a chat request JSON with names on every role, includingtool, a custom role, and"name": null, lowers tovllm-chatmessages with those names.Both fail when
with_namedoes not store the name.Python parity item
chat.messages[].name(category 2), prioritized by Bugen on 2026-10-20, implemented directly without a placeholder.🤖 Generated with Claude Code