Skip to content

[Rust Frontend] Support chat message names - #17

Draft
richard-for-bugen[bot] wants to merge 1 commit into
mainfrom
richard/parity-impl-message-name
Draft

richard-for-bugen[bot] wants to merge 1 commit into
mainfrom
richard/parity-impl-message-name

Conversation

@richard-for-bugen

Copy link
Copy Markdown

Motivation

  • Python: a chat message's optional name ("an optional name for the participant") reaches the chat template for any role: _parse_chat_message_content (vllm/renderers/chat_utils/chat_message.py) copies message["name"] onto every conversation message it produces, and HF templates read it as message.name. The model-specific encoders (DeepSeek V3.2/V4, Harmony) ignore it.
  • Rust today: the chat DTO parses name on system, user, assistant, and developer messages, then convert_message drops it, because vllm_chat::ChatMessage has no field for it. On tool messages and custom-role messages it is not even parsed. A template that prints participant names, as multi-agent clients rely on, renders a different prompt than Python.
  • This PR: carries name from the request through vllm-chat to HF templates for every role.

A 400 placeholder was not an option here: clients send name routinely, so rejecting it would break working requests.

Ownership and layering

name is a message attribute, so it lives on the vllm-chat message type: every ChatMessage variant gains name: Option<String> (omitted from serialization when None), with with_name and name() helpers. The HTTP DTO keeps owning the wire shape: Tool and custom-role messages now parse name too, and convert_message passes it through. The HF renderer adds name to its TemplateMessage for every role. The model-specific renderers only gain .. in their patterns, since they ignore name in Python too. Most of the diff is that mechanical ...

Validation

Commands from .buildkite/scripts/run-rust-frontend-cargo-ci.sh (style-clippy and test), run from the worktree root:

cargo fmt --manifest-path rust/Cargo.toml --all -- --check   # ok
cargo sort --workspace --check rust                          # ok
cargo deny --manifest-path rust/Cargo.toml --config rust/deny.toml check bans   # ok
cargo clippy --manifest-path rust/Cargo.toml --workspace --all-targets --all-features --locked -- -D warnings   # ok
cargo nextest run --manifest-path rust/Cargo.toml --workspace --all-features --locked --no-fail-fast
# 2184 tests run: 2184 passed, 13 skipped

Tests:

  • chat_template_passes_message_names_for_every_role (vllm-chat): an HF template prints role:name for root, system, developer, user, assistant, and tool messages, with - for a message without a name.
  • prepare_chat_request_keeps_message_names (vllm-server): a chat request JSON with names on every role, including tool, a custom role, and "name": null, lowers to vllm-chat messages with those names.

Both fail when with_name does not store the name.

Python parity item chat.messages[].name (category 2), prioritized by Bugen on 2026-10-20, implemented directly without a placeholder.

🤖 Generated with Claude Code

Python forwards a message's `name` to the chat template for any role
(`_parse_chat_message_content` in `chat_message.py`). The Rust frontend
parsed `name` on most roles but dropped it when lowering to `vllm-chat`,
and ignored it on tool and custom-role messages. Carry an optional
`name` on every `vllm-chat` message variant and expose it to HF
templates; the model-specific encoders ignore it, as in Python.

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
Signed-off-by: richard-for-bugen[bot] <342439978+richard-for-bugen[bot]@users.noreply.github.com>

@BugenZhao BugenZhao left a comment

Copy link
Copy Markdown
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What's the real purpose? When is it introduced?

@richard-for-bugen

Copy link
Copy Markdown
Author

What's the real purpose? When is it introduced?

Purpose. name is OpenAI's optional per-message field: on system, user, assistant, and developer messages it names the participant ("to differentiate between participants of the same role"), and on tool/function results it carries the function name. Python passes it to the chat template for every role; it means nothing to vLLM itself.

When. vllm-project#5649 (2024-09, the OpenAI tools API for Hermes and Mistral) added name to ConversationMessage (documented there as "The name of the function to call") together with the loop that copies it onto every parsed message. That loop is now _parse_chat_message_content in vllm/renderers/chat_utils/chat_message.py; vllm-project#60522 and vllm-project#60972 moved it without changing it.

Who reads it. Only templates that render it, so for most mainstream templates this PR changes nothing. The cases that change are:

  • tool-result templates, such as vLLM's own examples/tool_chat_template_functiongemma.jinja: Function result for {{ message.name | default('function') }}. On Rust today the name is dropped, so the prompt reads "Function result for function" where Python shows the real tool name;
  • templates that put the participant name into the turn header (<role> <name>), which multi-agent clients rely on to tell agents apart.

Position. Keep it as is. The cost is one optional field per vllm-chat message variant; most of the diff is .. in renderer patterns that ignore it, as Python's model-specific encoders do. The narrower alternative, tool messages only, would cover the FunctionGemma case but still render header-name templates differently from Python.

@BugenZhao
BugenZhao marked this pull request as draft October 20, 2026 09:51
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant