Skip to content

fix(sdk): store assistant tool calls under the tool_calls message key - #2330

Open
alemorale7777 wants to merge 1 commit into
EricLBuehler:masterfrom
alemorale7777:fix/sdk-tool-calls-message-key
Open

fix(sdk): store assistant tool calls under the tool_calls message key#2330
alemorale7777 wants to merge 1 commit into
EricLBuehler:masterfrom
alemorale7777:fix/sdk-tool-calls-message-key

Conversation

@alemorale7777

Copy link
Copy Markdown

Problem

RequestBuilder::add_message_with_tool_call (in the mistralrs SDK crate) attaches the assistant's tool calls to the message under the key "function":

self.messages.push(IndexMap::from([
    ("role".to_string(), Either::Left(role.to_string())),
    ("content".to_string(), Either::Left(text.to_string())),
    ("function".to_string(), Either::Right(tool_messages)),
]));

Every consumer of an assistant tool-call message reads the standard OpenAI/HF key "tool_calls", not "function":

  • chat_template.rs renders tool calls from message.get_mut("tool_calls") and clears content based on contains_key("tool_calls").
  • The agentic engine matches on "tool_calls" (agentic_session.rs, agentic_loop.rs).
  • The server path builds the identical structure under "tool_calls" (chat_completion.rs).

Because nothing reads a message-level "function" key, the tool call is silently dropped: the chat template renders the assistant turn without its tool call, and the following tool result then has no preceding call, breaking multi-round tool use.

This is the SDK's recommended API for tool-call messages, and it is used by the SDK's own Agent loop (agent.rs, both the streaming and non-streaming paths) and by the examples/advanced/tools example, so SDK-driven tool conversations are affected.

Fix

Use the "tool_calls" key. The value structure is already correct (each entry has id, type, and a nested function object with name/arguments) and matches what the server path emits, so this is a one-character-key change with no other behavior difference. Added a unit test asserting the emitted message uses tool_calls (not function) and preserves the nested call structure.

RequestBuilder::add_message_with_tool_call attached the tool call vector
to the assistant message under `function`, but every consumer (chat
template, agentic loop, server path) reads `tool_calls`. The call was
silently dropped, so the templated prompt omitted the assistant's tool
call and broke multi-round tool use. The value structure is unchanged.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
@github-actions

github-actions Bot commented Jul 9, 2026

Copy link
Copy Markdown
Code Metrics Report
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━
 Language              Files        Lines         Code     Comments       Blanks
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━
 C Header                 23         4454         3116          790          548
 CSS                       3          282          252            6           24
 CUDA                    119        23681        19217         1704         2760
 Dockerfile                1           35           19            9            7
 HTML                      2           27           27            0            0
 JavaScript                3          577          562           12            3
 Jinja2                    7          694          656            5           33
 JSON                     27        15864        15861            0            3
 Makefile                  1           18           16            0            2
 MDX                      36         5901            0         4327         1574
 Metal Shading Lan|       37        14287        11284         1136         1867
 PowerShell                1          656          570           31           55
 Python                  151        12284        10191          484         1609
 Shell                     3         1062          843          117          102
 Plain Text               53        10687            0         9209         1478
 TOML                     28         1368         1189           39          140
 TypeScript               11         1641         1404           66          171
 YAML                      3           25           23            2            0
─────────────────────────────────────────────────────────────────────────────────
 Jupyter Notebooks         3          122           83           23           16
 |- Markdown               1           60           30           22            8
 |- Python                 1          122          113            1            8
 (Total)                              304          226           46           32
─────────────────────────────────────────────────────────────────────────────────
 Markdown                273        11726            0         8736         2990
 |- BASH                  23          295          217           46           32
 |- Dockerfile             2            5            5            0            0
 |- JSON                   6          289          289            0            0
 |- PowerShell             1            1            1            0            0
 |- Python               135         7239         6021          310          908
 |- Rust                  62         3820         2838          388          594
 |- TOML                   7           77           65            0           12
 (Total)                            23452         9436         9480         4536
─────────────────────────────────────────────────────────────────────────────────
 Rust                    674       299702       267081         5872        26749
 |- Markdown             413         9761          452         8126         1183
 (Total)                           309463       267533        13998        27932
─────────────────────────────────────────────────────────────────────────────────
 Svelte                   19         1969         1826           51           92
 |- CSS                    1            4            4            0            0
 |- JavaScript            19          921          767           25          129
 (Total)                             2894         2597           76          221
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━
 Total                  1478       429656       345022        41537        43097
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant