[drakosha]

Purpose

FIX #49248

Under concurrent load, GLM-4.7-family models sporadically skip the opening <arg_value> tag when emitting tool calls, producing

<tool_call>search<arg_key>query</arg_key>how many vacation days left</arg_value></tool_call>

with the argument value fully intact (raw sampled tokens captured via logprobs: true; reproduces at temperature: 0 too, ~5–17% of calls at concurrency 6 on our production trace family, 0 sequentially).

_ARG_RE/_PARTIAL_ARG_RE in vllm/parser/glm47_moe.py require the opening tag, so such calls silently parse to arguments: "{}", which then fails the client's tool schema validation and degrades agent loops (the broken {} call gets rendered back into history, so retries tend to fail the same way).

This PR makes the opening <arg_value> optional in both regexes. The closing </arg_value> still delimits the value, so well-formed output parses as before.

Since #61253 an open value is streamed verbatim instead of being re-parsed per chunk, so an optional tag alone would count the value as open right after </arg_key> and send the tag itself into the arguments ({"city": "<arg_value>Saint Petersburg). The second commit keeps the value closed while its text can still turn out to be <arg_value>.

Not a duplicate

gh pr list --search "49248 in:body" finds only this PR. Other open glm47 parser PRs fix different malformations: a duplicated <arg_key> tag (#54971), a missing opening <arg_key> (#47190), an unclosed final </arg_value> (#58418), a stray <tool_call> (#60842). #61253 and #64535 (merged) change how arguments are streamed and do not handle the missing tag.

Test Plan

• tests/tool_parsers/test_glm47_moe_tool_parser.py: non-streaming cases (single arg with missing opening tag; mixed malformed + well-formed args in one call), a streaming case with the malformed sequence split across chunks, and test_open_tag_stays_out_of_arguments (tag in its own chunk with two args, tag split across chunks, a value that starts like the tag but is not).
• Live validation of the original fix on a 4×GPU GLM-5.2 deployment (--tool-call-parser glm47), replaying the production request that triggers the malformation at concurrency 6 with logprobs: true:
  • before: 12/120 responses parsed to arguments: "{}";
  • after: 12/72 responses still show the raw malformation in sampled tokens (model-side quirk, expected), all 12 parse to full arguments, 0 empty; streaming replay 0/72 empty.
• Live validation of the streaming fix on GLM-5.3-Flash, 2×H200, streaming tool call repeated 4 times: without the second commit 0/4 valid JSON, with it 4/4.

Test Result

pytest tests/tool_parsers/test_glm47_moe_tool_parser.py on nightly dd4e40142a: 19 passed with the second commit; without it test_with_args and two of the new cases fail. Run with the GLM-5.3-Flash tokenizer because the host has no zai-org/GLM-4.7 copy.

AI assistance (Claude Code) was used for this PR. I reviewed the change and ran the tests above.

🤖 Generated with Claude Code

[claude]

Claude Code Review

This pull request is from a fork — automated review is disabled. A repository maintainer can comment @claude review to run a one-time review.

[mergify]

This pull request has merge conflicts that must be resolved before it can be
merged. Please rebase the PR, @drakosha.

https://docs.github.com/en/pull-requests/collaborating-with-pull-requests/working-with-forks/syncing-a-fork