Repository navigation
[Fix] Reasoning parser: bare </think> from hybrid thinking models leaks into content - #36285
ChenZiHong-Gavin wants to merge 3 commits into
Conversation
…ontent Hybrid thinking models (thinks_internally=True: DeepSeek-V4, Qwen3, GLM-4.5, ...) keep the opening <think> tag in the chat template, so a model that enters thinking on its own emits only reasoning followed by a bare closing tag. The base detector, not being in reasoning state and never seeing an opening tag, routed the entire chain-of-thought plus the dangling </think> into normal content (observed live on DeepSeek-V4-Flash: content = 'thoughts...</think>answer', no reasoning_content). Treat a bare think_end_token as the close of template-opened reasoning: non-streaming splits the text around it; streaming reclassifies what is still buffered, strips the tag, and holds back a possible split tag suffix. Off unless thinks_internally, and disabled as soon as anything contradicts that reading: reasoning was explicitly entered or already closed, force_reasoning is set, or an opening tag has passed through as content (tracked across chunk boundaries), which makes a later bare closer quoted markup. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
|
Hi maintainers, this PR is ready for CI — could someone add the For context: this fixes reasoning leaking into |
|
Reopening — I closed this by mistake; nothing about the behavior on To restate the case: a detector with Worth noting this is not something a caller can reliably paper over after the fact — stripping a bare closer from an assembled response does nothing for streaming consumers, which see the raw reasoning as it arrives, token by token. The streaming path in this PR is what actually covers that case. No tests have run on this yet: every failing check is the gate reporting the missing |
|
I encounter the same problem. We found the "<//think>" leaked into chat and "<//DSML>" leaked into tool call, which leads to the failure of agent use. |
|
We met the same problem as well, please take a look @JustinTong0323 @hnyls2002 |
Motivation
Hybrid thinking models (
thinks_internally=True: DeepSeek-V4, Qwen3, GLM-4.5, ...) keep the opening<think>tag in the chat template, so a model that enters thinking on its own (thinking not enabled for the request) emits only reasoning followed by a bare closing tag:The base detector is not in reasoning state and never sees an opening tag, so it routes the entire chain-of-thought plus the dangling
</think>intocontent, andreasoning_contentstays empty. Observed live on DeepSeek-V4-Flash served with--reasoning-parser deepseek-v4(reproduced 7/7 on complex prompts that trip spontaneous thinking); downstream consumers ofcontentdisplay the raw chain-of-thought to end users.Related earlier reports of the same state-desync family: #17551, #17943, #4711, #15446. This PR addresses the direction where the output lands in
content(the inverse of #17551).Modifications
BaseReasoningFormatDetector: treat a barethink_end_tokenas the close of template-opened reasoning, for hybrid models only.textcontaining a bare closer splits around it — prefix becomesreasoning_content, suffixcontent.finish()if it never completes.thinks_internally(DeepSeek-R1-style force detectors already handle this shape natively);stripped_think_start) or already closed (_saw_think_end), orforce_reasoningis set;The existing chunk-size-invariance tests (
TestStreamingChunkSizeInvariance) pass unchanged: the guards were shaped so quoted-tag and tool-interrupt divergence sets stay exactly as documented.Accuracy Tests
Not applicable (no model forward changes). Parser unit tests:
test/registered/unit/parser/test_reasoning_parser.py— 119 tests pass, including 7 new ones covering: non-streaming split, non-hybrid detectors unchanged, closer-after-real-block stays content, streaming single-chunk split, tag split across chunks, closer after closed reasoning stays content, held-back partial flushes on finish.Speed Tests and Profiling
Not applicable — one extra
findon the not-in-reasoning streaming path for hybrid detectors, plus a rolling tail bounded bylen(think_start_token).Checklist
🤖 Generated with Claude Code
CI States
Latest PR Test (Base): ❌ Run #32930443543
Latest PR Test (Extra): ❌ Run #32930443359
Latest PR Test (AMD ROCm 7.2): ❌ Run #32930443472