fix: strip <think> tags from modal processor fallback responses (closes #159)#236
Merged
LarFii merged 1 commit intoHKUDS:mainfrom Apr 7, 2026
Conversation
HKUDS#159) Reasoning models (DeepSeek-R1, Qwen2.5-think, etc.) wrap their chain-of-thought in <think>…</think> blocks before emitting the final answer. When _robust_json_parse fails to extract a valid JSON object from the response, the four modal-processor parse methods (_parse_response, _parse_table_response, _parse_equation_response, _parse_generic_response) were returning the **raw** LLM response as the fallback caption and summary. This caused internal model reasoning to be stored in the knowledge graph instead of the actual content description. Fix: add a static helper `BaseModalProcessor._strip_thinking_tags` that removes <think>/<thinking> blocks (case-insensitive, multiline) and apply it in every fallback branch so only the final-answer text is stored or returned. The helper is tested in tests/test_strip_thinking_tags.py with 13 unit tests covering: tag variants, multiline blocks, multiple blocks, case-insensitivity, and the full fallback path for all four processor classes.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Problem
Reasoning models such as DeepSeek-R1 and Qwen2.5-think prepend their chain-of-thought inside
<think>…</think>blocks before emitting the final answer.When
_robust_json_parsecannot extract a valid JSON object (e.g. because the model emitted only thinking content without a trailing structured answer), the four modal-processor parse methods fell back to returning the raw LLM response string as theenhanced_captionandsummary. This caused internal model reasoning to be stored in the knowledge graph instead of the actual content description — exactly the symptom reported in #159.Affected methods (all shared the same pattern):
ImageModalProcessor._parse_responseTableModalProcessor._parse_table_responseEquationModalProcessor._parse_equation_responseGenericModalProcessor._parse_generic_responseFix
Added a static helper
BaseModalProcessor._strip_thinking_tags(text)that removes<think>…</think>and<thinking>…</thinking>blocks (case-insensitive, handles multiline content) and applies it in theexceptfallback branch of all four methods before the cleaned text is stored or returned.The helper is intentionally a
@staticmethodon the base class so subclasses and future processors can reuse it without extra imports.Tests
tests/test_strip_thinking_tags.py— 13 new unit tests:<think>,<thinking>, uppercase)All 13 pass;
ruff check+ruff format --checkclean.Checklist
rufflint + format checks pass