Repository navigation
[Test] Add unit tests for Qwen25Detector function call parser - #26386
CarrotSwordsman wants to merge 2 commits into
Conversation
There was a problem hiding this comment.
Code Review
This pull request introduces unit tests for the Qwen25Detector class, covering tool call detection, parsing, and streaming scenarios. A critical issue was identified in the streaming test test_streaming_single_call, where asserting on the final chunk's result will fail because parse_streaming_increment processes chunks incrementally and does not return the fully assembled tool call in the final closing tag chunk. A suggestion has been provided to accumulate the incremental calls and verify the combined result.
| result = None | ||
| for chunk in chunks: | ||
| result = self.detector.parse_streaming_increment(chunk, self.tools) | ||
|
|
||
| # After the closing tag, we should have a parsed call | ||
| self.assertIsNotNone(result) | ||
| self.assertEqual(len(result.calls), 1) | ||
| self.assertEqual(result.calls[0].name, "get_weather") |
There was a problem hiding this comment.
In streaming mode, parse_streaming_increment processes chunks incrementally and returns partial updates (e.g., first the tool name, then the argument diffs). It does not return the fully assembled tool call in the final chunk (which is just the closing tag \n</tool_call>). Therefore, asserting on the last chunk's result will fail because it is empty. Instead, you should accumulate the calls returned by each chunk and verify the combined result.
| result = None | |
| for chunk in chunks: | |
| result = self.detector.parse_streaming_increment(chunk, self.tools) | |
| # After the closing tag, we should have a parsed call | |
| self.assertIsNotNone(result) | |
| self.assertEqual(len(result.calls), 1) | |
| self.assertEqual(result.calls[0].name, "get_weather") | |
| calls = [] | |
| for chunk in chunks: | |
| res = self.detector.parse_streaming_increment(chunk, self.tools) | |
| if res and res.calls: | |
| calls.extend(res.calls) | |
| self.assertTrue(len(calls) >= 2) | |
| self.assertEqual(calls[0].name, "get_weather") | |
| full_params = "".join(c.parameters for c in calls) | |
| self.assertEqual(json.loads(full_params), {"city": "Beijing"}) |
Add comprehensive unit tests for Qwen25Detector covering: - has_tool_call detection (true/false/partial tag) - detect_and_parse (single/multiple calls, preceding text, no calls, malformed JSON, extra whitespace) - Streaming incremental parsing Part of sgl-project#20865 Signed-off-by: mershi <mershi@tencent.com> Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
9dcc094 to
e8adb48
Compare
Address gemini-code-assist review: parse_streaming_increment emits partial updates (name first, then argument increments), so asserting len(all_calls) == 1 is incorrect. Instead, accumulate calls across all chunks and verify: - the tool name appears in some partial call - concatenating all parameter increments yields the full JSON Signed-off-by: shiyichuan <93317314+CarrotSwordsman@users.noreply.github.com>
|
Thanks for the feedback @gemini-code-assist[bot] — applied in The streaming assertion now accumulates across all chunks and verifies:
This matches the actual contract of |
|
Warning You have reached your daily quota limit. Please wait up to 24 hours and I will start processing your requests again! |
|
FYI — #27158 (opened 2026-06-03) adds the same test file at the same path, so there's an unintentional overlap. Pinging maintainers (cc @whybeyoung @CatherineSue) to dedupe — happy to fold any unique coverage from #27158 in here, or to close this PR if the other one is preferred. This PR was opened 2026-05-27 and addresses #20865. Separately, the 9 "fail" CI runs above are all the |
|
Gentle ping #2 — this PR is in CI-gated state ( |
|
Gentle ping #3. This remains CI-gated ( |
|
Closing this along with the other open unit-test PRs referenced from #20865. In the AI era, adding unit tests for coverage's sake no longer provides enough value to justify the cost — each PR in this batch consumes reviewer bandwidth and CI time out of proportion to the coverage it adds. The tracking issue itself is already closed, and we are no longer accepting coverage-driven UT PRs from it. Thanks for the effort — test contributions tied to a concrete bug fix or behavior change are still welcome. |
Motivation
Part of #20865 — improving unit test coverage for function call detectors.
Changes
Add unit tests for
Qwen25Detectorcovering:has_tool_calldetection (true/false/partial tag)detect_and_parse(single call, multiple calls, preceding text, no calls, malformed JSON, extra whitespace)Notes
test_hermes_detector.pyconventionsCustomTestCaseand registers with CPU CICI States
Latest PR Test (Base): ❌ Run #26734080966
Latest PR Test (Extra): ❌ Run #26734080886