Skip to content

[Test] Add unit tests for Qwen25Detector function call parser - #26386

Closed
CarrotSwordsman wants to merge 2 commits into
sgl-project:mainfrom
CarrotSwordsman:test/qwen25-detector-unit-tests
Closed

CarrotSwordsman wants to merge 2 commits into
sgl-project:mainfrom
CarrotSwordsman:test/qwen25-detector-unit-tests

Conversation

@CarrotSwordsman

@CarrotSwordsman CarrotSwordsman commented May 26, 2026 •

Copy link
Copy Markdown

Motivation

Part of #20865 — improving unit test coverage for function call detectors.

Changes

Add unit tests for Qwen25Detector covering:

  • has_tool_call detection (true/false/partial tag)
  • detect_and_parse (single call, multiple calls, preceding text, no calls, malformed JSON, extra whitespace)
  • Streaming incremental parsing

Notes

  • No server launch or model loading required
  • Follows existing test_hermes_detector.py conventions
  • Uses CustomTestCase and registers with CPU CI

CI States

Latest PR Test (Base): ❌ Run #26734080966
Latest PR Test (Extra): ❌ Run #26734080886

@gemini-code-assist gemini-code-assist Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code Review

This pull request introduces unit tests for the Qwen25Detector class, covering tool call detection, parsing, and streaming scenarios. A critical issue was identified in the streaming test test_streaming_single_call, where asserting on the final chunk's result will fail because parse_streaming_increment processes chunks incrementally and does not return the fully assembled tool call in the final closing tag chunk. A suggestion has been provided to accumulate the incremental calls and verify the combined result.

Comment on lines +129 to +136
result = None
for chunk in chunks:
result = self.detector.parse_streaming_increment(chunk, self.tools)

# After the closing tag, we should have a parsed call
self.assertIsNotNone(result)
self.assertEqual(len(result.calls), 1)
self.assertEqual(result.calls[0].name, "get_weather")

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

medium

In streaming mode, parse_streaming_increment processes chunks incrementally and returns partial updates (e.g., first the tool name, then the argument diffs). It does not return the fully assembled tool call in the final chunk (which is just the closing tag \n</tool_call>). Therefore, asserting on the last chunk's result will fail because it is empty. Instead, you should accumulate the calls returned by each chunk and verify the combined result.

Suggested change
result = None
for chunk in chunks:
result = self.detector.parse_streaming_increment(chunk, self.tools)
# After the closing tag, we should have a parsed call
self.assertIsNotNone(result)
self.assertEqual(len(result.calls), 1)
self.assertEqual(result.calls[0].name, "get_weather")
calls = []
for chunk in chunks:
res = self.detector.parse_streaming_increment(chunk, self.tools)
if res and res.calls:
calls.extend(res.calls)
self.assertTrue(len(calls) >= 2)
self.assertEqual(calls[0].name, "get_weather")
full_params = "".join(c.parameters for c in calls)
self.assertEqual(json.loads(full_params), {"city": "Beijing"})

Add comprehensive unit tests for Qwen25Detector covering:
- has_tool_call detection (true/false/partial tag)
- detect_and_parse (single/multiple calls, preceding text, no calls,
  malformed JSON, extra whitespace)
- Streaming incremental parsing

Part of sgl-project#20865

Signed-off-by: mershi <mershi@tencent.com>
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
@CarrotSwordsman
CarrotSwordsman force-pushed the test/qwen25-detector-unit-tests branch from 9dcc094 to e8adb48 Compare May 26, 2026 10:46
Address gemini-code-assist review: parse_streaming_increment emits
partial updates (name first, then argument increments), so asserting
len(all_calls) == 1 is incorrect. Instead, accumulate calls across all
chunks and verify:
  - the tool name appears in some partial call
  - concatenating all parameter increments yields the full JSON

Signed-off-by: shiyichuan <93317314+CarrotSwordsman@users.noreply.github.com>
@CarrotSwordsman

Copy link
Copy Markdown
Author

Thanks for the feedback @gemini-code-assist[bot] — applied in 3251ec1.

The streaming assertion now accumulates across all chunks and verifies:

  • the tool name appears in some partial call (not necessarily the first/last),
  • concatenating all parameters increments yields the expected JSON.

This matches the actual contract of parse_streaming_increment (incremental partial updates rather than a single final call). Ready for another look.

@gemini-code-assist

Copy link
Copy Markdown
Contributor

Warning

You have reached your daily quota limit. Please wait up to 24 hours and I will start processing your requests again!

@CarrotSwordsman

Copy link
Copy Markdown
Author

FYI — #27158 (opened 2026-06-03) adds the same test file at the same path, so there's an unintentional overlap. Pinging maintainers (cc @whybeyoung @CatherineSue) to dedupe — happy to fold any unique coverage from #27158 in here, or to close this PR if the other one is preferred. This PR was opened 2026-05-27 and addresses #20865.

Separately, the 9 "fail" CI runs above are all the pr-gate 'Missing required label run-ci' gating (same as #26400/#26401), not real test failures — could a maintainer add the run-ci label so actual CI runs?

@CarrotSwordsman

Copy link
Copy Markdown
Author

Gentle ping #2 — this PR is in CI-gated state (run-ci label needed). #27158 was closed by @inin-zou in favor of this one (thanks again 🙏). Could a maintainer add the label so CI can run? cc @whybeyoung @CatherineSue @zhyncs

@CarrotSwordsman

Copy link
Copy Markdown
Author

Gentle ping #3. This remains CI-gated (run-ci label needed); #27158 was closed in favor of this PR a while back. Would appreciate the label whenever convenient so the suite can run. cc @whybeyoung @CatherineSue @zhyncs

@hnyls2002

Copy link
Copy Markdown
Collaborator

Closing this along with the other open unit-test PRs referenced from #20865.

In the AI era, adding unit tests for coverage's sake no longer provides enough value to justify the cost — each PR in this batch consumes reviewer bandwidth and CI time out of proportion to the coverage it adds. The tracking issue itself is already closed, and we are no longer accepting coverage-driven UT PRs from it.

Thanks for the effort — test contributions tied to a concrete bug fix or behavior change are still welcome.

@hnyls2002 hnyls2002 closed this Sep 18, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants