Skip to content

fix(llms): narrow _is_reasoning_model to not match gpt-5.x variants#4746

Merged
kartik-mem0 merged 3 commits intomem0ai:mainfrom
rudra717:fix/reasoning-model-substring-match
Apr 23, 2026
Merged

fix(llms): narrow _is_reasoning_model to not match gpt-5.x variants#4746
kartik-mem0 merged 3 commits intomem0ai:mainfrom
rudra717:fix/reasoning-model-substring-match

Conversation

@rudra717
Copy link
Copy Markdown
Contributor

@rudra717 rudra717 commented Apr 8, 2026

Linked Issue

Closes #4738

Description

_is_reasoning_model() used a substring match ("gpt-5" in model_lower) that incorrectly classified gpt-5.4-mini and other gpt-5.x variants as reasoning models. This caused temperature and other standard parameters to be silently stripped from API calls.

Root Cause

# Before — matches gpt-5, gpt-5o, BUT ALSO gpt-5.4-mini, gpt-5.4
if any(reasoning_model in model_lower for reasoning_model in ["gpt-5", "o1", "o3"]):
    return True

gpt-5.4-mini contains "gpt-5" as a substring, so it was misclassified. The _get_supported_params() method then stripped all parameters except messages, response_format, tools, and tool_choice — including temperature.

Fix

Replaced the broad substring match with:

  • Exact match against known reasoning models (o1, o3, gpt-5, gpt-5o, etc.)
  • Prefix match for dated variants (o1-2024-12-17, o3-2025-04-16)
  • Provider prefix stripping (openai/o3-minio3-mini)
  • Removed the "gpt-5" substring match entirely

Type of Change

  • Bug fix (non-breaking change that fixes an issue)

Test Coverage

  • I added/updated unit tests

Added 2 tests:

  • test_gpt5_mini_not_classified_as_reasoning — verifies gpt-5.4-mini passes temperature through
  • test_is_reasoning_model_classification — comprehensive classification test covering reasoning models (o1, o3, gpt-5, prefixed variants) and non-reasoning models (gpt-5.4-mini, gpt-5.4, gpt-4.1)

Checklist

  • My code follows the project's style guidelines
  • I have performed a self-review of my code
  • I have added tests that prove my fix works
  • New and existing tests pass locally
  • I have updated documentation if needed

The substring match 'gpt-5 in model_lower' incorrectly classified
gpt-5.4-mini and other gpt-5.x variants as reasoning models, causing
temperature and other standard parameters to be silently stripped.

Replace the broad substring match with:
- Exact match against known reasoning models (o1, o3, gpt-5, gpt-5o)
- Prefix match for dated variants (o1-2024-12-17, o3-2025-04-16)
- Provider prefix stripping (openai/o3-mini -> o3-mini)

gpt-5.x models (gpt-5.4-mini, gpt-5.4) support temperature and are
no longer misclassified.

Fixes mem0ai#4738

Co-Authored-By: Claude <noreply@anthropic.com>
@CLAassistant
Copy link
Copy Markdown

CLAassistant commented Apr 8, 2026

CLA assistant check
All committers have signed the CLA.

kartik-mem0
kartik-mem0 previously approved these changes Apr 13, 2026
@kartik-mem0
Copy link
Copy Markdown
Contributor

please address the ci failing here

@kartik-mem0 kartik-mem0 self-requested a review April 13, 2026 14:09
test_is_reasoning_model_classification created OpenAILLM without
mocking the OpenAI client, causing CI to fail with:
  openai.OpenAIError: The api_key client option must be set

Added the mock_openai_client fixture (same pattern as all other
tests in this file) so the real OpenAI client is never initialized.
Merged upstream main which added test_store_not_sent_by_default,
test_store_sent_when_explicitly_true, and
test_store_sent_when_explicitly_false (from mem0ai#4709 fix).

Kept both upstream store tests and our reasoning model tests.
@rudra717
Copy link
Copy Markdown
Contributor Author

Thanks @kartik-mem0, resolved the merge conflict kept both the upstream store tests from #4709 and the reasoning model classification tests from this PR. Also fixed the CI failure (missing mock_openai_client fixture on the classification test). should be green now

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

bug: _is_reasoning_model() incorrectly strips temperature for gpt-5.4-mini

3 participants