feat(backend): integrate LiteLLM API Backend#564
Merged
Conversation
…ture and hyperparameters
…the project ; with bus
…uration and test cases.
…ment handling;Enhance log Colors
you-n-g
force-pushed
the
litellm_backend
branch
from
a503889 to
3185bb0
Compare
you-n-g
reviewed
Feb 8, 2025
added 2 commits
February 8, 2025 06:32
you-n-g
reviewed
Feb 8, 2025
…n up unused code;add test on this function
you-n-g
reviewed
Feb 8, 2025
Jensen246
pushed a commit
to Jensen246/RD-Agent
that referenced
this pull request
Jun 6, 2025
* File structure for supporting litellm * more litellm support * feat: Add CachedAPIBackend class and dynamic API backend retrieval function * fix: update benchmark folder path and add default values for architecture and hyperparameters * feat: add LiteLLMAPIBackend and DeprecBackend ; changed structure of the project ; with bus * fix : deprec_backend * feat: Add LiteLLMAPIBackend class and related features; update configuration and test cases. * feat: Enhance LiteLLMAPIBackend with encoder support and dynamic argument handling;Enhance log Colors * lint * fix lint... * fix: Lint * fix:make auto-lint * fix:test oai * fix:redundant _abckend.py * fix: Optimize LiteLLMAPIBackend on token counting functiona, and clean up unused code;add test on this function * feat: Add LiteLLMSettings class and update model settings usage * fix: Update LiteLLMSettings environment variable prefix and model configurations * fix : gitignore * test: Consolidate and relocate test files for litellm backend and oai * fix : lint * fix: lint * auto lint * lint * LINT * lint * chore: remove deprecated backend configuration comments * refactor: Remove unused functions and imports from deprec.py and llm_utils.py * refactor: Move md5_hash function from deprec.py to llm_utils.py * chore: Remove extra newline and add missing import in deprec.py * lint * refactor: Move md5_hash function to utils module * lint * lint * lint --------- Co-authored-by: Young <afe.young@gmail.com> Co-authored-by: Yihua Chen <v-yihuachen@microsoft.com>
licong01-cloud
pushed a commit
to licong01-cloud/RD-Agent
that referenced
this pull request
Dec 13, 2025
* File structure for supporting litellm * more litellm support * feat: Add CachedAPIBackend class and dynamic API backend retrieval function * fix: update benchmark folder path and add default values for architecture and hyperparameters * feat: add LiteLLMAPIBackend and DeprecBackend ; changed structure of the project ; with bus * fix : deprec_backend * feat: Add LiteLLMAPIBackend class and related features; update configuration and test cases. * feat: Enhance LiteLLMAPIBackend with encoder support and dynamic argument handling;Enhance log Colors * lint * fix lint... * fix: Lint * fix:make auto-lint * fix:test oai * fix:redundant _abckend.py * fix: Optimize LiteLLMAPIBackend on token counting functiona, and clean up unused code;add test on this function * feat: Add LiteLLMSettings class and update model settings usage * fix: Update LiteLLMSettings environment variable prefix and model configurations * fix : gitignore * test: Consolidate and relocate test files for litellm backend and oai * fix : lint * fix: lint * auto lint * lint * LINT * lint * chore: remove deprecated backend configuration comments * refactor: Remove unused functions and imports from deprec.py and llm_utils.py * refactor: Move md5_hash function from deprec.py to llm_utils.py * chore: Remove extra newline and add missing import in deprec.py * lint * refactor: Move md5_hash function to utils module * lint * lint * lint --------- Co-authored-by: Young <afe.young@gmail.com> Co-authored-by: Yihua Chen <v-yihuachen@microsoft.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Description
Integration of LiteLLM API Backend:
LiteLLMAPIBackendclass implementing theAPIBackendinterface with methods for building chat sessions, creating embeddings, and calculating token counts. (rdagent/oai/backend/litellm_backend.py)LLMSettingsto include configuration options for LiteLLM chat and embedding models. (rdagent/oai/llm_conf.py)LiteLLMAPIBackendandDeprecBackendin the__init__.pyfile of the backend module. (rdagent/oai/backend/__init__.py)Test Cases for LiteLLM Backend:
test/litellm_backend/test_completion.py,test/litellm_backend/test_embedding_and_similarity.py)Motivation and Context
These changes are required to integrate a new LiteLLM API backend, which will provide improved language model support issue #504 . The updates to task loading and benchmarking functionalities, along with the new test cases, ensure that the project remains robust and reliable.
How Has This Been Tested?
Screenshots of Test Results (if appropriate):
test completion:
test embedding and similarity:
Types of changes
📚 Documentation preview 📚: https://RDAgent--564.org.readthedocs.build/en/564/