Feature Request: Venice AI Provider Integration

[actuallyrizzn]

Feature Request: Venice AI Provider Integration

Summary

Add Venice AI as a new LLM provider option in Letta, enabling users to use Venice AI models for chat completions, streaming, tool calling, and embeddings generation.

Motivation

Venice AI provides access to multiple LLM models through a unified API. Integrating Venice AI as a provider would give Letta users:

• Access to a diverse set of models through a single provider
• OpenAI-compatible API format for seamless integration
• Dynamic model discovery (models listed from API, not hardcoded)
• Support for chat completions, streaming, tool calling, and embeddings

Proposed Solution

Implement a Venice AI provider that:

1. Extends LLMClientBase following Letta's existing provider patterns
2. Supports all core LLM operations (chat, streaming, embeddings, tool calling)
3. Dynamically lists available models from Venice API
4. Handles errors and retries appropriately
5. Maintains 100% test coverage

Benefits

• More Model Options: Users gain access to Venice AI's model catalog
• Unified Interface: Works seamlessly with Letta's existing agent and LLM infrastructure
• OpenAI-Compatible: Venice uses OpenAI-compatible format, making integration straightforward
• Dynamic Discovery: Models are automatically discovered from API (no hardcoded lists)
• Full Feature Support: Chat, streaming, tool calling, and embeddings all supported

Implementation Details

The implementation would include:

• VeniceClient: LLM client extending LLMClientBase
• VeniceProvider: Provider class for dynamic model listing
• Configuration via VENICE_API_KEY environment variable or settings
• Comprehensive test coverage (unit, integration, E2E)
• Full documentation

Configuration

Users would configure Venice AI by setting:

export VENICE_API_KEY="your-api-key"

Or in settings:

venice_api_key = "your-api-key"

Models would be automatically discovered and available as venice/{model_id}.

Additional Context

Venice AI provides an OpenAI-compatible API, which means:

• Request/response formats are already compatible with Letta's OpenAI client patterns
• Streaming uses Server-Sent Events (SSE) format
• Tool calling follows OpenAI function calling format
• Embeddings use OpenAI-compatible endpoint

This makes the integration straightforward and maintainable.

[actuallyrizzn]

This feature has been implemented in PR #3079. The PR includes:

✅ Complete Venice AI provider integration
✅ 100% test coverage (97 unit tests, 6 live API tests)
✅ Full feature support (chat, streaming, tool calling, embeddings)
✅ Dynamic model discovery from Venice API
✅ Comprehensive error handling with retry logic
✅ Performance verified (< 2s response times)

You can review the implementation at: #3079