Skip to content

feat(llm): add Google Gemini, AWS Bedrock, io.net, Mistral, Yandex, and Cloudflare WS AI providers #676

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
ilblackdragon merged 2 commits into from
Mar 7, 2026
+176 −14
Merged

feat(llm): add Google Gemini, AWS Bedrock, io.net, Mistral, Yandex, and Cloudflare WS AI providers #676

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
2 commits
Select commit Hold shift + click to select a range
7dbe370
feat(llm): add Google Gemini and AWS Bedrock providers
Mffff4 Mar 7, 2026
b892abf
feat(llm): add io.net, Mistral, Yandex, and Cloudflare WS AI providers
Mffff4 Mar 7, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
10 changes: 7 additions & 3 deletions FEATURE_PARITY.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -215,9 +215,13 @@ This document tracks feature parity between IronClaw (Rust implementation) and O
| NEAR AI | ✅ | ✅ | - | Primary provider |
| Anthropic (Claude) | ✅ | 🚧 | - | Via NEAR AI proxy; Opus 4.5, Sonnet 4, Sonnet 4.6 |
| OpenAI | ✅ | 🚧 | - | Via NEAR AI proxy |
| AWS Bedrock | ✅ | ❌ | P3 | |
| Google Gemini | ✅ | ❌ | P3 | |
| NVIDIA API | ✅ | ❌ | P3 | New provider |
| AWS Bedrock | ✅ | ✅ | P3 | Via `openai_compatible` adapter (e.g. LiteLLM) |
| Google Gemini | ✅ | ✅ | P3 | Via `gemini` adapter |
| io.net | ✅ | ✅ | P3 | Via `ionet` adapter |
| Mistral | ✅ | ✅ | P3 | Via `mistral` adapter |
| Yandex AI Studio | ✅ | ✅ | P3 | Via `yandex` adapter |
| Cloudflare Workers AI | ✅ | ✅ | P3 | Via `cloudflare` adapter |
| NVIDIA API | ✅ | ✅ | P3 | Via `nvidia` adapter and `providers.json` |
| OpenRouter | ✅ | ✅ | - | Via OpenAI-compatible provider (RigAdapter) |
| Tinfoil | ❌ | ✅ | - | Private inference provider (IronClaw-only) |
| OpenAI-compatible | ❌ | ✅ | - | Generic OpenAI-compatible endpoint (RigAdapter) |
Expand Down
6 changes: 6 additions & 0 deletions docs/LLM_PROVIDERS.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -11,6 +11,12 @@ configurations.
| NEAR AI | `nearai` | OAuth (browser) | Default; multi-model |
| Anthropic | `anthropic` | `ANTHROPIC_API_KEY` | Claude models |
| OpenAI | `openai` | `OPENAI_API_KEY` | GPT models |
| Google Gemini | `gemini` | `GEMINI_API_KEY` | Gemini models |
| AWS Bedrock | `bedrock` | `BEDROCK_ACCESS_KEY` | Requires OpenAI proxy (e.g. LiteLLM) |
| io.net | `ionet` | `IONET_API_KEY` | Intelligence API |
| Mistral | `mistral` | `MISTRAL_API_KEY` | Mistral models |
| Yandex AI Studio | `yandex` | `YANDEX_API_KEY` | YandexGPT models |
| Cloudflare Workers AI | `cloudflare` | `CLOUDFLARE_API_KEY` | Access to Workers AI |
| Ollama | `ollama` | No | Local inference |
| OpenRouter | `openai_compatible` | `LLM_API_KEY` | 300+ models |
| Together AI | `openai_compatible` | `LLM_API_KEY` | Fast inference |
Expand Down
172 changes: 161 additions & 11 deletions providers.json
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,7 +1,9 @@
[
{
"id": "openai",
"aliases": ["open_ai"],
"aliases": [
"open_ai"
],
"protocol": "open_ai_completions",
"api_key_env": "OPENAI_API_KEY",
"api_key_required": true,
Expand All @@ -19,7 +21,9 @@
},
{
"id": "anthropic",
"aliases": ["claude"],
"aliases": [
"claude"
],
"protocol": "anthropic",
"api_key_env": "ANTHROPIC_API_KEY",
"api_key_required": true,
Expand Down Expand Up @@ -52,7 +56,10 @@
},
{
"id": "openai_compatible",
"aliases": ["openai-compatible", "compatible"],
"aliases": [
"openai-compatible",
"compatible"
],
"protocol": "open_ai_completions",
"base_url_env": "LLM_BASE_URL",
"base_url_required": true,
Expand Down Expand Up @@ -89,7 +96,9 @@
},
{
"id": "openrouter",
"aliases": ["open_router"],
"aliases": [
"open_router"
],
"protocol": "open_ai_completions",
"default_base_url": "https://openrouter.ai/api/v1",
"api_key_env": "OPENROUTER_API_KEY",
Expand Down Expand Up @@ -126,7 +135,10 @@
},
{
"id": "nvidia",
"aliases": ["nvidia_nim", "nim"],
"aliases": [
"nvidia_nim",
"nim"
],
"protocol": "open_ai_completions",
"default_base_url": "https://integrate.api.nvidia.com/v1",
"api_key_env": "NVIDIA_API_KEY",
Expand All @@ -144,7 +156,10 @@
},
{
"id": "venice",
"aliases": ["venice_ai", "veniceai"],
"aliases": [
"venice_ai",
"veniceai"
],
"protocol": "open_ai_completions",
"default_base_url": "https://api.venice.ai/api/v1",
"api_key_env": "VENICE_API_KEY",
Expand All @@ -162,7 +177,10 @@
},
{
"id": "together",
"aliases": ["together_ai", "togetherai"],
"aliases": [
"together_ai",
"togetherai"
],
"protocol": "open_ai_completions",
"default_base_url": "https://api.together.xyz/v1",
"api_key_env": "TOGETHER_API_KEY",
Expand All @@ -180,7 +198,9 @@
},
{
"id": "fireworks",
"aliases": ["fireworks_ai"],
"aliases": [
"fireworks_ai"
],
"protocol": "open_ai_completions",
"default_base_url": "https://api.fireworks.ai/inference/v1",
"api_key_env": "FIREWORKS_API_KEY",
Expand All @@ -198,7 +218,9 @@
},
{
"id": "deepseek",
"aliases": ["deep_seek"],
"aliases": [
"deep_seek"
],
"protocol": "open_ai_completions",
"default_base_url": "https://api.deepseek.com/v1",
"api_key_env": "DEEPSEEK_API_KEY",
Expand Down Expand Up @@ -234,7 +256,9 @@
},
{
"id": "sambanova",
"aliases": ["samba_nova"],
"aliases": [
"samba_nova"
],
"protocol": "open_ai_completions",
"default_base_url": "https://api.sambanova.ai/v1",
"api_key_env": "SAMBANOVA_API_KEY",
Expand All @@ -249,5 +273,131 @@
"display_name": "SambaNova",
"can_list_models": false
}
},
{
"id": "gemini",
"aliases": [
"google_gemini",
"google"
],
"protocol": "open_ai_completions",
"default_base_url": "https://generativelanguage.googleapis.com/v1beta/openai",
"api_key_env": "GEMINI_API_KEY",
"api_key_required": true,
"model_env": "GEMINI_MODEL",
"default_model": "gemini-2.5-flash",
Copy link
Copy Markdown
Contributor

@gemini-code-assist gemini-code-assist Bot Mar 7, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

high

The default_model for Gemini is set to gemini-2.5-flash. This appears to be a typo, as the current flash model from Google is gemini-1.5-flash. Using an incorrect model name will lead to errors for users.

Suggested change
"default_model": "gemini-2.5-flash",
"default_model": "gemini-1.5-flash",

"description": "Google Gemini (via OpenAI-compatible endpoint)",
"setup": {
"kind": "api_key",
"secret_name": "llm_gemini_api_key",
"key_url": "https://aistudio.google.com/app/apikey",
"display_name": "Google Gemini",
"can_list_models": true
}
},
{
"id": "bedrock",
"aliases": [
"aws_bedrock",
"aws"
],
"protocol": "open_ai_completions",
"api_key_env": "BEDROCK_ACCESS_KEY",
"api_key_required": false,
"base_url_env": "BEDROCK_BASE_URL",
"model_env": "BEDROCK_MODEL",
"default_model": "anthropic.claude-3-5-sonnet-20241022-v2:0",
Copy link
Copy Markdown
Contributor

@gemini-code-assist gemini-code-assist Bot Mar 7, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

high

The default_model for AWS Bedrock, anthropic.claude-3-5-sonnet-20241022-v2:0, appears to be incorrect. The date 20241022 is in the future and the version v2:0 is unusual. The official model ID for Claude 3.5 Sonnet on Bedrock is anthropic.claude-3-5-sonnet-20240620-v1:0. Using an invalid model ID will cause requests to fail.

Suggested change
"default_model": "anthropic.claude-3-5-sonnet-20241022-v2:0",
"default_model": "anthropic.claude-3-5-sonnet-20240620-v1:0",

"description": "AWS Bedrock (requires LiteLLM or OpenAI-compatible proxy)",
"setup": {
"kind": "open_ai_compatible",
"secret_name": "llm_bedrock_api_key",
"display_name": "AWS Bedrock",
"can_list_models": false
}
},
{
"id": "ionet",
"aliases": [
"io_net",
"io.net"
],
"protocol": "open_ai_completions",
"default_base_url": "https://api.intelligence.io.solutions/api/v1",
"api_key_env": "IONET_API_KEY",
"api_key_required": true,
"model_env": "IONET_MODEL",
"default_model": "deepseek-coder-v2-instruct",
"description": "io.net Intelligence API",
"setup": {
"kind": "api_key",
"secret_name": "llm_ionet_api_key",
"key_url": "https://cloud.io.net/intelligence",
"display_name": "io.net",
"can_list_models": true
}
},
{
"id": "mistral",
"aliases": [
"mistral_ai",
"mistralai"
],
"protocol": "open_ai_completions",
"default_base_url": "https://api.mistral.ai/v1",
"api_key_env": "MISTRAL_API_KEY",
"api_key_required": true,
"model_env": "MISTRAL_MODEL",
"default_model": "mistral-large-latest",
"description": "Mistral AI API",
"setup": {
"kind": "api_key",
"secret_name": "llm_mistral_api_key",
"key_url": "https://console.mistral.ai/api-keys",
"display_name": "Mistral",
"can_list_models": true
}
},
{
"id": "yandex",
"aliases": [
"yandex_ai_studio",
"yandexgpt",
"yandex_gpt"
],
"protocol": "open_ai_completions",
"default_base_url": "https://ai.api.cloud.yandex.net/v1",
"api_key_env": "YANDEX_API_KEY",
"api_key_required": true,
"model_env": "YANDEX_MODEL",
"extra_headers_env": "YANDEX_EXTRA_HEADERS",
"default_model": "yandexgpt-lite",
"description": "Yandex AI Studio (YandexGPT)",
"setup": {
"kind": "api_key",
"secret_name": "llm_yandex_api_key",
"key_url": "https://aistudio.yandex.ru/platform/folders/",
"display_name": "Yandex AI Studio",
"can_list_models": true
}
},
{
"id": "cloudflare",
"aliases": [
"cloudflare_ai",
"cf_ai"
],
"protocol": "open_ai_completions",
"api_key_env": "CLOUDFLARE_API_KEY",
"api_key_required": true,
"base_url_env": "CLOUDFLARE_BASE_URL",
"model_env": "CLOUDFLARE_MODEL",
"default_model": "@cf/meta/llama-3.3-70b-instruct-fp8-fast",
Copy link
Copy Markdown
Contributor

@gemini-code-assist gemini-code-assist Bot Mar 7, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

high

The default_model for Cloudflare, @cf/meta/llama-3.3-70b-instruct-fp8-fast, seems to contain a typo. Meta has not released llama-3.3; the latest version is Llama 3.1. A valid model on Cloudflare is @cf/meta/llama-3.1-70b-instruct. Using a non-existent model will result in errors.

Suggested change
"default_model": "@cf/meta/llama-3.3-70b-instruct-fp8-fast",
"default_model": "@cf/meta/llama-3.1-70b-instruct",

"description": "Cloudflare Workers AI",
"setup": {
"kind": "open_ai_compatible",
"secret_name": "llm_cloudflare_api_key",
"display_name": "Cloudflare Workers AI",
"can_list_models": false
}
}
]
]
2 changes: 2 additions & 0 deletions src/llm/registry.rs
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -450,6 +450,8 @@ mod tests {
if def.protocol == ProviderProtocol::OpenAiCompletions
&& def.id != "openai"
&& def.id != "openai_compatible"
&& def.id != "bedrock"
&& def.id != "cloudflare"
{
assert!(
def.default_base_url.is_some(),
Expand Down