Skip to content

Commit 9125423

Browse files
authored
Merge pull request #19 from andrewginns/december-2025-updates
Jan/December 2025 updates
2 parents 697c3fc + 6c9982d commit 9125423

20 files changed

Lines changed: 4294 additions & 949 deletions

.env.example

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -1,5 +1,6 @@
11
GEMINI_API_KEY=
22
OPENAI_API_KEY=
3+
OPENROUTER_API_KEY=
34
LOGFIRE_TOKEN=
45
AWS_REGION=
56
AWS_PROFILE=

AGENTS.md

Lines changed: 41 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,41 @@
1+
# Repository Guidelines
2+
3+
## Project Structure & Module Organization
4+
- `agents_mcp_usage/` is the main Python package.
5+
- `basic_mcp/` contains **single-server** MCP demos across frameworks (`basic_mcp_use/`).
6+
- `multi_mcp/` contains **multi-server** MCP demos across frameworks (`multi_mcp_use/`).
7+
- `factory/` contains model/provider factory helpers (see `MODEL_FACTORY.md` and `model_factory.py`).
8+
- `evaluations/mermaid_evals/` holds the Merbench benchmark: runners (`evals_pydantic_mcp.py`, `run_multi_evals.py`), Streamlit UI (`merbench_ui.py`), scripts (`scripts/`), and config (`costs.json`, schemas).
9+
- `mcp_servers/` contains local MCP servers used by the agents (e.g., `example_server.py`, `mermaid_validator.py`).
10+
- `docs/` holds images and documentation assets.
11+
- Evaluation CSV/log outputs are written under `mermaid_eval_results/<timestamp>/`.
12+
- Processed/merged dashboard datasets live under `agents_mcp_usage/evaluations/mermaid_evals/results/` (JSON).
13+
14+
## Build, Test, and Development Commands
15+
- `make install` — install Python deps via `uv sync` and globally install Mermaid CLI via `npm` (needed for validation).
16+
- `make upgrade` — refresh deps (`uv sync -U`).
17+
- `make lint` — run `ruff` over the repo.
18+
- `make leaderboard` — launch Merbench Streamlit dashboard.
19+
- `make adk_basic_ui` / `make adk_multi_ui` — start Google ADK web UI for demos.
20+
- Example runs:
21+
- `uv run agents_mcp_usage/basic_mcp/basic_mcp_use/pydantic_mcp.py`
22+
- `uv run agents_mcp_usage/basic_mcp/basic_mcp_use/oai-agent_mcp.py`
23+
- `uv run agents_mcp_usage/basic_mcp/basic_mcp_use/pydantic_mcp_factory.py --list-models`
24+
- `uv run agents_mcp_usage/evaluations/mermaid_evals/run_multi_evals.py --runs 5 --parallel`
25+
26+
## Coding Style & Naming Conventions
27+
- Python 3.13+; 4‑space indentation; prefer type hints throughout.
28+
- `snake_case` for functions/variables, `UpperCamelCase` for classes, constants in `UPPER_SNAKE_CASE`.
29+
- `ruff` is the linter of record; keep imports clean, avoid unused vars, and fix lint before PRs.
30+
31+
## Testing Guidelines
32+
- No dedicated unit test suite yet. Validate changes by:
33+
- running the affected demo/eval script with `uv run …`, and/or
34+
- opening the dashboard on sample CSVs.
35+
- If you add reusable logic, consider introducing lightweight tests in a new `tests/` folder and wire them into the Makefile.
36+
37+
## Commit & Pull Request Guidelines
38+
- Use Conventional Commit prefixes seen in history: `feat:`, `fix:`, `docs:`, `chore:`, `refactor:` (or `refact:`). Keep subjects short and imperative.
39+
- PRs should include: what changed, why, how to run/verify, and any schema/output impacts.
40+
- For UI or JSON schema changes, attach screenshots and/or a small example CSV/JSON.
41+
- Never commit API keys; use `.env` locally and update `.env.example` when adding new config.

README.md

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -63,6 +63,7 @@ This project aims to teach:
6363
- `langgraph_mcp.py` - Example of using MCP with LangGraph
6464
- `oai-agent_mcp.py` - Example of using MCP with OpenAI Agents
6565
- `pydantic_mcp.py` - Example of using MCP with Pydantic-AI
66+
- `pydantic_mcp_factory.py` - Example demonstrating the model factory pattern for multi-provider support
6667

6768

6869
- **[agents_mcp_usage/multi_mcp/](agents_mcp_usage/multi_mcp/)** - Advanced multi-MCP server integration examples

agents_mcp_usage/basic_mcp/README.md

Lines changed: 28 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -225,6 +225,33 @@ Key features:
225225
- Employs context manager for server lifecycle management
226226
- Includes comprehensive instrumentation for both MCP and Pydantic-AI
227227

228+
### Pydantic-AI with Model Factory Pattern
229+
230+
**File:** `pydantic_mcp_factory.py`
231+
232+
This example demonstrates how to use the model factory pattern to create agents with different providers while maintaining the same MCP functionality.
233+
234+
```bash
235+
# Use default Gemini model
236+
uv run agents_mcp_usage/basic_mcp/basic_mcp_use/pydantic_mcp_factory.py
237+
238+
# Use DeepSeek model
239+
uv run agents_mcp_usage/basic_mcp/basic_mcp_use/pydantic_mcp_factory.py --model deepseek
240+
241+
# Use OpenAI model with explicit provider
242+
uv run agents_mcp_usage/basic_mcp/basic_mcp_use/pydantic_mcp_factory.py --model "openai:gpt-4"
243+
244+
# List available model aliases
245+
uv run agents_mcp_usage/basic_mcp/basic_mcp_use/pydantic_mcp_factory.py --list-models
246+
```
247+
248+
Key features:
249+
- Demonstrates the model factory pattern for multi-provider support
250+
- Supports 13+ providers including Google, OpenAI, Anthropic, AWS Bedrock, DeepSeek, OpenRouter, GitHub, Grok, Perplexity, and local Ollama models
251+
- Command-line interface for easy model switching
252+
- Shows how to handle provider-specific configurations
253+
- Maintains the same MCP server functionality across all providers
254+
228255

229256
## Understanding the Examples
230257

@@ -245,4 +272,4 @@ All examples connect to the same MCP server defined in `mcp_servers/example_serv
245272
- A time tool (`get_current_time()`)
246273
- A dynamic greeting resource (`greeting://{name}`)
247274

248-
You can modify the MCP server to add your own tools and resources for experimentation.
275+
You can modify the MCP server to add your own tools and resources for experimentation.
Lines changed: 216 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,216 @@
1+
"""
2+
Pydantic MCP Example using Model Factory Pattern
3+
4+
This example demonstrates how to use the model factory to create agents
5+
with different providers while maintaining the same MCP server setup.
6+
7+
Supported model formats:
8+
- Standard: "gemini-2.5-pro", "gpt-4", "claude-3-opus"
9+
- Provider-prefixed: "openai:gpt-4", "deepseek:deepseek-chat", "bedrock:us.amazon.nova-pro-v1:0"
10+
- Local models: "ollama:llama3.2", "ollama:qwen2.5-coder:7b"
11+
"""
12+
13+
import asyncio
14+
import argparse
15+
from typing import Optional
16+
17+
import logfire
18+
from dotenv import load_dotenv
19+
from pydantic_ai.mcp import MCPServerStdio
20+
21+
from agents_mcp_usage.utils import get_mcp_server_path
22+
from agents_mcp_usage.factory.model_factory import create_agent
23+
24+
load_dotenv()
25+
26+
# Configure logging to logfire if LOGFIRE_TOKEN is set in environment
27+
logfire.configure(send_to_logfire="if-token-present", service_name="pydantic-basic-mcp-factory")
28+
logfire.instrument_mcp()
29+
logfire.instrument_pydantic_ai()
30+
31+
# Preset model aliases demonstrating different provider formats
32+
MODEL_ALIASES = {
33+
# Standard providers (auto-detected)
34+
"gemini": "gemini-2.5-pro-preview-06-05",
35+
"gemini-flash": "gemini-2.5-flash",
36+
"gpt4": "gpt-4",
37+
"claude": "claude-3-opus-20240229",
38+
39+
# Explicit provider prefix
40+
"openai-o4": "openai:o4-mini",
41+
"deepseek": "deepseek:deepseek-chat",
42+
"deepseek-reasoner": "deepseek:deepseek-reasoner",
43+
44+
# AWS Bedrock models
45+
"bedrock-nova": "bedrock:us.amazon.nova-pro-v1:0",
46+
"bedrock-claude": "bedrock:us.anthropic.claude-3-5-sonnet-20240620-v1:0",
47+
48+
# GitHub Models (free tier)
49+
"github-grok": "github:xai/grok-3-mini",
50+
51+
# Local models via Ollama
52+
"ollama-llama": "ollama:llama3.2",
53+
"ollama-qwen": "ollama:qwen2.5-coder:7b",
54+
55+
# Other providers
56+
"perplexity": "perplexity:sonar-pro",
57+
"grok": "grok:grok-2-1212",
58+
}
59+
60+
61+
def get_mcp_server() -> MCPServerStdio:
62+
"""Get the MCP server configuration."""
63+
return MCPServerStdio(
64+
command="uv",
65+
args=[
66+
"run",
67+
str(get_mcp_server_path("example_server.py")),
68+
"stdio",
69+
],
70+
)
71+
72+
73+
async def main(
74+
model: str = "gemini-2.5-pro-preview-06-05",
75+
query: str = "Greet Andrew and give him the current time",
76+
provider_kwargs: Optional[dict] = None
77+
) -> None:
78+
"""Runs the Pydantic agent with a given query using the model factory.
79+
80+
This function demonstrates how to use the model factory to create agents
81+
with different providers while maintaining the same MCP functionality.
82+
83+
Args:
84+
model: The model identifier (e.g., "gemini-2.5-pro", "deepseek:deepseek-chat")
85+
query: The query to run the agent with
86+
provider_kwargs: Optional provider-specific configuration (e.g., for Azure)
87+
"""
88+
# Create MCP server
89+
server = get_mcp_server()
90+
91+
# Create agent using the factory
92+
print(f"Creating agent with model: {model}")
93+
agent = create_agent(
94+
model=model,
95+
mcp_servers=[server],
96+
provider_kwargs=provider_kwargs
97+
)
98+
99+
# Run the agent
100+
try:
101+
async with agent.run_mcp_servers():
102+
result = await agent.run(query)
103+
print(f"\nResult from {model}:")
104+
print(result.output)
105+
106+
# Print usage information if available
107+
usage = result.usage()
108+
if usage:
109+
print(f"\nToken usage: {usage.total_tokens} total "
110+
f"({usage.request_tokens} in, {usage.response_tokens} out)")
111+
112+
except Exception as e:
113+
print(f"Error running agent: {e}")
114+
print("\nMake sure you have the required API keys set:")
115+
print("- GEMINI_API_KEY for Gemini models")
116+
print("- OPENAI_API_KEY for OpenAI models")
117+
print("- DEEPSEEK_API_KEY for DeepSeek models")
118+
print("- AWS credentials for Bedrock models")
119+
print("- Other provider-specific keys as needed")
120+
121+
122+
def parse_arguments():
123+
"""Parse command-line arguments."""
124+
parser = argparse.ArgumentParser(
125+
description="Run Pydantic MCP agent with different model providers",
126+
formatter_class=argparse.RawDescriptionHelpFormatter,
127+
epilog="""
128+
Examples:
129+
# Use default Gemini model
130+
python pydantic_mcp_factory.py
131+
132+
# Use a specific model by alias
133+
python pydantic_mcp_factory.py --model deepseek
134+
135+
# Use a model by full name
136+
python pydantic_mcp_factory.py --model "openai:gpt-4"
137+
138+
# Use a custom query
139+
python pydantic_mcp_factory.py --query "What's the weather like?"
140+
141+
# List available model aliases
142+
python pydantic_mcp_factory.py --list-models
143+
"""
144+
)
145+
146+
parser.add_argument(
147+
"--model", "-m",
148+
default="gemini",
149+
help="Model to use (alias or full model string)"
150+
)
151+
152+
parser.add_argument(
153+
"--query", "-q",
154+
default="Greet Andrew and give him the current time",
155+
help="Query to send to the agent"
156+
)
157+
158+
parser.add_argument(
159+
"--list-models", "-l",
160+
action="store_true",
161+
help="List available model aliases and exit"
162+
)
163+
164+
# Provider-specific options
165+
parser.add_argument(
166+
"--azure-endpoint",
167+
help="Azure endpoint for Azure AI Foundry models"
168+
)
169+
170+
parser.add_argument(
171+
"--azure-api-version",
172+
help="Azure API version for Azure AI Foundry models"
173+
)
174+
175+
parser.add_argument(
176+
"--ollama-base-url",
177+
default="http://localhost:11434/v1",
178+
help="Base URL for Ollama server"
179+
)
180+
181+
return parser.parse_args()
182+
183+
184+
if __name__ == "__main__":
185+
args = parse_arguments()
186+
187+
# List models and exit if requested
188+
if args.list_models:
189+
print("Available model aliases:")
190+
print("-" * 50)
191+
for alias, model_string in MODEL_ALIASES.items():
192+
print(f"{alias:<20} -> {model_string}")
193+
print("\nYou can also use any model string directly, e.g.:")
194+
print(" openrouter:google/gemini-2.5-pro-preview")
195+
print(" fireworks:accounts/fireworks/models/qwq-32b")
196+
print(" together:meta-llama/Llama-3.3-70B-Instruct-Turbo-Free")
197+
exit(0)
198+
199+
# Resolve model alias if applicable
200+
model = MODEL_ALIASES.get(args.model, args.model)
201+
202+
# Build provider kwargs if Azure options are provided
203+
provider_kwargs = {}
204+
if args.azure_endpoint:
205+
provider_kwargs["azure_endpoint"] = args.azure_endpoint
206+
if args.azure_api_version:
207+
provider_kwargs["api_version"] = args.azure_api_version
208+
if args.ollama_base_url and model.startswith("ollama:"):
209+
provider_kwargs["base_url"] = args.ollama_base_url
210+
211+
# Run the main function
212+
asyncio.run(main(
213+
model=model,
214+
query=args.query,
215+
provider_kwargs=provider_kwargs if provider_kwargs else None
216+
))

0 commit comments

Comments
 (0)