Skip to content

Commit b5a7310

Browse files
daniviberclaudegavrielc
authored
feat: add /add-ollama skill for local model inference (qwibitai#712)
* feat: add /add-ollama skill for local model inference Adds a skill that integrates Ollama as an MCP server, allowing the container agent to offload tasks to local models (summarization, translation, general queries) while keeping Claude as orchestrator. Skill contents: - ollama-mcp-stdio.ts: stdio MCP server with ollama_list_models and ollama_generate tools - ollama-watch.sh: macOS notification watcher for Ollama activity - Modifications to index.ts (MCP config) and container-runner.ts (log surfacing) Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * chore: rename skill from /add-ollama to /add-ollama-tool Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com> Co-authored-by: gavrielc <gabicohen22@yahoo.com>
1 parent 6fbc503 commit b5a7310

8 files changed

Lines changed: 1699 additions & 0 deletions

File tree

‎.claude/skills/add-ollama-tool/SKILL.md‎

Lines changed: 152 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,152 @@
1+
---
2+
name: add-ollama-tool
3+
description: Add Ollama MCP server so the container agent can call local models for cheaper/faster tasks like summarization, translation, or general queries.
4+
---
5+
6+
# Add Ollama Integration
7+
8+
This skill adds a stdio-based MCP server that exposes local Ollama models as tools for the container agent. Claude remains the orchestrator but can offload work to local models.
9+
10+
Tools added:
11+
- `ollama_list_models` — lists installed Ollama models
12+
- `ollama_generate` — sends a prompt to a specified model and returns the response
13+
14+
## Phase 1: Pre-flight
15+
16+
### Check if already applied
17+
18+
Read `.nanoclaw/state.yaml`. If `ollama` is in `applied_skills`, skip to Phase 3 (Configure). The code changes are already in place.
19+
20+
### Check prerequisites
21+
22+
Verify Ollama is installed and running on the host:
23+
24+
```bash
25+
ollama list
26+
```
27+
28+
If Ollama is not installed, direct the user to https://ollama.com/download.
29+
30+
If no models are installed, suggest pulling one:
31+
32+
> You need at least one model. I recommend:
33+
>
34+
> ```bash
35+
> ollama pull gemma3:1b # Small, fast (1GB)
36+
> ollama pull llama3.2 # Good general purpose (2GB)
37+
> ollama pull qwen3-coder:30b # Best for code tasks (18GB)
38+
> ```
39+
40+
## Phase 2: Apply Code Changes
41+
42+
Run the skills engine to apply this skill's code package.
43+
44+
### Initialize skills system (if needed)
45+
46+
If `.nanoclaw/` directory doesn't exist yet:
47+
48+
```bash
49+
npx tsx scripts/apply-skill.ts --init
50+
```
51+
52+
### Apply the skill
53+
54+
```bash
55+
npx tsx scripts/apply-skill.ts .claude/skills/add-ollama-tool
56+
```
57+
58+
This deterministically:
59+
- Adds `container/agent-runner/src/ollama-mcp-stdio.ts` (Ollama MCP server)
60+
- Adds `scripts/ollama-watch.sh` (macOS notification watcher)
61+
- Three-way merges Ollama MCP config into `container/agent-runner/src/index.ts` (allowedTools + mcpServers)
62+
- Three-way merges `[OLLAMA]` log surfacing into `src/container-runner.ts`
63+
- Records the application in `.nanoclaw/state.yaml`
64+
65+
If the apply reports merge conflicts, read the intent files:
66+
- `modify/container/agent-runner/src/index.ts.intent.md` — what changed and invariants
67+
- `modify/src/container-runner.ts.intent.md` — what changed and invariants
68+
69+
### Copy to per-group agent-runner
70+
71+
Existing groups have a cached copy of the agent-runner source. Copy the new files:
72+
73+
```bash
74+
for dir in data/sessions/*/agent-runner-src; do
75+
cp container/agent-runner/src/ollama-mcp-stdio.ts "$dir/"
76+
cp container/agent-runner/src/index.ts "$dir/"
77+
done
78+
```
79+
80+
### Validate code changes
81+
82+
```bash
83+
npm run build
84+
./container/build.sh
85+
```
86+
87+
Build must be clean before proceeding.
88+
89+
## Phase 3: Configure
90+
91+
### Set Ollama host (optional)
92+
93+
By default, the MCP server connects to `http://host.docker.internal:11434` (Docker Desktop) with a fallback to `localhost`. To use a custom Ollama host, add to `.env`:
94+
95+
```bash
96+
OLLAMA_HOST=http://your-ollama-host:11434
97+
```
98+
99+
### Restart the service
100+
101+
```bash
102+
launchctl kickstart -k gui/$(id -u)/com.nanoclaw # macOS
103+
# Linux: systemctl --user restart nanoclaw
104+
```
105+
106+
## Phase 4: Verify
107+
108+
### Test via WhatsApp
109+
110+
Tell the user:
111+
112+
> Send a message like: "use ollama to tell me the capital of France"
113+
>
114+
> The agent should use `ollama_list_models` to find available models, then `ollama_generate` to get a response.
115+
116+
### Monitor activity (optional)
117+
118+
Run the watcher script for macOS notifications when Ollama is used:
119+
120+
```bash
121+
./scripts/ollama-watch.sh
122+
```
123+
124+
### Check logs if needed
125+
126+
```bash
127+
tail -f logs/nanoclaw.log | grep -i ollama
128+
```
129+
130+
Look for:
131+
- `Agent output: ... Ollama ...` — agent used Ollama successfully
132+
- `[OLLAMA] >>> Generating` — generation started (if log surfacing works)
133+
- `[OLLAMA] <<< Done` — generation completed
134+
135+
## Troubleshooting
136+
137+
### Agent says "Ollama is not installed"
138+
139+
The agent is trying to run `ollama` CLI inside the container instead of using the MCP tools. This means:
140+
1. The MCP server wasn't registered — check `container/agent-runner/src/index.ts` has the `ollama` entry in `mcpServers`
141+
2. The per-group source wasn't updated — re-copy files (see Phase 2)
142+
3. The container wasn't rebuilt — run `./container/build.sh`
143+
144+
### "Failed to connect to Ollama"
145+
146+
1. Verify Ollama is running: `ollama list`
147+
2. Check Docker can reach the host: `docker run --rm curlimages/curl curl -s http://host.docker.internal:11434/api/tags`
148+
3. If using a custom host, check `OLLAMA_HOST` in `.env`
149+
150+
### Agent doesn't use Ollama tools
151+
152+
The agent may not know about the tools. Try being explicit: "use the ollama_generate tool with gemma3:1b to answer: ..."

‎.claude/skills/add-ollama-tool/add/container/agent-runner/src/ollama-mcp-stdio.ts‎

Lines changed: 147 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,147 @@
1+
/**
2+
* Ollama MCP Server for NanoClaw
3+
* Exposes local Ollama models as tools for the container agent.
4+
* Uses host.docker.internal to reach the host's Ollama instance from Docker.
5+
*/
6+
7+
import { McpServer } from '@modelcontextprotocol/sdk/server/mcp.js';
8+
import { StdioServerTransport } from '@modelcontextprotocol/sdk/server/stdio.js';
9+
import { z } from 'zod';
10+
11+
import fs from 'fs';
12+
import path from 'path';
13+
14+
const OLLAMA_HOST = process.env.OLLAMA_HOST || 'http://host.docker.internal:11434';
15+
const OLLAMA_STATUS_FILE = '/workspace/ipc/ollama_status.json';
16+
17+
function log(msg: string): void {
18+
console.error(`[OLLAMA] ${msg}`);
19+
}
20+
21+
function writeStatus(status: string, detail?: string): void {
22+
try {
23+
const data = { status, detail, timestamp: new Date().toISOString() };
24+
const tmpPath = `${OLLAMA_STATUS_FILE}.tmp`;
25+
fs.mkdirSync(path.dirname(OLLAMA_STATUS_FILE), { recursive: true });
26+
fs.writeFileSync(tmpPath, JSON.stringify(data));
27+
fs.renameSync(tmpPath, OLLAMA_STATUS_FILE);
28+
} catch { /* best-effort */ }
29+
}
30+
31+
async function ollamaFetch(path: string, options?: RequestInit): Promise<Response> {
32+
const url = `${OLLAMA_HOST}${path}`;
33+
try {
34+
return await fetch(url, options);
35+
} catch (err) {
36+
// Fallback to localhost if host.docker.internal fails
37+
if (OLLAMA_HOST.includes('host.docker.internal')) {
38+
const fallbackUrl = url.replace('host.docker.internal', 'localhost');
39+
return await fetch(fallbackUrl, options);
40+
}
41+
throw err;
42+
}
43+
}
44+
45+
const server = new McpServer({
46+
name: 'ollama',
47+
version: '1.0.0',
48+
});
49+
50+
server.tool(
51+
'ollama_list_models',
52+
'List all locally installed Ollama models. Use this to see which models are available before calling ollama_generate.',
53+
{},
54+
async () => {
55+
log('Listing models...');
56+
writeStatus('listing', 'Listing available models');
57+
try {
58+
const res = await ollamaFetch('/api/tags');
59+
if (!res.ok) {
60+
return {
61+
content: [{ type: 'text' as const, text: `Ollama API error: ${res.status} ${res.statusText}` }],
62+
isError: true,
63+
};
64+
}
65+
66+
const data = await res.json() as { models?: Array<{ name: string; size: number; modified_at: string }> };
67+
const models = data.models || [];
68+
69+
if (models.length === 0) {
70+
return { content: [{ type: 'text' as const, text: 'No models installed. Run `ollama pull <model>` on the host to install one.' }] };
71+
}
72+
73+
const list = models
74+
.map(m => `- ${m.name} (${(m.size / 1e9).toFixed(1)}GB)`)
75+
.join('\n');
76+
77+
log(`Found ${models.length} models`);
78+
return { content: [{ type: 'text' as const, text: `Installed models:\n${list}` }] };
79+
} catch (err) {
80+
return {
81+
content: [{ type: 'text' as const, text: `Failed to connect to Ollama at ${OLLAMA_HOST}: ${err instanceof Error ? err.message : String(err)}` }],
82+
isError: true,
83+
};
84+
}
85+
},
86+
);
87+
88+
server.tool(
89+
'ollama_generate',
90+
'Send a prompt to a local Ollama model and get a response. Good for cheaper/faster tasks like summarization, translation, or general queries. Use ollama_list_models first to see available models.',
91+
{
92+
model: z.string().describe('The model name (e.g., "llama3.2", "mistral", "gemma2")'),
93+
prompt: z.string().describe('The prompt to send to the model'),
94+
system: z.string().optional().describe('Optional system prompt to set model behavior'),
95+
},
96+
async (args) => {
97+
log(`>>> Generating with ${args.model} (${args.prompt.length} chars)...`);
98+
writeStatus('generating', `Generating with ${args.model}`);
99+
try {
100+
const body: Record<string, unknown> = {
101+
model: args.model,
102+
prompt: args.prompt,
103+
stream: false,
104+
};
105+
if (args.system) {
106+
body.system = args.system;
107+
}
108+
109+
const res = await ollamaFetch('/api/generate', {
110+
method: 'POST',
111+
headers: { 'Content-Type': 'application/json' },
112+
body: JSON.stringify(body),
113+
});
114+
115+
if (!res.ok) {
116+
const errorText = await res.text();
117+
return {
118+
content: [{ type: 'text' as const, text: `Ollama error (${res.status}): ${errorText}` }],
119+
isError: true,
120+
};
121+
}
122+
123+
const data = await res.json() as { response: string; total_duration?: number; eval_count?: number };
124+
125+
let meta = '';
126+
if (data.total_duration) {
127+
const secs = (data.total_duration / 1e9).toFixed(1);
128+
meta = `\n\n[${args.model} | ${secs}s${data.eval_count ? ` | ${data.eval_count} tokens` : ''}]`;
129+
log(`<<< Done: ${args.model} | ${secs}s | ${data.eval_count || '?'} tokens | ${data.response.length} chars`);
130+
writeStatus('done', `${args.model} | ${secs}s | ${data.eval_count || '?'} tokens`);
131+
} else {
132+
log(`<<< Done: ${args.model} | ${data.response.length} chars`);
133+
writeStatus('done', `${args.model} | ${data.response.length} chars`);
134+
}
135+
136+
return { content: [{ type: 'text' as const, text: data.response + meta }] };
137+
} catch (err) {
138+
return {
139+
content: [{ type: 'text' as const, text: `Failed to call Ollama: ${err instanceof Error ? err.message : String(err)}` }],
140+
isError: true,
141+
};
142+
}
143+
},
144+
);
145+
146+
const transport = new StdioServerTransport();
147+
await server.connect(transport);

‎.claude/skills/add-ollama-tool/add/scripts/ollama-watch.sh‎

Lines changed: 41 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,41 @@
1+
#!/bin/bash
2+
# Watch NanoClaw IPC for Ollama activity and show macOS notifications
3+
# Usage: ./scripts/ollama-watch.sh
4+
5+
cd "$(dirname "$0")/.." || exit 1
6+
7+
echo "Watching for Ollama activity..."
8+
echo "Press Ctrl+C to stop"
9+
echo ""
10+
11+
LAST_TIMESTAMP=""
12+
13+
while true; do
14+
# Check all group IPC dirs for ollama_status.json
15+
for status_file in data/ipc/*/ollama_status.json; do
16+
[ -f "$status_file" ] || continue
17+
18+
TIMESTAMP=$(python3 -c "import json; print(json.load(open('$status_file'))['timestamp'])" 2>/dev/null)
19+
[ -z "$TIMESTAMP" ] && continue
20+
[ "$TIMESTAMP" = "$LAST_TIMESTAMP" ] && continue
21+
22+
LAST_TIMESTAMP="$TIMESTAMP"
23+
STATUS=$(python3 -c "import json; d=json.load(open('$status_file')); print(d['status'])" 2>/dev/null)
24+
DETAIL=$(python3 -c "import json; d=json.load(open('$status_file')); print(d.get('detail',''))" 2>/dev/null)
25+
26+
case "$STATUS" in
27+
generating)
28+
osascript -e "display notification \"$DETAIL\" with title \"NanoClaw → Ollama\" sound name \"Submarine\"" 2>/dev/null
29+
echo "$(date +%H:%M:%S) 🔄 $DETAIL"
30+
;;
31+
done)
32+
osascript -e "display notification \"$DETAIL\" with title \"NanoClaw ← Ollama ✓\" sound name \"Glass\"" 2>/dev/null
33+
echo "$(date +%H:%M:%S) ✅ $DETAIL"
34+
;;
35+
listing)
36+
echo "$(date +%H:%M:%S) 📋 Listing models..."
37+
;;
38+
esac
39+
done
40+
sleep 0.5
41+
done

‎.claude/skills/add-ollama-tool/manifest.yaml‎

Lines changed: 17 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,17 @@
1+
skill: ollama
2+
version: 1.0.0
3+
description: "Local Ollama model inference via MCP server"
4+
core_version: 0.1.0
5+
adds:
6+
- container/agent-runner/src/ollama-mcp-stdio.ts
7+
- scripts/ollama-watch.sh
8+
modifies:
9+
- container/agent-runner/src/index.ts
10+
- src/container-runner.ts
11+
structured:
12+
npm_dependencies: {}
13+
env_additions:
14+
- OLLAMA_HOST
15+
conflicts: []
16+
depends: []
17+
test: "npm run build"

0 commit comments

Comments
 (0)