Commit 7a9496b
feat: Sandbox jobs (nearai#4)
* Orchestrating jobs and running them in sandboxes
* Fix heartbeat: dynamic max_tokens, empty content guard, notification fallback
- Query /v1/models API for context_length and set max_tokens to half
(floor 4096) instead of hardcoded 1024; reasoning models like GLM-4.7
need much larger budgets
- Guard against empty LLM content (reasoning models can burn all tokens
on chain-of-thought and return content: null)
- Simplify notification routing: try configured channel first, fall back
to broadcast_all so heartbeat alerts always reach someone
- Add ModelMetadata struct and model_metadata() to LlmProvider trait
- Refactor NearAiChatProvider::list_models into shared fetch_models()
- Add standalone test_heartbeat example for isolated debugging
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
* Add job detail view with drill-down from jobs list
Click a job row to see full details across four sub-tabs:
Overview (metadata grid, description, state transitions timeline),
Actions (expandable tool call cards with input/output JSON),
Thinking (conversation messages styled by role), and
Files (embedded workspace tree browser).
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
* Strip model-internal XML tags from LLM responses, fix Telegram parse_mode 400
Some models (GLM-4.7, etc.) emit <tool_call>tool_list</tool_call> in the
content field instead of using the OpenAI tool_calls array. This XML leaks
through to channels as text, and Telegram's Markdown parser chokes on the
underscores, returning 400 "can't parse entities".
Two fixes:
- Generalize clean_response() to strip <tool_call>, <function_call>,
<tool_calls>, and pipe-delimited variants (<|tool_call|>) alongside
the existing <thinking> tag stripping
- Add Telegram send_message helper with parse_mode fallback: try Markdown
first, retry as plain text on "can't parse entities" 400 errors
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
* Add SystemCommand submission type for thread-state-independent commands
System commands (/help, /model, /version, /tools, /ping, /debug) now
bypass thread-state checks and safety validation via a dedicated
Submission::SystemCommand variant. Previously these flowed through
process_user_input() which blocked them during Processing/AwaitingApproval
/Completed states.
- Add /model [name] for runtime model switching with provider validation
- Add active_model_name()/set_model() to LlmProvider trait with RwLock
hot-swap in both NEAR AI providers
- Rewrite /help with aligned columns grouped by category
- Expand REPL tab-completion from 10 to 23 slash commands
- Remove REPL-local /help interception (now handled by agent)
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
* Add per-tool execution timeouts, auto-create sandbox project dirs, serve built files
The sandbox e2e pipeline (agent -> container -> built website -> browsable URL)
was broken by three gaps: hardcoded 60s timeouts killed sandbox jobs that need
minutes, no auto-created project directory meant container output vanished, and
no HTTP route to browse the built files.
- Add `execution_timeout()` to the `Tool` trait (default 60s), replace all four
hardcoded `Duration::from_secs(60)` call sites (agent_loop, worker, scheduler,
worker/runtime) with the per-tool value
- Override to 660s in `RunInSandboxTool` (10 min polling + 60s buffer)
- Auto-create `~/.ironclaw/projects/{uuid}/` when no `project_dir` is specified,
so every sandbox job gets a persistent bind mount
- Include `project_dir` and `browse_url` in sandbox tool output JSON
- Add `/projects/{id}` and `/projects/{id}/{path}` static file serving routes
to the web gateway with path traversal protection and MIME type detection
- Add `mime_guess` dependency for content-type detection
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
* Apply cargo fmt to wizard.rs after merge
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
* Persist sandbox jobs in DB, fix web UI, unify job model
Sandbox container jobs were invisible to the web UI because they lived
only in ContainerJobManager's in-memory HashMap while the API queried
ContextManager. This persists them to the agent_jobs table and fixes
all six front-end bugs (empty job list, broken back button, empty
actions/thinking tabs, wrong files tab, stuck status, no persistence).
Key changes:
- V4 migration adds project_dir and user_id columns to agent_jobs
- Embedded migrations via refinery (no external CLI needed)
- SandboxJobRecord CRUD in Store with fire-and-forget DB writes
- Unified job_id: sandbox tool generates UUID, passes to ContainerJobManager
- Web API queries DB for sandbox jobs, merges with ContextManager direct jobs
- New endpoints: restart, project file list/read with path traversal protection
- Front-end: rebuild DOM on back navigation, sandbox-aware tabs, job cards in
chat stream, source badges, restart button for failed/interrupted jobs
- Gateway defaults to enabled, prints Web UI URL on startup
- Stale jobs marked "interrupted" on restart for visibility and restartability
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
* Secure in-chat auth: tokens never touch the LLM or chat history
Remove the token parameter from tool_auth so the LLM cannot pass raw
API keys. Add dedicated REST (POST /api/chat/auth-token) and WebSocket
(auth_token) endpoints that route tokens directly to ext_mgr.auth(),
completely bypassing the message pipeline, turns, history, and compaction.
Web UI shows an auth card (password input + OAuth button) when the agent
enters auth mode, submitted via the dedicated endpoint. CLI auth mode
interception is unchanged (already secure).
New StatusUpdate::AuthRequired/AuthCompleted variants propagate through
all channels (SSE, WebSocket, REPL, WASM).
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
* feat: Add Claude Code mode for sandbox jobs
Run Claude Code CLI inside Docker containers as an alternative to the
standard worker mode. The bridge spawns `claude -p` with stream-json
output, posts events to the orchestrator, and supports follow-up
prompts via `--resume`.
Key additions:
- `claude-bridge` CLI subcommand and ClaudeBridgeRuntime
- JobMode enum (Worker vs ClaudeCode) with per-mode container config
- Orchestrator endpoints for Claude events and prompt polling
- SSE event variants for real-time Claude Code streaming to frontend
- Claude Code sub-tab in web UI with terminal-style output and input bar
- Database migration for job_mode column and claude_code_events table
- ClaudeCodeConfig with env var support (CLAUDE_CODE_ENABLED, etc.)
- Mode parameter on run_in_sandbox tool schema
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
* fix: Skip create_job tool when sandbox is enabled to prevent duplicate jobs
When sandbox mode is on, the LLM would call create_job (creating a
pending "direct" entry) then run_in_sandbox (creating a second "sandbox"
entry), producing two jobs in the list for a single user request.
Now register_job_tools() skips create_job when sandbox is enabled since
run_in_sandbox already creates tracked jobs. Also improved the
run_in_sandbox description to guide the LLM to use it directly and to
mention wait=false for async execution.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
* feat: Web gateway UI quality-of-life improvements
Phase 1: Send button disabled state to prevent double-sends, copy button
on code blocks, confirm() guards on destructive actions, SSE-driven job
list auto-refresh, log filters re-applied on tab switch, jobEvents memory
leak fix (cap at 500, cleanup after 60s).
Phase 2: Toast notification system replacing chat-based system messages,
memory search highlighting with centered snippets, keyboard shortcuts
(Ctrl+1-5 tabs, Ctrl+K focus, Ctrl+N new thread, Escape close/blur),
activity tab toolbar with event type filter and auto-scroll toggle.
Phase 3: Thread sidebar with load/switch/create, thread_id passed with
messages, collapsible to hamburger. Memory inline editing with textarea,
Save/Cancel, POST to /api/memory/write.
Phase 4: Gateway status popover on hover (polls every 30s), extension
install form (name/URL/kind), markdown rendering in memory viewer for
.md files, mobile responsive layout at 768px breakpoint.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
* feat: Add routines system, remove non-sandbox job mode from web UI
Routines: scheduled & reactive job system with cron and event triggers,
lightweight (single LLM call) and full-job execution modes, guardrails
(cooldown, max concurrent, dedup), and LLM-facing tools for CRUD.
Web UI: remove ContextManager-backed "direct" job mode entirely. Jobs
are now exclusively sandbox-backed (DB + container). Simplify job detail
response, drop dead types (ActionInfo, MessageInfo, MessageToolCallInfo),
fix Browse Files CSS loading (trailing-slash redirect), fix Activity tab
event rendering.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
* fix: Re-enable chat input on agent completion, auto-auth on tool_activate, recover tool calls from content XML
Three fixes:
1. Chat input stays disabled after agent finishes: the "Done" status
SSE event now calls enableChatInput() as a safety net when the
response event is empty or lost. Same for auth_completed and
cancelAuth().
2. tool_activate never triggers auth: when activation fails due to
missing authentication, it now auto-initiates the auth flow
(same pattern as the web API handler). detect_auth_awaiting()
also matches tool_activate results now.
3. Models like GLM-4.7 emit tool calls as XML tags in content
(<tool_call>tool_list</tool_call>) instead of using the structured
tool_calls array. recover_tool_calls_from_content() extracts and
validates these before falling back to plain text.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
* feat: Add routines web UI tab, update docs for sandbox-jobs branch
Add full routines management to the web gateway (list, detail, trigger,
toggle, delete) with 7 new API endpoints, response types, and frontend
(HTML, JS, CSS). Update FEATURE_PARITY.md (~23 rows), CLAUDE.md (new
subsystems, config, TODOs), and README.md (architecture diagram,
features, components, fix onboard command).
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
* fix: Bind Telegram bot to owner account during setup
Without owner binding, anyone who discovers the bot can send it messages.
The setup wizard now prompts the user to message their bot, captures their
Telegram user ID via getUpdates, and persists it as telegram_owner_id in
settings. On startup, the owner_id is injected into the WASM channel config
so the existing owner restriction logic drops messages from non-owners.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
* feat: Move settings from disk to PostgreSQL database
Settings previously lived in three JSON files on disk (settings.json,
mcp-servers.json, session.json). This made them inaccessible from the
web UI and caused redundant disk reads (Settings::load() called 8+
times during startup).
Now all settings live in a `settings` table (user_id + key -> JSONB)
with only 4 bootstrap fields remaining on disk (database_url, pool
size, secrets key source, onboard_completed) since they're needed
before the DB connection exists.
- Add V8 migration for settings table
- Add BootstrapConfig (thin disk file) and Settings DB round-trip
- Add Store CRUD methods for settings (get/set/delete/list/bulk)
- Refactor Config to load from DB (env > DB > default cascade)
- Add SessionManager DB persistence for session tokens
- Add DB-backed MCP server config load/save functions
- Add 6 settings web API endpoints (list/get/set/delete/export/import)
- Add one-time disk-to-DB migration on first boot
- Make CLI config commands async with DB access (disk fallback)
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
* feat: Seed workspace on boot, fix gateway duplicate logs and URL auto-auth
- Add Workspace::seed_if_empty() to create core identity files (README,
MEMORY, IDENTITY, SOUL, AGENTS, USER, HEARTBEAT) when missing, called
on every boot without overwriting existing user edits
- Remove duplicate gateway log lines from web/mod.rs (main.rs has the
useful clickable ?token= URL)
- Auto-authenticate from ?token= URL parameter in the web UI and strip
the token from the address bar after successful auth
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
* fix: Harden sandbox security (path traversal + orchestrator auth)
Two vulnerabilities fixed:
1. project_dir path traversal: The create_job tool let the LLM specify
arbitrary host paths for Docker bind mounts. Removed project_dir from
the tool schema entirely, and added canonicalization + prefix validation
at both resolve_project_dir() and the job_manager bind mount point.
2. Orchestrator API auth bypass: worker_auth_middleware was defined but
never applied. Each handler manually called validate_token(), so any
new endpoint that forgot would be publicly accessible. Applied the
middleware as route_layer on all /worker/ routes, removed manual auth
from all 7 handlers. Bind to 127.0.0.1 on macOS/Windows (Linux keeps
0.0.0.0 since containers reach host via docker bridge, not loopback).
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
* feat: Rework gateway chat with pinned assistant, pagination, and NEAR AI response chaining
Implements the 4-phase plan for overhauling the web gateway chat:
- Phase 1: Pinned "Assistant" thread at top of sidebar, regular threads below
- Phase 2: Cursor-based history pagination with infinite scroll
- Phase 3: NEAR AI previous_response_id chaining (delta-only messages),
with fallback to full history on chain errors, and DB persistence of
chain state across restarts
- Phase 4: SSE thread isolation (events filtered by thread_id)
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
* fix: Add per-request HTTP timeout to WASM host, redact credentials in errors
Three fixes for WASM channel reliability:
1. Per-request timeout: Add optional timeout-ms parameter to http-request
in both channel and tool WIT interfaces. Telegram long-poll now specifies
35s (outliving the 30s server-side hold), while regular API calls use
the 30s default. Fixes the triple-30s timeout race that caused polling
failures.
2. Credential redaction: reqwest::Error includes the full URL (with injected
bot tokens) in its Display output. Scrub credential values from error
messages before logging or returning to WASM.
3. Webhook route registration: Remove tunnel URL gate so webhook routes are
always available when webhook channels exist, not only when TUNNEL_URL
is configured.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
* chore: Fix clippy warnings in WASM tools and channels
- slack channel: allow dead_code on signing_secret_name (forward compat field)
- gmail tool: use div_ceil() instead of manual (n+2)/3
- google-calendar tool: extract CreateEventParams/UpdateEventParams structs
to fix too-many-arguments warnings
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
* Fix approval flow
* fix: Rebuild bundled telegram.wasm with updated WIT interface
The bundled WASM binary must match the host's WIT definition.
Previous binary was compiled against the old 4-arg http-request;
this rebuild includes the new timeout-ms parameter.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
* refactor: Load WASM channels from disk instead of bundling in binary
Remove include_bytes! embedding of telegram.wasm. Channels are now
loaded from their build output directories (channels-src/<name>/target/)
during onboarding, then from ~/.ironclaw/channels/ at runtime.
- bundled.rs: locate_channel_artifacts() finds WASM + capabilities from
build output; IRONCLAW_CHANNELS_SRC env var overrides the default path
- available_channel_names(): only lists channels with build artifacts
- bundled_channel_names(): lists all known channels (manifest)
- Setup wizard uses available_channel_names() to offer installable channels
- Add *.wasm to .gitignore, remove tracked telegram.wasm
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
* fix: Persist gateway auth token, fix thread hydration race, polish auth screen
Three web gateway UX fixes:
1. Token persistence: Store auth token in sessionStorage so refreshing
the page doesn't force re-authentication. Hide the auth screen
immediately when a saved token exists to prevent flash.
2. Thread hydration: Remove the !msgs.is_empty() bail-out in
maybe_hydrate_thread so that even brand-new (empty) assistant threads
get hydrated with their correct DB UUID. Previously resolve_thread
would mint a fresh UUID, causing messages to land in the wrong
conversation and duplicate threads to appear.
3. Auth screen: Redesign as a centered card with brand, tagline, labeled
input, and hint text.
Also adds 34 new tests covering session/thread lifecycle, thread
resolution isolation (user, channel, external ID), hydration edge cases,
serialization round-trips, approval flows, and stale mapping recovery.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
* fix: Use bindgen! for WASM tool wrapper, add dev tool loading
Three changes:
1. Rewrite src/tools/wasm/wrapper.rs to use wasmtime::component::bindgen!
instead of manual linker.root().func_wrap(). This fixes the
"component imports instance 'near:agent/host', but a matching
implementation was not found in the linker" error. All 6 host functions
(log, now-millis, workspace-read, http-request, secret-exists,
tool-invoke) are now properly registered under the near:agent/host
namespace. Also adds WASI support, credential injection, and leak
detection for HTTP requests made by WASM tools.
2. Add dev tool loading to src/tools/wasm/loader.rs. During startup, the
loader now also scans tools-src/*/target/wasm32-wasip2/release/ for
build artifacts that are newer than installed copies. This means during
development you just rebuild the WASM and restart the host; no manual
copy step needed. Set IRONCLAW_TOOLS_SRC to override the source dir.
3. Wire up load_dev_tools() in main.rs alongside the existing
load_from_dir() call.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
* feat: Wire main startup and CLI to use DB-backed settings
main.rs now reloads Config from the database after connecting,
attaches the store to the session manager for dual-write tokens,
and loads MCP servers from DB instead of disk. ExtensionManager
and MCP CLI commands use DB when available with disk fallback.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
---------
Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>1 parent 6ed6ffc commit 7a9496b
File tree
99 files changed
+17271
-1338
lines changed- channels-src
- slack/src
- telegram
- src
- whatsapp/src
- examples
- migrations
- src
- agent
- channels
- wasm
- web
- static
- cli
- extensions
- history
- llm
- orchestrator
- sandbox
- setup
- tools
- builder
- builtin
- mcp
- wasm
- worker
- workspace
- tests
- tools-src
- gmail/src
- google-calendar/src
- google-docs/src
- google-drive/src
- google-sheets/src
- google-slides/src
- slack/src
- telegram/src
- wit
Some content is hidden
Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.
99 files changed
+17271
-1338
lines changed| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
| 1 | + | |
| 2 | + | |
| 3 | + | |
| 4 | + | |
| 5 | + | |
| 6 | + | |
| 7 | + | |
| 8 | + | |
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
4 | 4 | | |
5 | 5 | | |
6 | 6 | | |
| 7 | + | |
| 8 | + | |
| 9 | + | |
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
11 | 11 | | |
12 | 12 | | |
13 | 13 | | |
14 | | - | |
| 14 | + | |
15 | 15 | | |
| 16 | + | |
| 17 | + | |
| 18 | + | |
| 19 | + | |
| 20 | + | |
16 | 21 | | |
17 | 22 | | |
18 | 23 | | |
| |||
59 | 64 | | |
60 | 65 | | |
61 | 66 | | |
62 | | - | |
| 67 | + | |
| 68 | + | |
| 69 | + | |
63 | 70 | | |
64 | 71 | | |
65 | 72 | | |
| |||
72 | 79 | | |
73 | 80 | | |
74 | 81 | | |
75 | | - | |
76 | | - | |
| 82 | + | |
| 83 | + | |
| 84 | + | |
| 85 | + | |
| 86 | + | |
| 87 | + | |
| 88 | + | |
| 89 | + | |
| 90 | + | |
| 91 | + | |
| 92 | + | |
| 93 | + | |
| 94 | + | |
| 95 | + | |
| 96 | + | |
| 97 | + | |
| 98 | + | |
| 99 | + | |
| 100 | + | |
| 101 | + | |
| 102 | + | |
| 103 | + | |
| 104 | + | |
| 105 | + | |
| 106 | + | |
| 107 | + | |
| 108 | + | |
77 | 109 | | |
78 | 110 | | |
79 | 111 | | |
| |||
96 | 128 | | |
97 | 129 | | |
98 | 130 | | |
| 131 | + | |
| 132 | + | |
| 133 | + | |
99 | 134 | | |
100 | 135 | | |
101 | 136 | | |
| |||
236 | 271 | | |
237 | 272 | | |
238 | 273 | | |
| 274 | + | |
| 275 | + | |
| 276 | + | |
| 277 | + | |
| 278 | + | |
| 279 | + | |
| 280 | + | |
| 281 | + | |
| 282 | + | |
| 283 | + | |
| 284 | + | |
| 285 | + | |
| 286 | + | |
| 287 | + | |
| 288 | + | |
| 289 | + | |
| 290 | + | |
| 291 | + | |
| 292 | + | |
| 293 | + | |
| 294 | + | |
| 295 | + | |
| 296 | + | |
| 297 | + | |
239 | 298 | | |
240 | 299 | | |
241 | 300 | | |
| |||
297 | 356 | | |
298 | 357 | | |
299 | 358 | | |
300 | | - | |
301 | | - | |
302 | | - | |
303 | | - | |
304 | | - | |
305 | | - | |
306 | | - | |
| 359 | + | |
| 360 | + | |
| 361 | + | |
| 362 | + | |
| 363 | + | |
| 364 | + | |
| 365 | + | |
| 366 | + | |
307 | 367 | | |
308 | 368 | | |
309 | 369 | | |
| |||
320 | 380 | | |
321 | 381 | | |
322 | 382 | | |
| 383 | + | |
| 384 | + | |
| 385 | + | |
| 386 | + | |
| 387 | + | |
| 388 | + | |
| 389 | + | |
323 | 390 | | |
324 | 391 | | |
325 | 392 | | |
| |||
Some generated files are not rendered by default. Learn more about customizing how changed files appear on GitHub.
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
58 | 58 | | |
59 | 59 | | |
60 | 60 | | |
| 61 | + | |
| 62 | + | |
| 63 | + | |
61 | 64 | | |
62 | 65 | | |
63 | 66 | | |
| |||
99 | 102 | | |
100 | 103 | | |
101 | 104 | | |
| 105 | + | |
102 | 106 | | |
103 | 107 | | |
104 | 108 | | |
| |||
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
| 1 | + | |
| 2 | + | |
| 3 | + | |
| 4 | + | |
| 5 | + | |
| 6 | + | |
| 7 | + | |
| 8 | + | |
| 9 | + | |
| 10 | + | |
| 11 | + | |
| 12 | + | |
| 13 | + | |
| 14 | + | |
| 15 | + | |
| 16 | + | |
| 17 | + | |
| 18 | + | |
| 19 | + | |
| 20 | + | |
| 21 | + | |
| 22 | + | |
| 23 | + | |
| 24 | + | |
| 25 | + | |
| 26 | + | |
| 27 | + | |
| 28 | + | |
| 29 | + | |
| 30 | + | |
| 31 | + | |
| 32 | + | |
| 33 | + | |
| 34 | + | |
| 35 | + | |
| 36 | + | |
| 37 | + | |
| 38 | + | |
| 39 | + | |
| 40 | + | |
| 41 | + | |
| 42 | + | |
| 43 | + | |
| 44 | + | |
| 45 | + | |
| 46 | + | |
| 47 | + | |
| 48 | + | |
| 49 | + | |
| 50 | + | |
| 51 | + | |
| 52 | + | |
| 53 | + | |
| 54 | + | |
| 55 | + | |
| 56 | + | |
| 57 | + | |
| 58 | + | |
| 59 | + | |
| 60 | + | |
| 61 | + | |
| 62 | + | |
| 63 | + | |
0 commit comments