feat(sandbox): remote & multi-backend sandbox support (Vercel, Daytona, Firecracker)

[penso]

Summary

Add support for remote and multi-backend sandboxes, enabling sandboxed command execution on cloud deployments where Docker-in-Docker is unavailable (DigitalOcean 1-click, Fly.io, Render).

Inspired by sandcastle's provider architecture (Docker, Podman, Vercel Firecracker, Daytona), this PR extends moltis's sandbox system with:

• Multi-backend router: Multiple sandbox backends can be configured simultaneously, with per-session routing
• Workspace sync: Tar-based file transfer for isolated backends that don't have host filesystem access
• Vercel Sandbox backend: Firecracker microVMs via the Vercel API
• Daytona backend: Open-source cloud sandboxes via the Daytona API
• Local Firecracker backend: VM-based isolation on Linux without Docker
• Web UI: Settings page for configuring remote backends with API key inputs

Commits

1. Multi-backend router — Refactor SandboxRouter from single-backend to HashMap<String, Arc<dyn Sandbox>> registry with per-session backend_overrides, resolve_backend(), register_backend(), available_backends(). Session metadata column + API + persistence.

2. Workspace sync — sync.rs module with tar-based sync_in()/sync_out() for isolated backends. Integrated into exec flow (sync-in on first run) and cleanup (sync-out before destroy).

3. Vercel Sandbox — Full Sandbox trait impl using Vercel REST API: create sandbox, NDJSON command streaming, gzipped tar file writes, raw file reads, stop. Feature-gated vercel-sandbox (default-on). Auto-detected from VERCEL_TOKEN.

4. Daytona Sandbox — Full Sandbox trait impl using Daytona REST API: workspace creation, toolbox exec, multipart file upload/download, delete. Auto-detected from DAYTONA_API_KEY.

5. Local Firecracker — SSH-based exec over TAP networking, Firecracker REST API over Unix socket for VM lifecycle, copy-on-write rootfs, automatic /30 subnet allocation. Linux-only.

6. Web UI settings — RemoteBackendsSection in Sandbox settings page with Vercel and Daytona configuration cards. SandboxBackendId typed enum on both Rust and TypeScript sides.

Validation

Completed

• cargo fmt --all -- --check
• cargo clippy -p moltis-tools -p moltis-sessions -p moltis-gateway --lib -- -D warnings
• cargo test -p moltis-tools -p moltis-sessions -p moltis-gateway --lib (1382 tests, 0 failures)
• biome check --write on changed TS files
• New multi-backend router tests (7 tests)
• Sync module tests (5 tests: tar roundtrip, empty dir handling)
• Vercel backend tests (4 tests: identity, config, tar roundtrip, error handling)
• Daytona backend tests (3 tests: identity, config, error handling)
• Firecracker backend tests (4 tests: identity, config, subnet allocation, error handling)

Remaining

• ./scripts/local-validate.sh (full CI validation)
• Manual QA with live Vercel token
• Manual QA with live Daytona API key
• E2E tests for remote backend settings UI

Manual QA

1. Vercel: Set VERCEL_TOKEN, configure backend = "vercel" or leave auto on a cloud deploy → verify sandbox creates, exec works, cleanup stops VM
2. Daytona: Set DAYTONA_API_KEY, configure backend = "daytona" → verify sandbox lifecycle
3. Multi-backend: Configure both Vercel and Daytona, verify per-session backend switching via sandboxBackend in session patch
4. Web UI: Navigate to Settings → Sandboxes → verify Remote Backends section renders with Vercel/Daytona cards, save credentials, see "configured" badge
5. Auto-detection: On a cloud deploy without Docker, set VERCEL_TOKEN → verify auto-detection picks Vercel backend

[greptile-apps]

Greptile Summary

This PR introduces multi-backend sandbox support (Vercel Firecracker, Daytona, local Firecracker) with a SandboxRouter that routes per-session to named backends, tar-based workspace sync for isolated backends, and a settings UI for credential management. Most of the critical issues surfaced in prior review rounds have been addressed — register_backend is now called at startup, floor_char_boundary is used for output truncation, Firecracker's boot_vm kills the child on all error paths, and the Daytona shell injection is replaced with a base64/eval wrapper.

Remaining concerns are P2: the Box::leak in DaytonaSandbox::workspace_dir still accumulates one allocation per distinct workspace path seen over the process lifetime; the web API's PUT /api/sandbox/remote-backends silently no-ops on unknown backend values and stores tokens in plaintext TOML; and the Vercel settings card allows submitting metadata-only saves (project ID / team ID) without a token, giving misleading feedback to the user.

Confidence Score: 4/5

Safe to merge with awareness of the remaining P2 issues; all previously identified P0/P1 bugs have been resolved in this revision.

All P0/P1 findings from prior review rounds (register_backend never called, truncate panic, subnet overflow, wrong ip= format, write lock held over async, boot_vm process leak, shell injections, etc.) have been fixed. The remaining open issues are P2: Box::leak accumulation in Daytona, silent no-op on unknown backend in the API, plaintext token storage, and misleading partial Vercel save UX. None of these block functionality.

crates/tools/src/sandbox/daytona.rs (Box::leak workspace path accumulation), crates/web/src/api.rs (silent no-op on unknown backend, plaintext credentials).

Security Review

• Plaintext credential storage (crates/web/src/api.rs): api_set_remote_backend_handler persists vercel_token and daytona_api_key to TOML config in plaintext. The GET endpoint correctly omits the raw tokens, but the on-disk file is unencrypted — risk if the config is backed up, synced, or world-readable.
• Prior issues resolved: Shell injection in Daytona write_file mkdir path (now uses replace('\'', "'\\''")), Firecracker unquoted cd path (now single-quoted with escaping), Daytona subshell ) injection (replaced with base64/eval wrapper).

Important Files Changed

Filename	Overview
crates/tools/src/sandbox/vercel.rs	New Vercel Sandbox backend: correct floor_char_boundary truncation, proper workspace_dir override, isolated flag set; log streaming and gzipped tar upload look correct.
crates/tools/src/sandbox/daytona.rs	New Daytona backend: base64 wrapper correctly separates stderr; however LAST_DAYTONA_WORKSPACE Box::leak accumulates one leaked allocation per distinct workspace path, and workspace_dir() is globally shared across all sessions.
crates/tools/src/sandbox/firecracker.rs	New local Firecracker backend: subnet overflow guard, corrected kernel ip= format, process killed on all boot_vm error paths, write lock dropped before async cleanup — prior critical issues addressed.
crates/tools/src/sandbox/sync.rs	Workspace sync module: resolve_sync_workspace now has a non-home-persistence fallback path; sync_in/sync_out use per-backend workspace_dir() so each backend's correct path is used.
crates/tools/src/sandbox/router.rs	SandboxRouter refactored to HashMap of backends with per-session backend_overrides; mark_synced/is_synced/clear_synced lifecycle correct; resolve_backend returns Arc clone safely.
crates/gateway/src/server/prepare_core/sandbox.rs	New sandbox startup helper: build_sandbox_router now registers vercel, daytona, and firecracker backends into the router, fixing the multi-backend routing that was previously broken.
crates/web/src/api.rs	New remote-backends API endpoints: GET correctly hides tokens, but PUT silently no-ops on unknown backend names, and stores credentials in plaintext TOML.
crates/web/ui/src/pages/ImagesPage.tsx	RemoteBackendsSection UI added: Save Vercel button can be submitted with only metadata fields (no token), yielding a misleading no-op; Daytona and Vercel cards otherwise render correctly.
crates/tools/src/exec.rs	Sync-in integrated into first-run path: resolve_backend used correctly per session, concurrent waiters unblocked via mark_synced (intentionally unconditional per the comment), overall flow correct.

Sequence Diagram

sequenceDiagram
    participant Client
    participant ExecTool
    participant SandboxRouter
    participant Backend
    participant SyncModule

    Client->>ExecTool: exec(command, session_key)
    ExecTool->>SandboxRouter: resolve_backend(sk)
    SandboxRouter-->>ExecTool: Arc<dyn Sandbox>
    ExecTool->>SandboxRouter: mark_preparing_once(sk)
    alt First caller
        ExecTool->>Backend: ensure_ready(id, image)
        Backend-->>ExecTool: VM/sandbox ready
        ExecTool->>SyncModule: sync_in(backend, id, host_workspace, workspace_dir)
        SyncModule->>Backend: write_file(tar.gz)
        SyncModule->>Backend: exec(tar -xzf ...)
        ExecTool->>SandboxRouter: mark_synced(sk)
    else Concurrent caller
        loop poll until synced or 120s timeout
            ExecTool->>SandboxRouter: is_synced(sk)
        end
    end
    ExecTool->>Backend: exec(id, command, opts)
    Backend-->>ExecTool: ExecResult
    Note over SandboxRouter: cleanup_session(sk)
    SandboxRouter->>SyncModule: sync_out(backend, id, host_workspace, workspace_dir)
    SandboxRouter->>Backend: cleanup(id)

Loading

_{Reviews (12): Last reviewed commit: "Merge remote-tracking branch 'origin/mai..." | Re-trigger Greptile}

[codspeed-hq]

Merging this PR will not alter performance

✅ 39 untouched benchmarks
⏩ 5 skipped benchmarks¹

---

_{Comparing flaxen-cemetery (41567c3) with main (4cb2751)}

Footnotes

1. 5 benchmarks were skipped, so the baseline results were used instead. If they were deleted from the codebase, click here and archive them to remove them from the performance reports. ↩

[penso]

@greptile review

[penso]

@greptile review

[penso]

@greptile review

[penso]

@greptile review