"Because we have LiteLLM at home"

Multi-Provider AI Gateway

Important

AI-Assisted Project Disclaimer:
Human judgment applied at every stage, particularly around architectural decisions, UX flows, and quality control.

<sub>Localised by AI - expect mistakes - translation fixes welcome as PRs against web/src/i18n/locales/!
Made with OpenCode + oh-my-opencode-slim & Claude Code


Poke around a real instance at mh.site19.ddns.net - rebuilds fresh every 30 minutes</sub>

A single OpenAI-compatible endpoint that sits in front of all your LLM providers. Models are auto-discovered the moment you add a provider and optionally on schedule; failover groups form automatically around shared model names and retry transparently when a provider goes down; no prompt data is ever stored.

_{Model Hotel Dashboard}

Quick Start

git clone https://github.com/hugalafutro/model-hotel.git
cd model-hotel

cp .env.example .env
nano .env          # set a strong MASTER_KEY and POSTGRES_PASSWORD

docker compose up --build -d

For local development, layer the compose.dev.yml override instead. It mounts the Docker socket, turns on DEBUG_LOG, and allows embedding, so use it only in a trusted environment:

# Development only:
docker compose -f docker-compose.yml -f compose.dev.yml up --build -d

To run a prebuilt image instead of building from source, edit docker-compose.yml: comment out the build: block and uncomment one of the image: lines.

On first run the admin token is printed once, in a boxed ADMIN TOKEN block in the startup banner, and never shown again:

docker compose logs app

If you lose it, delete .data/admin-token and restart to generate a new one. The ADMIN_TOKEN environment variable seeds the token on first boot only: once .data/admin-token exists the file wins and the variable is ignored.

Open http://localhost:8081, log in with that token, add your first provider, and start proxying.

High Availability

Run several instances behind one client endpoint with no client-side change: a Front Desk control plane manages the fleet and replicates config to every member, while Traefik load-balances them with health checks and automatic failover. Members share one MASTER_KEY (so encrypted provider keys port across the fleet) and each keeps its own admin token.

_{Front Desk (HA control app) Dashboard}

Full deployment in the High Availability wiki.

Bellhop Companion App

Bellhop, the native Android companion app for Front Desk, turns a paired phone into a pocket view of the fleet: live member health, request traffic, provider quota badges, the event log, and, for operator devices, one-tap drain, activate, and config-sync behind a biometric prompt. A home-screen widget keeps the fleet and its badges on the launcher without opening anything. It talks only to Front Desk, holds no provider credentials, and authenticates with a device token you can revoke from either side.

_{Bellhop (Android HA companion) Dashboard - Fleet member details}

_{Bellhop Android Home screen widget}

Note

Full walkthrough in the Bellhop wiki; source under android/.
APK download: (signed; Obtainium-compatible).

One Endpoint, Many Providers

Pick a provider family when you add a provider: Azure AI Foundry, DeepSeek, Kimi Code, KoboldCPP, LMStudio, MiniMax, NanoGPT, NeuralWatt, OpenRouter, Z.AI, x.ai, Google AI Studio, Vertex AI (express keys), Cohere, Ollama, Ollama Cloud, OpenCode Go, OpenCode Zen, OpenAI, or a custom OpenAI-compatible endpoint of your own. Anthropic and AWS Bedrock are native families rather than OpenAI-compatible ones, and a hand-entered endpoint that speaks Anthropic's native /v1/messages has its own type (anthropic-messages). All of them are called through the same /v1/chat/completions endpoint. The proxy handles model ID mapping and failover transparently. Provider API keys are encrypted with AES-256-GCM at rest using your MASTER_KEY; only the proxy ever sees the decrypted credentials. Keyless providers (e.g. OpenCode Zen free models, local Ollama) are also supported (no API key required). Self-hosted servers (Ollama, LM Studio, KoboldCPP) run wherever you put them: pick the type, enter the address and port, and Model Hotel checks that the server really is that type before saving it, telling you what answered if it is not.

_{Provider management screen overview}

Transparent Failover

Requests that fail (server errors, rate limits, auth issues, request timeouts, and TTFT probe timeouts) are automatically retried on the next available provider. For streaming requests, a TTFT probe reads ahead to confirm the first token arrives before committing the stream to your client; if the provider fails to produce a token within the configured timeout (default 60s), the request fails over to the next provider. Once streaming begins, a stall watchdog monitors for silence: if no data arrives within the configured window (default 30s), the connection is terminated and the circuit breaker records a failure. After 50 chunks the stall threshold is multiplied by 3 to tolerate tool-call pauses and long reasoning chains. Both timeouts are configurable in Settings → Proxy (set to 0s to disable). Retries are paced with exponential backoff and jitter to avoid overloading failing providers.

Streaming requests to a failover group can also be hedged (off by default, enabled under Settings → Circuit Breaker & Failover → Hedging): instead of trying group members strictly in sequence, the first candidate is launched immediately and every hedge_delay without a first token (default 4s) launches the next one in parallel, with the first provider to confirm a first token winning and the rest cancelled. That trades duplicate upstream load on slow starts for lower tail latency, so it uses provider rate limits and capacity faster; turn it on only if slow first tokens hurt more than the extra load.

_{Failover groups management}

Hotel Routing

Prefix a model with hotel/ to use its failover group. hotel/glm-4.6 resolves to every provider offering glm-4.6, tried in priority order. Groups form automatically when 2+ providers share a model name (auto-created groups show an "auto" badge and are deleted when they drop below 2 providers). Manually created groups persist regardless of provider count. Individual entries can be toggled on/off, priorities are preserved across syncs, and stale entries are pruned when a model is deleted from a provider or leaves the provider's listing (discovery re-syncs the affected groups automatically). The UI shows each entry's effective state: entries whose model or provider is disabled are greyed out with a badge, since the router skips them regardless of the entry toggle. A manual sync can be triggered from the dashboard or via POST /api/failover-groups/sync.

Provider health is tracked with a circuit breaker, keyed per (provider, model) rather than per provider: after a configurable number of consecutive failures (default 5) that one model's circuit moves to Open and requests for that model skip that provider. The provider as a whole is only skipped once enough distinct model circuits are open (default 2, the "span" setting), so one broken model never condemns a healthy provider.

After a cooldown period (default 60s), a single HalfOpen probe is allowed; if it succeeds the circuit closes. If it fails, the circuit re-opens with the cooldown doubled for every probe that has failed since it last closed, up to a ceiling (default 15 minutes; set the backoff limit to 0 to switch the doubling off). A circuit blocked by an exhausted provider quota is instead pinned until the quota window resets, up to a separate quota pin limit (default 24 hours, 0 to switch pinning off). State transitions are broadcast as SSE events, and the breaker can be disabled entirely in Settings. See Failover and Hotel Routing for the full breakdown.

Per-Client Virtual Keys

Issue separate API keys for different users or services. Each key is SHA-256 hashed before storage, so raw keys are never persisted. Track token usage per key, set per-key rate limits (requests/sec and burst) plus an optional tokens-per-minute (TPM) cap, give a key a dollar budget per day, week or month (requests are refused with 429 once the period's spend reaches it, and the key shows how much of it is used), restrict which providers a key may reach, delete a key to immediately cut off access, and never expose your real provider credentials. Keys can be created and deleted from the dashboard or the admin API.

_{Virtual keys management}

No Prompts Logged

Note

User Prompts and request content are never captured, logged, or inspected. The proxy forwards requests to the provider exactly as received, without reading or modifying message contents.

The only information recorded is what is strictly necessary to route and meter the request: timestamp, duration, latency, time-to-first-token (TTFT, measured during the streaming probe), token counts (including cache-hit/miss breakdown), tokens per second, HTTP status code, error messages (upstream provider failures only, never user content), proxy overhead breakdown (parse, model lookup, provider lookup, key decryption), streaming flag, failover attempt count, resolved model ID (the actual upstream model used, which may differ from the requested hotel/ name), request state, virtual key identifier, and target provider/model identifiers.

The optional Arena History feature (disabled by default, toggled under Settings → Data Storage → Arena History) can persist completed arena and compare session results in your browser's local storage. When enabled:

• Model-generated responses (output text, thinking blocks, metrics) are stored locally so you can review past results.
• Preset prompts and personas are saved by reference (e.g. "Dilemma preset", "Merlin persona"), storing only their built-in IDs, never the text content you didn't write yourself.
• Custom user-entered text is never logged. If you type your own prompt or persona system prompt, it is intentionally excluded from history records. Only the fact that a custom prompt was used is recorded (shown as "Custom prompt" in the history UI), with no content retained.

History data never leaves your browser. It can be cleared at any time from the Settings page.

Request Logging with Overhead Breakdown

Every request is logged with full latency decomposition:

• TTFT (time to first token, measured by the streaming probe)
• Total duration (end-to-end wall time)
• Proxy overhead split into request parsing, model/failover lookup, provider lookup, and key decryption
• Tokens per second, prompt / completion counts
• Cost in dollars, priced from the serving model's per-token prices (cache-hit tokens at the cache-hit price where the model has one)

_{Overview of requests served with proxy overhead breakdown}

Streaming requests are captured as they start and updated as they finish, so you can see in-flight requests in the Logs view. The overhead breakdown helps you determine whether latency is coming from your provider or from the proxy itself.

The Dashboard reads the same prices: its header toggles between tokens, requests and dollars (T / R / $), and in the $ state the spend tile, the spend chart and the per-provider, per-model and per-key panels all show what the period cost. A model with no known prices meters at zero; hovering the spend tile shows how many served requests went unpriced.

_{Dashboard in the $ state: spend per week, per provider, per model and per key}

Built-In Model Discovery

Add a provider and the service pulls the model list automatically via the provider's own API. Models are kept in sync on a schedule you control (default every 6 hours, configurable). Models that disappear from a provider's listing are disabled (never deleted) and come back automatically if the provider lists them again; manual disables are always respected. After a manual scan, a summary modal shows exactly what changed: models added, re-enabled, or disabled, any live pricing or context-length changes on existing models, plus any failover groups that were updated or deleted as a result. Changes detected by scheduled/startup background discovery instead surface as a count badge on the Models nav item; clicking the badge opens a summary of those changes and clears it. Discovery-disabled models carry a "not listed by the provider since…" tooltip on the Models page so they're easy to tell apart from manual disables. The following providers get enriched metadata beyond what the generic OpenAI-compatible endpoint returns:

_{Models overview}

Provider	Context Length	Pricing	Reasoning Flags	Input/Output Modalities	Source
DeepSeek	✅	✅	✅	✅ (text, plus image on the vision model)	API (/models) + Catalog
NanoGPT	✅	✅	✅	✅	API (/models?detailed=true)
Z.AI	✅	✅ (partial)	✅	Derived	API (/models) + Catalog
OpenCode Go	✅	✅	✅	✅	API (/models) + Catalog
OpenCode Zen	✅	✅	✅	✅	API (/models) + Catalog
OpenAI	✅	✅	✅	✅	API (/models) + Catalog
OpenRouter	✅	✅	✅	✅	API (/models)
Anthropic	✅	✅	(none)	✅ (partial)	API + Pricing catalog
xAI (Grok)	✅	✅	✅	✅	API (/language-models) + Catalog
Kimi Code	✅	(none)	✅	✅	API (/models)
Google AI Studio (Gemini)	✅	✅	✅	✅	API (/v1beta/models) + Pricing catalog
Cohere	✅	✅	✅	✅ (vision)	API (/v1/models, paginated) + Pricing catalog
Ollama / Ollama Cloud	✅	(none)	✅	✅	API (/api/show)

Z.AI, xAI, OpenAI, DeepSeek, and OpenCode (Go & Zen) combine a live /models listing with a built-in catalog: the API supplies the authoritative model list (plus live pricing and modalities for xAI) and the catalog backfills the fields the API leaves out (context window, max output, capability flags, pricing). For Z.AI, xAI, and OpenCode the catalog also surfaces models the listing doesn't advertise but that still work - a freshly released GLM the listing hasn't caught up to, or older Grok models xAI keeps callable without listing them. Live values always win; the catalog only fills gaps. xAI (on 403/429) and OpenCode Go (on 404) fall back to the pure catalog when the account or endpoint can't list; the others abort the scan on error so a transient failure never disables existing models. Google AI Studio provides rich metadata (context, thinking support) from its native API, supplemented with a pricing catalog. Cohere uses its native API with full pagination for model discovery, enriched with a pricing catalog for cost data, capability detection (tool calling, vision, structured output, reasoning), and modality mapping. NanoGPT and Anthropic expose richer model metadata through their own APIs; Anthropic additionally uses a pricing catalog for per-model cost data. Ollama and Ollama Cloud enrich models via the /api/show endpoint.

Models that aren't covered by any built-in catalog are automatically enriched from models.dev, an open-source model catalogue that provides pricing, context limits, capabilities, and modality data for 40+ providers. The enrichment is non-destructive: it only fills fields that are empty or missing, never overwriting data that was already populated. This makes the full precedence per field live provider data → built-in catalog → models.dev → empty: you get the freshest values the provider reports, the catalog and models.dev only fill what's missing, and a stale catalog can never mask fresh live data. If models.dev is unreachable, discovery proceeds normally using whatever data the provider returned, so your existing catalogue is never at risk.

Every discovered model carries three classification fields with closed vocabularies. input_modalities lists what the model accepts (text, image, audio, video, pdf); output_modalities lists what it produces (text, image, audio, video, plus embedding and rerank for those endpoint families); and modality is an endpoint class derived from the arrays (chat, embedding, rerank, image, video, tts, or stt). The class is never hand-set per provider: one central deriver computes it after enrichment, so a vision chat model ("understands images") can't be confused with an image-generation model ("produces images"), and non-chat models are reliably kept out of the chat and arena pickers while remaining visible on /v1/models and in failover groups.

Model Health at a Glance

Test any model from the Models page with a single click. The test sends a minimal chat completion directly to the provider and reports total duration and the actual model response, so you know the provider is alive and responsive. DeepSeek providers show live account balance; NanoGPT, Z.AI, Kimi Code, MiniMax and OpenCode Go providers show quota and usage data; NeuralWatt providers show energy quota and credit balance (Standard plan or higher). All fetched from their respective APIs and displayed on both the provider cards and the sidebar quota panel.

Provider Quotas & Usage

For providers that expose it, click a provider's quota badge (on its card or in the sidebar panel) to open a live usage breakdown - no need to leave the dashboard for the provider's billing page. OpenRouter shows credit balance and per-key spend; Z.ai Coding Plan shows its 5-hour, weekly, and MCP token quotas; Kimi Code shows its 5-hour and weekly quotas plus parallel-request limit and membership tier; MiniMax shows its 5-hour and weekly Token Plan quotas by model class; NanoGPT shows weekly token and daily image quotas with subscription details; OpenCode Go shows its rolling 5-hour, weekly and monthly plan quotas with their reset times; NeuralWatt shows energy-based quota with subscription and lifetime usage. Each modal toggles between quota used and quota remaining, and refreshes on demand. Some providers surface usage without a dedicated modal - DeepSeek shows account balance and Ollama Cloud shows plan status on their cards and sidebar badges.

        
_{Quota limits/balance/spend modals for supported providers}

Themeable UI

Make the dashboard your own from the Appearance settings. Pick one of three UI styles: Clean SaaS (refined and minimal, the default), Cyber Terminal (high-contrast, developer-centric), or Glassmorphism (slick translucent surfaces). Then toggle dark / light mode, and choose an accent color (each style ships a tasteful default, or pick your own). Everything persists locally in the browser. The animated dashboard at the top of this page cycles through all three.

   
_{Available themes with their default color accents}

Interactive Chat & Arena

The dashboard includes a built-in Chat interface for testing models interactively, with support for system personas (presets or custom prompts), generation parameters (temperature, top_p, max_tokens, min_p, top_k, frequency/presence penalties), and streaming responses with collapsible thinking-block rendering. Vision-capable models show an image upload button: attach a photo for the model to describe or analyze. Audio-capable models show an audio upload button for sending audio input. Attachments are sent as OpenAI-compatible multimodal content parts (image_url, input_audio). Switch to Conversation mode to watch two models talk to each other: enter a starter prompt, set the number of rounds and optional delay between turns, and observe the back-and-forth with per-message metrics (duration, tokens, chars/sec).

_{Test conversational capabilities of models served by the proxy}

Arena mode offers two sub-modes: Competition runs bracket tournaments where models face off in pairwise matchups. Vote for winners, and the bracket auto-advances to the next round until a champion emerges. Compare places two or more models in a grid with the same prompt for parallel evaluation, with per-slot personas and voting. Both modes support per-model generation parameters, streaming with thinking-block rendering, and per-response metrics. Past sessions are saved to an arena history modal for review and restoration.

_{Compare outputs of models served by the proxy}

Real-Time Events & System Status

A live SSE event bus delivers toast notifications for discovery outcomes, model disabling events, token counting errors, circuit breaker state transitions, and stale-request alerts straight to the dashboard. Failover retries during proxying are logged but not pushed as SSE events. The sidebar polls system stats every 10 seconds, showing CPU, memory, disk I/O, and network throughput with color-coded warnings (orange at 75%, red at 90%). When running under Docker Compose, stats are aggregated across containers; otherwise, cgroup metrics are used. Goroutine count, database health (size, connections, cache hit ratio), API uptime, and process count are also displayed.

_{Options and settings}

Security & Privacy

Provider API keys are encrypted at rest with AES-256-GCM. The MASTER_KEY is strengthened via Argon2id key derivation (with per-provider random salts) before use as the AES key. Virtual keys are SHA-256 hashed. The admin token is SHA-256 hashed before storage: the plaintext token is displayed once on first run and never stored on disk. To regenerate a lost token, delete the admin-token file in your configured DATA_DIR and restart. Outbound connections to providers are protected against SSRF and DNS rebinding attacks: the proxy resolves hostnames and blocks connections to private, loopback, link-local, and cloud-metadata IP addresses, then dials by IP (not hostname) to close the DNS-rebinding TOCTOU gap. Redirect targets are also validated. Use KNOWN_PROXIES to allow specific private CIDR ranges for internal LLM servers, and ALLOWED_PROVIDER_HOSTS to allow specific hostnames. Standard security headers (X-Content-Type-Options, X-Frame-Options, Referrer-Policy, Strict-Transport-Security (when TLS is active), Content-Security-Policy) are applied to all responses. Decrypted provider keys are cached in memory for up to 10 minutes (configurable via the key_cache_ttl setting) to avoid repeated key derivation overhead. WebAuthn session tokens are SHA-256 hashed and never stored in plaintext, with a 3-day idle TTL and a 30-day absolute cap. For blocking abusive clients at the edge, the repository ships CrowdSec parsers and scenarios under contrib/crowdsec/ that read the gateway's container logs and hand repeated authentication failures and rate-limit abuse to a bouncer; see the CrowdSec wiki page.

Passkey Authentication

Log into the admin dashboard using a FIDO2/WebAuthn passkey (Touch ID, Windows Hello, YubiKey, etc.) instead of the admin token. Register passkeys from the Settings page and use them on the login screen alongside the traditional admin token.

_{Various login mechanisms supported}

Passkey login is disabled by default. Enable it by setting WEBAUTHN_RP_ID (your domain) in the environment; WEBAUTHN_RP_ORIGINS (your origin URLs) falls back to CORS_ORIGINS, then to http://localhost:<port>. Session tokens are SHA-256 hashed, never stored in plaintext, and expire after 3 days without use; each use slides that window forward, up to an absolute cap of 30 days from login.

Authenticator App (TOTP)

Add a time-based one-time password (TOTP, RFC 6238) from an authenticator app (Google Authenticator, Authy, 1Password, etc.) as a true second factor on the admin login. Enable it from the Settings page: scan the QR code with your app, enter the 6-digit code it shows, then save the one-time recovery codes you are shown.

When TOTP is enabled the raw admin token no longer authenticates API requests on its own. It becomes a first factor that, combined with a valid 6-digit code, is exchanged for a session token on the login screen (the same session infrastructure passkeys use). Only that session token authorizes subsequent API calls, which closes the static-token replay that a bare bearer would otherwise allow. Disable is gated on a current TOTP or recovery code.

If you lose your authenticator, a recovery code signs you in once so you can disable or re-enroll TOTP. Recovery codes are single-use, stored as SHA-256 hashes, and displayed only at enable time (the TOTP secret itself is AES-256-GCM encrypted at rest with MASTER_KEY, like provider keys). If you lose both the authenticator and every recovery code, an operator can remove 2FA directly from the database: run make totp-disable, or run DELETE FROM admin_totp_recovery; DELETE FROM admin_totp; via psql against the stack's Postgres (both tables, or the stale recovery codes survive). TOTP is independent of passkeys and needs no environment variable: it is opt-in at runtime from Settings.

Single Sign-On (OIDC)

Let admins sign in through an external OpenID Connect provider (Authentik, Authelia, Keycloak, Pocket-ID, Okta, Google, Entra, and so on). Configure it from the Settings page: paste the issuer URL, client ID, and client secret from an app you register with your provider, then list the verified email addresses allowed to sign in. A "Sign in with SSO" button appears on the login screen. Any standards-compliant OpenID Connect provider works (the names above are just examples): the login flow uses only standard discovery, PKCE, and ID-token verification, so the single requirement is that the provider releases the signing-in user's verified email (in the ID token, or from its UserInfo endpoint), because the allowlist is email-based and fails closed.

  
_{The Authentication settings page, split into its three sections. Click any panel for the full view.}

SSO is a third login path, not a replacement: after the provider confirms an allowlisted, email-verified identity it mints the same session token as passkey and TOTP login, so nothing downstream changes. Logins are gated by the email allowlist (empty allowlist denies everyone) and matched only on verified emails, while the provider's stable sub and issuer are logged on each login (app log, source oidc). The client secret is AES-256-GCM encrypted at rest with MASTER_KEY, the flow uses PKCE plus single-use state and nonce, and the minted session rides an HttpOnly cookie set on the callback, so the token never appears in the URL or in the callback's 302 Location response header.

Because it is self-hosted, there is no turnkey "Google login": each operator registers their own OIDC app with their provider and points it at this app's redirect URI (<public base URL>/api/auth/oidc/callback, shown in Settings). The client must allow the openid, email, and profile scopes (all three are requested; a client permitting fewer fails with invalid_scope), and the Settings allowlist must hold the signing-in account's exact verified email. SSO never removes local login, so a misconfigured or unreachable provider cannot lock you out: the admin token, passkeys, and TOTP all keep working. SSO is opt-in at runtime from Settings and needs no environment variable. The Security wiki page has a copy-paste provider client example.

GitHub works the same way as a separate option. GitHub is OAuth2 only (no OpenID Connect, no ID token), so instead of verifying an ID token it reads the account's verified emails from the GitHub API and matches them against the same kind of allowlist: an unverified address never counts, and the account's stable numeric id and login are logged on each sign-in (source github). Register a GitHub OAuth App, set its Authorization callback URL to <public base URL>/api/auth/github/callback, and paste the Client ID and secret into Settings. A "Sign in with GitHub" button then appears alongside the SSO button. As with OIDC, the session is delivered over an HttpOnly cookie, and local login always keeps working.

Multi-User Access

Beyond the shared admin token, you can provision named dashboard accounts that sign in with a username and password (plus their own optional TOTP second factor) on the same login screen. Two roles: admin sees and does everything, while user accounts are scoped by granular grants (Chat/Arena, Usage dashboards, Request Logs, Models, Virtual Keys) so a teammate gets exactly the access they need and nothing more. Virtual keys belong to a user, and per-account rate limits (RPS/burst/TPM) and a per-account dollar budget aggregate across the keys that user owns.

_{User overview}

Manage accounts from the Users page (admin only): create a user, assign grants, set an initial password, reset a password or second factor, enable or disable, and read last-login and TOTP status at a glance. The username/password form appears on the login screen only once at least one user exists, so a fresh install keeps the single admin-token flow, and local token login is never removed so you cannot lock yourself out. See the Multi-User wiki page for roles, grants, and the per-user rate-limit model.

Deploy without Git

No git clone needed. Create two files and go:

1. Create .env with your secrets:

# Generate strong secrets:
#   MASTER_KEY:       openssl rand -base64 32
#   POSTGRES_PASSWORD: openssl rand -hex 16
#   ADMIN_TOKEN:      openssl rand -hex 16   (optional; auto-generated if empty)

MASTER_KEY=<your-master-key>
POSTGRES_PASSWORD=<your-postgres-password>
ADMIN_TOKEN=

# Optional: WebAuthn/FIDO2 passkey login (only WEBAUTHN_RP_ID is required)
# WEBAUTHN_RP_ID=your-domain.com
# WEBAUTHN_RP_ORIGINS=https://your-domain.com

2. Create docker-compose.yml:

docker-compose.yml (click to expand, then copy)

    name: model-hotel
    services:
        app:
            # Build from source (default):
            build:
                context: .
                args:
                    VERSION: ${VERSION:-dev}
                    COMMIT: ${COMMIT:-unknown}
            # Prebuilt images (uncomment 1 image according to registry preference, comment out build above):
            # image: ghcr.io/hugalafutro/model-hotel:latest
            # image: hugalafutro/model-hotel:latest
            labels:
                app.group: model-hotel
            ports:
                - "${HOST_PORT:-8081}:8080"
            environment:
                - MASTER_KEY=${MASTER_KEY:?MASTER_KEY must be set in .env}
                - POSTGRES_USER=${POSTGRES_USER:-modelhotel}
                - POSTGRES_PASSWORD=${POSTGRES_PASSWORD:?POSTGRES_PASSWORD must be set in .env}
                - POSTGRES_HOST=db
                - POSTGRES_DB=${POSTGRES_DB:-modelhotel}
                - ADMIN_TOKEN=${ADMIN_TOKEN:-}
                - ALLOW_HTTP_PROVIDERS=false
                - ALLOW_EMBED=false
                - DATA_DIR=/data
                - RATE_LIMIT_ENABLED=true
                - DEBUG_LOG=false
                - CORS_ORIGINS=http://localhost:5173,http://localhost:${HOST_PORT:-8081}
                - WEBAUTHN_RP_ID=${WEBAUTHN_RP_ID:-}
                - WEBAUTHN_RP_ORIGINS=${WEBAUTHN_RP_ORIGINS:-}
                - ALLOWED_PROVIDER_HOSTS=
                - TRUSTED_PROXIES=
                - KNOWN_PROXIES=
            volumes:
                - ./.data:/data
                # Docker socket (disabled by default for security).
                # Enable to show container-level stats in the sidebar (CPU, memory per container).
                # ⚠️  Granting Docker socket access allows the container to control the Docker daemon.
                #     Only enable if you trust the deployment environment.
                # - /var/run/docker.sock:/var/run/docker.sock:ro
            restart: unless-stopped
            # Model Hotel winds down in stages on SIGTERM. Worst case, in order:
            # 10s HTTP drain (open SSE tabs and proxied streams are ended first, so
            # this is usually quick) + 35s background join (the 30s ceiling of the
            # scheduled-disable sweep, which deliberately finishes the statement it
            # has already started, plus a 5s margin; the retention and stale-log
            # sweeps have no ceiling and the join cancels them instead of waiting)
            # + 10s audit drain (one record's 5s insert plus the 5s retention prune
            # it piggybacks) + 5s app-log writer stop + 5s OTLP flush = 65s. The
            # closes around them (the event bus, the proxy handler, discovery, the
            # docker client, the rate limiters and the database pool) carry no budget
            # of their own, so this is a ceiling with headroom over the 65s, not the
            # sum. Docker's default grace is 10s, which would SIGKILL partway through
            # the drain and take the audit rows and the last log lines with it.
            stop_grace_period: 75s
            depends_on:
                db:
                    condition: service_healthy
    
        db:
            image: postgres:16-alpine
            labels:
                app.group: model-hotel
            command: ["postgres", "-c", "log_min_error_statement=panic", "-c", "log_min_messages=error", "-c", "log_checkpoints=off"]
            environment:
                - POSTGRES_USER=${POSTGRES_USER:-modelhotel}
                - POSTGRES_PASSWORD=${POSTGRES_PASSWORD:?POSTGRES_PASSWORD must be set in .env}
                - POSTGRES_DB=${POSTGRES_DB:-modelhotel}
            volumes:
                - ./.data/pgdata:/var/lib/postgresql/data
            restart: unless-stopped
            healthcheck:
                test: ["CMD-SHELL", "pg_isready -U ${POSTGRES_USER:-modelhotel}"]
                interval: 5s
                timeout: 5s
                retries: 5
    
        # Optional: outbound alerting via Apprise. Uncomment to run a stateless
        # apprise-api container, then in Settings → Alerts switch alerting on and press
        # "Set up alerts": the wizard checks http://apprise:8000, builds the destination
        # URL for you (ntfy, Telegram, Discord, email, or a raw Apprise URL), tests it,
        # and saves only at Finish. The same fields sit under "Manual configuration (advanced)" if you
        # would rather paste tgram://<bot_token>/<chat_id> yourself. Model Hotel POSTs
        # event summaries here and Apprise fans them out to your service. No request
        # content is ever sent.
        # apprise:
        #     image: caronc/apprise:latest
        #     labels:
        #         app.group: model-hotel
        #     restart: unless-stopped
        #     # Not exposed to the host: only Model Hotel needs to reach it.
        #     expose:
        #         - "8000"

3. Deploy:

docker compose up --build -d

Note

The docker-compose.yml content above is the production compose (auto-synced by a GitHub Action). See Quick Start for the development override and the prebuilt-image option.

Note

WEBAUTHN_RP_ID enables FIDO2/WebAuthn passkey login (leave empty to disable); WEBAUTHN_RP_ORIGINS is optional and falls back to CORS_ORIGINS. TRUSTED_PROXIES is for trusting inbound X-Forwarded-For headers from reverse proxies (rate limiting/logging). KNOWN_PROXIES is for allowing outbound connections to internal LLM servers on private networks (bypasses SSRF protection). See Configuration for details.

API Example

# List available models
curl http://localhost:8081/v1/models \
  -H "Authorization: Bearer $VIRTUAL_KEY"

# Chat completion (with hotel routing for automatic failover)
curl -X POST http://localhost:8081/v1/chat/completions \
  -H "Authorization: Bearer $VIRTUAL_KEY" \
  -H "Content-Type: application/json" \
  -d '{"model": "hotel/glm-4.6", "messages": [{"role": "user", "content": "Hello!"}]}'

# Anthropic Messages API (point Claude Code or the anthropic SDK at the gateway;
# x-api-key is accepted alongside Authorization: Bearer)
curl -X POST http://localhost:8081/v1/messages \
  -H "x-api-key: $VIRTUAL_KEY" \
  -H "Content-Type: application/json" \
  -d '{"model": "hotel/claude-sonnet-4-6", "max_tokens": 1024, "messages": [{"role": "user", "content": "Hello!"}]}'

# Embeddings (multimodal endpoints support the same provider/model and hotel/ routing)
curl -X POST http://localhost:8081/v1/embeddings \
  -H "Authorization: Bearer $VIRTUAL_KEY" \
  -H "Content-Type: application/json" \
  -d '{"model": "OpenAI/text-embedding-3-small", "input": "Hello!"}'

# Speech-to-text (multipart upload)
curl -X POST http://localhost:8081/v1/audio/transcriptions \
  -H "Authorization: Bearer $VIRTUAL_KEY" \
  -F model="OpenAI/whisper-1" -F file=@speech.mp3

The proxy also serves /v1/rerank (Cohere-style document rerank, common in RAG stacks), /v1/images/generations, /v1/images/edits, /v1/images/variations, /v1/audio/speech, and /v1/audio/translations as transparent OpenAI-compatible pass-through (failover, circuit breaker, and virtual-key access control included; request/response content is never logged). Gemini TTS models, which Google serves through generateContent alone, are reached through the native route and answer as wav or pcm. See the API Reference for the full endpoint listing.

A native Anthropic Messages API (POST /v1/messages) lets Claude Code and the anthropic SDKs drive the gateway directly, so an Anthropic client fails over across every provider in a hotel/ group, not just Claude. Requests routed to a non-Anthropic provider are translated to and from the OpenAI shape (text, vision, tools, and tool results); requests routed to an Anthropic-family provider are forwarded natively, so extended-thinking blocks and prompt caching survive end to end. Auth accepts x-api-key (what Anthropic clients send) as well as Authorization: Bearer.

OpenAI's newest models (the gpt-5.4+ and gpt-5.6 families) reject tool calling combined with reasoning on /v1/chat/completions and demand OpenAI's Responses API instead. The gateway heals this transparently: the first such request gets the upstream 400, is retried against /v1/responses on the spot, and the requirement is remembered per model so every later tools+reasoning request routes there directly. Clients keep speaking plain Chat Completions in both directions (streaming included); reasoning summaries come back as reasoning_content, and the gateway always sends store: false so OpenAI keeps no conversation state. The pro tier (o1-pro, o3-pro, gpt-5.x-pro), which OpenAI serves over the Responses API alone, routes there from the first request on api.openai.com, and any other model that refuses the chat endpoint with OpenAI's "not a chat model" 404 is learned and re-routed the same way.

Metrics & log shipping

A Prometheus endpoint is exposed at /metrics (request rates by provider/model/status, latency and TTFT histograms, token counters, a dollar spend counter per provider and model, failover attempts per provider, upstream 429s by class, circuit-breaker opens by cause and state, failover exhaustion by reason, plus Go runtime metrics; see the wiki's Failover page for the failover series). It is authenticated - set a dedicated METRICS_TOKEN so your scrape config need not carry the admin token (the admin token also works). No prompt content is ever exposed. deploy/observability/ ships a Prometheus + Grafana compose stack with a provisioned fleet dashboard (traffic, latency, tokens, spend, breakers); see the wiki's Observability page.

# prometheus.yml
scrape_configs:
  - job_name: model-hotel
    authorization:
      credentials: "${METRICS_TOKEN}"
    static_configs:
      - targets: ["model-hotel:8080"]
        labels:
          member: mh1

For logs, set LOG_FORMAT=json to emit one structured JSON object per line on stdout for Fluent Bit / Vector / Promtail / Datadog and friends - no extra endpoint, and (like everything here) never any prompt content. To push those same structured logs to an OpenTelemetry collector, set OTEL_EXPORTER_OTLP_ENDPOINT (standard OTEL_EXPORTER_OTLP_* vars apply; http/protobuf by default, OTEL_EXPORTER_OTLP_PROTOCOL=grpc to switch) - logs only, no tracing. Need verbose debug output without the flood? DEBUG_LOG=true turns on Debug for everything; DEBUG_LOG_SCOPES=failover,resolve turns it on for just those areas. The Settings → Observability section shows which of these three exporters are active and how to enable the rest. See the Configuration wiki.

For push notifications rather than scraping, Settings → Alerts can POST short summaries of operational events (a provider going down, a circuit breaker tripping, a failover group failing to sync) to a stateless Apprise container, which fans them out to Telegram, email, Discord, Slack, Matrix, a raw webhook, and around 80 other destinations; only the event summary is sent, never request content. See the Alerting wiki.

Full Documentation

• Configuration: Environment variables, runtime settings, Docker Compose
• API Reference: Proxy and admin endpoints
• Security: AES-256-GCM encryption, Argon2id key derivation, hashing, URL validation
• Privacy: What is and isn't captured, data retention, local deployment
• Failover and Hotel Routing: Failover groups, circuit breaker, backoff
• Model Discovery: Automatic sync, provider-specific metadata, enrichment
• Virtual Keys: Creating, using, and deleting client keys
• Multi-User: Dashboard accounts, roles, grants, per-user rate limits
• Request Logging: Log fields, overhead breakdown, retention
• Alerting: Outbound event notifications via Apprise
• CrowdSec: Parsers and scenarios for banning abusive clients at the edge
• Backup & Restore: Creating backups, restoring, critical requirements
• High Availability: Front Desk control plane + Traefik, drop-in HA across multiple instances
• Bellhop: Android companion app, pairing, roles, monitoring and operator controls
• Development: Local setup, build commands, contributing

Backup & Restore

Backups are created via the Settings page or the admin API (POST /api/backups) using an unfiltered pg_dump --format=custom with zstd compression (level 12 on request, level 19 for scheduled backups). The resulting .dump files therefore contain every database table, not just the configuration ones: providers (encrypted keys), models, virtual key hashes, failover groups, and settings, but also request logs, app logs, the audit log, discovery history, quota snapshots, dashboard user accounts, TOTP secrets and recovery-code hashes, and WebAuthn credentials and sessions. Treat a .dump as sensitive and store it accordingly.

Restoring a backup

The dumps are zstd-compressed, so restoring outside the app needs pg_restore 16 or later built with zstd (the postgres:16-alpine image qualifies).

# Direct
pg_restore --clean --if-exists -d YOUR_DB backup_file.dump

# Via Docker
docker exec -i postgres-container pg_restore --clean --if-exists -U user -d dbname < backup_file.dump

Critical requirements for a working restore

Requirement	Details
MASTER_KEY must match	Provider API keys are AES-256-GCM encrypted using a key derived from MASTER_KEY via Argon2id. Restoring with a different MASTER_KEY will leave all provider keys unrecoverable. The app will start, but key decryption will fail.
Admin token is not in the backup	The admin token hash lives in DATA_DIR/admin-token on the filesystem, not in the database. If that file is lost, a new token is auto-generated on next boot. Check startup logs for the new token.
Virtual keys are irrecoverable	Virtual keys are stored as SHA-256 hashes only. Plaintext virtual keys are never persisted. If you lose the plaintext keys, they cannot be recovered from the backup (by design).

What is and isn't in the backup

Included (in the database, captured by pg_dump): providers (encrypted keys, nonces, salts), models, virtual keys (hashes only), failover groups, settings, request and app logs, the audit log, discovery history, quota snapshots, user accounts, TOTP secrets and recovery-code hashes, WebAuthn credentials and sessions.

Not included (filesystem only): DATA_DIR/admin-token (admin token hash), DATA_DIR/backups/ (the backup files themselves), MASTER_KEY (environment variable).

License

MIT. See CONTRIBUTING.md for the contributor license agreement.