Give your AI agents a shared view of GPU memory. Through MCP, they can inspect what's using VRAM, make room for Ollama models, and see which sessions are already relying on them.
Each client session runs its own server, of about 100 MB. Sessions under the
same OS user share claims, reservations, pending-operation records, and an
audit log, kept in ~/.cache/vram-mcp by default. Capacity and process
readings apply to one configured GPU. Claimed or recently busy models are
protected from eviction by default.
On Windows, vram-mcp also measures whether the driver is paging VRAM, so
pressure tells a GPU that is merely full from one that is thrashing. It also
names model runtimes on the GPU that the configured Ollama does not serve, such
as a separate llama-server.
Use Python 3.10+, a running Ollama instance, and an NVIDIA GPU with
drivers for memory readings. Windows and Linux are tested. Without
nvidia-smi or NVML, claims and unloads still work, but some readings are
unavailable, and warm needs force=True because no fit can be verified.
Run vram-mcp on the machine hosting Ollama and the GPU. Changing
OLLAMA_BASE_URL redirects model requests; GPU and process inspection always
remain local. On a multi-GPU host, set VRAM_MCP_GPU_INDEX in every client's
server environment to select the same device; the default is GPU index 0.
With uv and Git
installed, choose your client below. uvx fetches vram-mcp from GitHub into an
isolated environment; no source checkout is needed.
These commands install 0.3.1, the latest release. This page describes
main, which is ahead of it: paging detection on Windows, warm with
num_ctx, strict_fit and partial-fit reporting, unmonitored-runtime
reporting, and the Kokoro worker are unreleased. To run main, drop @v0.3.1
from the Git URL.
claude mcp add vram --scope user -- uvx --from git+https://github.com/sushiHex/vram-mcp@v0.3.1 vram-mcpcodex mcp add vram -- uvx --from git+https://github.com/sushiHex/vram-mcp@v0.3.1 vram-mcpHermes — merge into ~/.hermes/config.yaml:
mcp_servers:
vram:
command: uvx
args: ["--from", "git+https://github.com/sushiHex/vram-mcp@v0.3.1", "vram-mcp"]Claude Desktop and other clients using mcpServers JSON
Merge this entry into your client's MCP configuration:
{
"mcpServers": {
"vram": {
"command": "uvx",
"args": ["--from", "git+https://github.com/sushiHex/vram-mcp@v0.3.1", "vram-mcp"]
}
}
}If the client cannot find uvx, use its absolute executable path. In JSON,
Windows paths need escaped backslashes, such as C:\\tools\\uvx.exe.
On Windows, the MCP SDK needs pywin32, which has no free-threaded build. If
uv picks a free-threaded Python (3.14t) and the install fails, add
--python 3.13 right after uvx.
Some clients start servers with a minimal environment that lacks
ProgramFiles. Under it, nvidia-smi fails with "Failed to initialize NVML:
Unknown Error", and every GPU reading is unavailable. Add ProgramFiles (for
example C:\Program Files) to the server's environment in the client
configuration. #52 makes
vram-mcp supply it.
Reconnect your client after registration. It launches the server over stdio;
running vram-mcp by hand waits for MCP input. Prefer a local installation?
See installing from source.
Upgrading from 0.2? Stop every client using vram-mcp first; old and new servers must never share one ledger. See the 0.3.0 notes and the 0.3.1 fix.
Ask your agent:
Check GPU memory with vram-mcp. Show what's loaded, who is using it, and how much room is left. Report any unavailable readings.
The agent should call vram_status() and list_claims(). Status includes a
readable summary, the selected GPU, loaded models, available process details
(command lines cut to 160 characters unless cmdline_chars says otherwise; see
configuration), and observations metadata. Check each observation's status; unavailable
means the corresponding value is unknown, rather than empty or zero.
list_claims() also shows capacity reserved for training or other non-Ollama
work.
These examples are MCP tool calls made inside a connected client, not shell
commands or a Python API. Use a descriptive session label for owner and by,
such as codex:review, so another agent can identify your work.
-
Inspect before changing memory. Call
vram_status()andlist_claims(). Model names are canonicalized consistently: a bare final name such asllama3becomesllama3:latest; comparison is case-insensitive, and Ollama's default registry/library prefix is removed. -
Claim a model before relying on it. Replace the example model below with one already installed in Ollama. Continue only if the claim succeeds, and save its returned
claim_id.claim(model="llama3:latest", owner="codex:review", purpose="Review local code") -
Make room when needed.
ensure_free(gb=8, by="codex:review")unloads unprotected models largest-first. Choose the target for your workload and checkoutcome,free_mb,declined,reserved_mb, andobservations. It does not evict when selected-GPU capacity or Ollama residency is unknown. Reaching the target does not give you ownership of that space. -
Load and check. Call
warm(model="llama3:latest", keep_alive="10m", by="codex:review")when needed. Inspectoutcome,reason, andobservations.succeededmeans residency was reconciled after the request;refusedmeans no Ollama request was sent;failedmeans Ollama definitively rejected it; andunknownmeans the request may still be running and includespending_until. Do not retry a same-model mutation until that pending window ends. Unknown free VRAM refuses a new load (reason="free_unknown") rather than guessing. Passnum_ctxto request a context length; admission then counts an estimated KV cache (kv_cache_mb), timesnum_parallelif the server runs parallel slots. Passstrict_fit=True(withnum_ctx) to also refuse a load that would not fit in free VRAM minus a 1 GiB margin, even when nothing is reserved;forcecannot override it. A load slower thantimeout_seconds(defaultVRAM_MCP_LOAD_TIMEOUT, 110 s) ends asunknownwithreason="load_timeout". Warm admission is also serialized across models sharing the selected GPU. A model that was already resident has zero incremental residency cost (reason="already_resident"). Loading successfully does not guarantee full GPU residency: when Ollama places part of the model outside VRAM, the result readspartial_fit: truewithoffloaded_mb, and the outcome stayssucceeded.partial_fitisnullwhen residency couldn't be read. vram-mcp reports a spill but never unloads or retries on its account. Every result also carrieselapsed_sand, when the request failed, theexceptionclass. The audit event keeps these, along with admission's inputs and the names of any unavailable sources. Run inference through your usual Ollama client. -
Renew and release. Claims expire after one hour by default. Call
renew(claim_id="<returned claim_id>")before expiry for longer work andrelease(claim_id="<returned claim_id>")when finished, including if loading fails. Releasing a claim does not unload the model; claim expiry and Ollama'skeep_aliveare separate.
For training or diffusion, use reserve(gb=8, owner="codex:training", purpose="LoRA training") to declare capacity, then renew/release its claim_id
in the same way. A reservation records intent; it does not allocate memory.
Validation rejects invalid requests: capacities, TTLs, and trend windows
must be finite and positive; model, owner, purpose, and caller labels must be
nonblank; warm accepts a positive Ollama duration or -1 for indefinite
residency. Use unload() rather than a zero keep_alive.
Coordination is cooperative. Direct Ollama calls can bypass it. force=True
on unload, ensure_free, or warm overrides claims, busy protection, or
reservation admission; use it only after resolving competing work. It does not
bypass an unreadable coordination ledger or another pending mutation.
busy=null means unknown and does not block eviction on its own. See
coordination details.
pressure.state |
Meaning |
|---|---|
ok |
Nothing is wrong, and every input that claim rests on was actually read. Missing evidence yields unknown, never ok. |
tight |
Less than 1 GiB is free on the selected GPU. |
degraded |
Ollama placed part of a model on the CPU; expect slower inference. |
thrashing |
The driver is paging VRAM: in at least 5 of the last 10 seconds, WDDM's memory manager kept the GPU busy. |
Missing required telemetry produces pressure.state="unknown". Read
pressure.coverage and top-level observations to see which evidence was
available. Paging detection is best effort. It needs a paging reading for the
selected GPU, which vram-mcp takes on Windows from a sampler that starts with
the server. On Linux no such reading exists, so pressure reads unknown
where it would read ok. The detection is a measured correlation, not a WDDM
contract; see the acceptance notes.
Non-local memory is reported but does not decide the verdict. CPU
offload is tracked separately.
vram_status lists, under unmonitored_runtimes, model runners (llama-server,
ollama_llama_server) on the selected GPU that no model in the configured
Ollama accounts for, and its summary names them. Their models are not in
loaded, and vram-mcp can't unload them. Check list_claims() before acting
on one: another session may own it. Disabling auditing
stops history detection and trend sampling, but does not disable current GPU or
process observations.
Use history() to investigate a model disappearing and trend(hours=1) to
review memory changes. Trends are sampled when vram_status() reads the GPU,
not in the background. The Windows paging sampler is separate: it only informs
pressure. See diagnostics.
The client's MCP schema provides full arguments and defaults.
| Tool | Purpose |
|---|---|
vram_status |
Inspect GPU memory, models, claims, activity, and pressure. |
list_loaded |
List resident models and their claim/busy details. |
ensure_free |
Try to reach a free-memory target by unloading unprotected models. |
unload |
Evict a named model, respecting claims and recent activity. |
warm |
Load a model for a chosen keep_alive, checking reservations. |
claim, reserve |
Declare model use or capacity needed for other GPU work. |
renew, release |
Extend or end your claim or reservation. |
list_claims |
See active model claims and capacity reservations. |
history, trend |
Review recorded actions, observed changes, and memory samples. |
advise |
Get configuration suggestions; no settings are changed. |
load_kokoro, kokoro_status, stop_kokoro |
Optional Windows-only launch and settlement of the operator-configured reviewed Kokoro worker, with bounded exit diagnostics; launch acceptance is not GPU readiness. |
| Guide | Read it for |
|---|---|
| Configuration | Source installation, environment variables, and connection troubleshooting. |
| Coordination and diagnostics | Claims, reservations, protection limits, pressure, and audit history. |
| Hardware acceptance | The opt-in run against a real GPU and Ollama, and the environments it has validated. |
| Owned Kokoro worker | Optional private profile, supported worker pin, admission, process ownership and separate deployment/qualification limits. |
| Release notes | What changed in each version, and how to upgrade. |
| Releasing | How a GitHub Release builds, attests and publishes the distributions. |
git clone https://github.com/sushiHex/vram-mcp.git
cd vram-mcp
python -m venv .venvActivate with source .venv/bin/activate on Linux or
.\.venv\Scripts\Activate.ps1 in PowerShell, then run:
python -m pip install -e ".[dev]"
python -m pytest -qTests use mocked GPU, process, and HTTP calls; no GPU or Ollama daemon is needed. Validation on real hardware is a separate, opt-in run; see hardware acceptance. CI covers Python 3.10–3.14 on Windows and Linux. Current model management targets Ollama and NVIDIA. Runners outside Ollama are reported, not managed; AMD/Intel telemetry and vLLM/llama.cpp management are future work. Contributors, human or agent, should read AGENTS.md for layout, style and testing conventions.
MIT — see LICENSE. Companion project: hardline-mcp for agent messaging.