Skip to content

Commit 1c08be0

Browse files
authored
Merge pull request #130 from garrettHensley/readme
Add a readme for clarity
2 parents ebb4723 + 44e847d commit 1c08be0

File tree

1 file changed

+77
-0
lines changed

1 file changed

+77
-0
lines changed

README.md

Lines changed: 77 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,77 @@
1+
# Docker Model CLI
2+
3+
A powerful command-line interface for managing, running, packaging, and deploying AI/ML models using Docker. This CLI lets you install and control the Docker Model Runner, interact with models, manage model artifacts, and integrate with OpenAI and other backends—all from your terminal.
4+
5+
## Features
6+
- **Install Model Runner**: Easily set up the Docker Model Runner for local or cloud environments with GPU support.
7+
- **Run Models**: Execute models with prompts or in interactive chat mode, supporting multiline input and OpenAI-style backends.
8+
- **List Models**: View all models available locally or via OpenAI, with options for JSON and quiet output.
9+
- **Package Models**: Convert GGUF files into Docker model OCI artifacts and push them to registries, including license and context size options.
10+
- **Configure Models**: Set runtime flags and context sizes for models.
11+
- **Logs & Status**: Stream logs and check the status of the Model Runner and individual models.
12+
- **Tag, Pull, Push, Remove, Unload**: Full lifecycle management for model artifacts.
13+
- **Compose & Desktop Integration**: Advanced orchestration and desktop support for model backends.
14+
15+
## Building
16+
1. **Clone the repo:**
17+
```bash
18+
git clone https://github.com/docker/model-cli.git
19+
cd model-cli
20+
```
21+
2. **Build the CLI:**
22+
```bash
23+
make build
24+
```
25+
3. **Install Model Runner:**
26+
```bash
27+
./model install-runner
28+
```
29+
Use `--gpu cuda` for GPU support, or `--gpu auto` for automatic detection.
30+
31+
## Usage
32+
Run `./model --help` to see all commands and options.
33+
34+
### Common Commands
35+
- `model install-runner` — Install the Docker Model Runner
36+
- `model run MODEL [PROMPT]` — Run a model with a prompt or enter chat mode
37+
- `model list` — List available models
38+
- `model package --gguf <path> --push <target>` — Package and push a model
39+
- `model logs` — View logs
40+
- `model status` — Check runner status
41+
- `model configure MODEL [flags]` — Configure model runtime
42+
- `model unload MODEL` — Unload a model
43+
- `model tag SOURCE TARGET` — Tag a model
44+
- `model pull MODEL` — Pull a model
45+
- `model push MODEL` — Push a model
46+
- `model rm MODEL` — Remove a model
47+
48+
## Example: Interactive Chat
49+
```bash
50+
./model run llama.cpp "What is the capital of France?"
51+
```
52+
Or enter chat mode:
53+
```bash
54+
./model run llama.cpp
55+
Interactive chat mode started. Type '/bye' to exit.
56+
> """
57+
Tell me a joke.
58+
"""
59+
```
60+
61+
## Advanced
62+
- **Packaging:**
63+
Add licenses and set context size when packaging models for distribution.
64+
65+
## Development
66+
- **Run unit tests:**
67+
```bash
68+
make unit-tests
69+
```
70+
- **Generate docs:**
71+
```bash
72+
make docs
73+
```
74+
75+
## License
76+
[Apache 2.0](LICENSE)
77+

0 commit comments

Comments
 (0)