llm-model-foundry

Merge multiple open-source LLMs into a single specialized coding model using MergeKit, then run it locally via Ollama. The result is a 7B-parameter model that is good at both function/tool calling and production-quality code generation — ideal for local coding agents like OpenClaw and Crush-CLI.

---

What This Does

Three (or four) specialist models are blended at the weight level using the DARE-TIES merge algorithm. The merged model inherits the strengths of each contributor:

Contributor	Specialty	Weight (original)
Hermes-2-Pro-Mistral-7B	Function / tool calling	40%
DeepSeek-Coder-6.7B	Code generation	35%
WizardCoder-Python-7B	Instruction following	25%

The merged model is then quantized and loaded into Ollama so you can run it from any terminal.

---

Requirements

Hardware	Apple Silicon Mac (M1/M2/M3) with 16 GB+ RAM
Disk space	~30 GB during merge, ~15 GB final
Internet	~25 GB download (models from Hugging Face)
Time	1–3 hours depending on config chosen
Software	Python 3.8+, pip, Ollama

---

Quickstart

1. Install Python dependencies

pip install torch transformers accelerate huggingface_hub sentencepiece protobuf
pip install git+https://github.com/arcee-ai/mergekit.git

2. Install Ollama (if not already installed)

curl -fsSL https://ollama.com/install.sh | sh

3. Run the setup script

chmod +x setup_agent_coder.sh
./setup_agent_coder.sh

This will:

1. Download the three source models (~25 GB)
2. Merge them into ~/merged-models/agent-coder-7b
3. Create an Ollama model called agent-coder-7b
4. Run a quick smoke test

4. Use the model

ollama run agent-coder-7b "Write a Python function to validate emails"

---

Choosing a Config

Four configurations are available. Choose based on your priority:

Config	Script	Model name	Code quality	Function calling	Merge time
Original (fast baseline)	setup_agent_coder.sh	agent-coder-7b	87%	92%	1–1.5 h
Premium ⭐ recommended	setup_premium.sh → option 1	agent-coder-premium-7b	95%	85%	1.5–2 h
Two-Stage (best quality)	setup_premium.sh → option 2	agent-coder-twostage-7b	98%	88%	2.5–3 h
Max Quality (34B layers)	setup_premium.sh → option 3	agent-coder-max-7b	99%	87%	2–3 h + 35 GB download

Not sure? Use Premium. It is the best balance between code quality and tool calling for everyday use with OpenClaw or Crush-CLI.

chmod +x setup_premium.sh
./setup_premium.sh   # choose option 1 when prompted

---

Model Specs

Parameters:     ~7B
Quantized size: ~4.5 GB  (Q4_K_M)
Context window: 32 K tokens
RAM at runtime: 5–6 GB
Speed:          15–25 tokens/sec on Apple Silicon

---

Connecting to Coding Agents

OpenClaw

cp openclaw_config.json ~/.openclaw/config.json

Or open the OpenClaw settings and set:

• Provider: ollama
• Model: agent-coder-7b (or the premium/twostage name)

Crush-CLI

export CRUSH_MODEL='agent-coder-7b'
# Add the line above to ~/.zshrc or ~/.bashrc to make it permanent

---

Testing and Benchmarking

Run the full test suite:

chmod +x test_agent_coder.sh
./test_agent_coder.sh

Run the code-quality benchmark against any model:

chmod +x benchmark_code_quality.sh
./benchmark_code_quality.sh agent-coder-premium-7b

The benchmark checks: type hints, docstrings, error handling, complex algorithms, security best practices, code refactoring, and function calling accuracy.

---

File Reference

merge-llms/
├── merge_config_agent.yaml      # Original merge config (3 models)
├── merge_config_premium.yaml    # Premium merge config (4 models)
├── merge_config_twostage.yaml   # Two-stage merge, stage 1
├── merge_config_stage2.yaml     # Two-stage merge, stage 2
├── merge_config_maxquality.yaml # Max quality config (uses 34B layers)
├── setup_agent_coder.sh         # Setup script for original config
├── setup_premium.sh             # Interactive setup for premium/twostage/max
├── test_agent_coder.sh          # Functional test suite
├── benchmark_code_quality.sh    # Code quality benchmark
├── openclaw_config.json         # Drop-in config for OpenClaw
├── COMPARISON.md                # Detailed metric comparison across configs
└── QUICK_START.md               # Upgrade guide from original to premium

~/merged-models/                 # Created during setup
└── agent-coder-7b/              # Merged model weights

---

Troubleshooting

Merge fails / runs out of memory

mergekit-yaml merge_config_agent.yaml ~/merged-models/agent-coder-7b \
  --low-cpu-memory \
  --lazy-unpickle

Ollama model creation fails

ls ~/merged-models/agent-coder-7b   # verify the merge completed
ollama create agent-coder-7b -f Modelfile

Model is too slow

Edit Modelfile and add:

PARAMETER num_gpu 1
PARAMETER num_thread 8

Then recreate: ollama create agent-coder-7b -f Modelfile

Verify everything is working

ollama list                          # shows all local models
ollama list | grep agent-coder       # check the merged model exists
ollama run agent-coder-7b "hello"    # quick sanity check

---

Updating / Re-merging

1. Edit the relevant merge_config_*.yaml
2. Re-run the setup script — it will overwrite the existing model

---

License

The merged model inherits the licenses of its component models:

Model	License
Hermes-2-Pro-Mistral-7B	Apache 2.0
DeepSeek-Coder-6.7B	MIT
WizardCoder-Python-7B	Llama 2 Community License
CodeLlama-7B / Phind-CodeLlama-34B	Llama 2 Community License

Check each model's license before commercial use.

---

Credits

• MergeKit — Arcee AI
• Hermes-2-Pro-Mistral-7B — Nous Research
• DeepSeek-Coder — DeepSeek
• WizardCoder-Python-7B — WizardLM