An LLM that sleeps, gets tired, and is born — a biologically inspired architecture for continual learning.
SomnolentLLM is an experimental system that treats an LLM as a living organism with its own rhythms:
- 💤 Sleep — NREM/REM/Deep Sleep cycles for knowledge consolidation
- 😴 Fatigue — homeostatic monitoring for automatic learning regulation
- 🧬 Embryogenesis — "birthing" a model from the knowledge of parent models
- 🌐 Online Feeding — an embryo that grows on your data via HTTP API
Hypothesis: biologically inspired mechanisms make continual LLM learning more stable and efficient.
pip install -e ".[dev]"somnolent-train --model_path <model> --data_path <data>somnolent-birth --config config.qwen35.yaml# Prenatal phase (feeding)
python3 server.py --mode embryonic --port 8080
# Feeding text
curl -X POST http://localhost:8080/embryo/feed \
-d '{"text": "Hello world", "source": "ambient"}'
# After saturation → birth → inference
python3 server.py --mode inference --port 8000| Document | Description |
|---|---|
| CONCEPT | 🆕 Project Idea — what and why |
| ARCHITECTURE | Detailed component architecture |
| EMBRYOGENESIS | Model birth process |
| LORA | LoRA initialization methods |
| MATH | Mathematics behind the algorithms |
┌─────────────────────────────────────────────────────────┐
│ SomnolentLLM │
│ │
│ ┌──────────────┐ ┌──────────────┐ ┌───────────────┐ │
│ │ Sleep Engine │ │ Fatigue │ │ Manifold │ │
│ │ NREM/REM/DS │ │ Monitor │ │ Accumulator │ │
│ └──────────────┘ └──────────────┘ └───────────────┘ │
│ │
│ ┌──────────────┐ ┌──────────────┐ ┌───────────────┐ │
│ │ Episodic │ │ DMN │ │ LoRA Adapter │ │
│ │ Buffer │ │ Generator │ │ (trainable) │ │
│ └──────────────┘ └──────────────┘ └───────────────┘ │
└─────────────────────────────────────────────────────────┘
| Component | Role |
|---|---|
| SleepEngine | FSM of sleep cycles (NREM → REM → Deep Sleep) |
| FatigueMonitor | Homeostatic sleep pressure |
| ManifoldAccumulator | Geometrically correct gradients |
| EpisodicBuffer | Activation storage with LRU eviction |
| DMNGenerator | "Dream" generation in REM phase |
| LoRA Adapter | Low-rank adaptation (target layers only) |
Results:
- Perplexity: 9.62 (baseline: 9.69) — quality preserved
- Generation: coherent text
- OOM errors: 0
- Genetic phase — SVD initialization of LoRA from parent models
- Synthesis phase — generation of synthetic memories
- Awakening — first sleep cycle, wake up
# Check embryo
python3 debug/quick_sanity_check.py --adapter ./output/born_adapter_v5.pt
python3 debug/evaluate_embryo.py --adapter ./output/born_adapter_v5.ptWe use Qwen3.5-2B with a hybrid architecture:
- linear_attention (18 layers) — frozen, fast
- full_attention (6 layers) — trained via LoRA
A balance between speed and quality during training.
Data → Model → Fatigue → Sleep → Improved Model
Parent Model A + Parent Model B → Embryogenesis → Birth → Fine-tuning
HTTP API → Embryo → Saturation → Birth → Inference
somnolent_llm/
├── core/ # Core: sleep, fatigue, birth
├── adapters/ # LoRA adapters for Qwen3.5
├── memory/ # Episodic memory and dream generators
├── utils/ # Configuration, utilities
├── examples/ # Usage examples
├── docs/ # Documentation
├── scripts/ # Embryo feeding scripts
└── client/ # HTTP API clients
- Python 3.9+
- PyTorch 2.0+
- Transformers 4.35+
- CUDA 11.8+ (for GPU training)
✅ Embryogenesis completed — model was born successfully
⏸️ Project on pause — code works, but development is suspended
This project was started on March 15, 2026. Over the following weeks it was actively developed with sleep mechanics, fatigue monitoring, and embryogenesis features. The codebase represents a snapshot of the system at the point when development was paused.
GitHub Issues for questions and discussions