Local whisper transcription

[dilidin2]

Summary

Adds support for local voice transcription via faster-whisper, a fast C++/ONNX reimplementation of Whisper that runs entirely on the host machine — no API key or network access required.

This is useful for users who:

• Do not want to depend on Groq or OpenAI for transcription
• Need offline or air-gapped deployments
• Want to reduce costs by avoiding transcription API calls

Changes

• nanobot/providers/transcription.py — adds FasterWhisperTranscriptionProvider with:
  • Lazy model loading (the model is loaded on first use, not at startup)
  • Class-level model cache with a 10-minute idle TTL — the model is automatically unloaded after inactivity to free RAM/VRAM
  • asyncio.Lock to prevent concurrent calls from loading the model twice
  • Auto-detection of CUDA via torch (falls back to CPU if torch is not installed)
• nanobot/channels/base.py — adds transcription_model_size and transcription_device fields to BaseChannel
• nanobot/config/schema.py — adds the same two fields to ChannelsConfig; transcription_provider now accepts "local" in addition to "groq" and "openai"
• pyproject.toml — adds optional dependency group local-transcription
• tests/providers/test_transcription.py — adds tests for missing file, missing package, device auto-detection, model caching, and concurrent call safety

Installation

faster-whisper is an optional dependency and must be explicitly installed:

uv tool install "nanobot-ai[local-transcription]"

The Whisper model weights (~500 MB for small) are downloaded automatically on first use and cached in ~/.cache/huggingface/hub/.

Configuration

Add the following to your nanobot config:

{
  "transcriptionProvider": "local",
  "transcriptionModelSize": "small",
  "transcriptionDevice": "cpu"
}

Available model sizes (trade-off between quality, speed and RAM):

Model	RAM	Notes
tiny	~390 MB	Fastest, lower accuracy
base	~500 MB	Good for simple speech
small	~960 MB	Recommended default
medium	~3 GB	Higher accuracy
large-v3	~6 GB	Best quality, GPU recommended

Notes

• No changes to the existing Groq and OpenAI provider paths
• If faster-whisper is not installed and provider: local is configured, the transcription returns an empty string and logs a clear install instruction
• torch is not a required dependency — CUDA auto-detection gracefully falls back to CPU if torch is not present

This implementation was developed with AI assistance.