Add Smallest AI TTS service integration#4092
Merged
markbackman merged 3 commits intopipecat-ai:mainfrom Mar 24, 2026
Merged
Conversation
4 tasks
Adds SmallestTTSService, a WebSocket-based TTS service using Smallest AI's Lightning v3.1 model. Follows current Pipecat service conventions: - SmallestTTSSettings dataclass with runtime-updatable settings (voice, language, speed, etc.) - Reconnects on model change; keepalive every 30s to prevent idle timeout - TTS settings default to None so the API applies its own defaults - Model enum: SmallestTTSModel.LIGHTNING_V3_1 Includes a foundational example (07zl-interruptible-smallest.py) using Deepgram STT + Smallest TTS + OpenAI LLM. STT integration will follow in a separate PR once the hallucination/finalize behaviour is resolved. Made-with: Cursor
Codecov Report❌ Patch coverage is
... and 5 files with indirect coverage changes 🚀 New features to boost your workflow:
|
…ystem - Route audio through audio contexts (append_to_audio_context) instead of pushing frames directly, enabling proper turn management and interruptions - Add push_stop_frames and push_start_frame so the base class handles TTSStartedFrame/TTSStoppedFrame lifecycle - Remove manual context_id tracking (self._context_id) in favor of get_active_audio_context_id() - Don't call remove_audio_context on "complete" — Smallest sends one per request, not per turn; let the base class timeout handle cleanup - Guard v2-only params (consistency, similarity, enhancement) so they aren't sent to lightning-v3.1 - Remove request_id from request payload (not a documented request field) - Add flush_audio override to send flush to WebSocket
markbackman
force-pushed
the
harshita/smallest-tts-only
branch
from
c2106f5 to
f68b322
Compare
markbackman
approved these changes
Mar 24, 2026
Contributor
markbackman
left a comment
markbackman
left a comment
There was a problem hiding this comment.
LGTM! I pushed a few additional changes to this branch to align with some TTS service changes we've made. Everything looks good now. Thanks for the help on this one 🙌
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Adds Smallest AI WebSocket-based TTS service, aligned with current Pipecat conventions.
This PR has the following history:
The TTS implementation is taken directly from #4014 with no changes. The only difference is:
stt.pyremoved — will be submitted as a follow-up PR once the STT issue is resolvedstt_latency.pyunchanged —SMALLEST_TTFS_P99will be added with the STT PR07zl-interruptible-smallest.py) usesDeepgramSTTServiceas a stand-in for STTEverything else — settings pattern, keepalive,
_update_settingsoverrides, model enum — is identical to #4014.Key details
SmallestTTSSettingsdataclass with runtime-updatable fields_update_settingsoverride: TTS reconnects only on model changeSmallestTTSModel.LIGHTNING_V3_1