GeminiLiveLLMService doesn't handle server_content.interrupted - causes delayed interruptions

[Punchy25]

Problem

When using GeminiLiveLLMService with Gemini Live API, interruptions are noticeably delayed compared to using LiveKit Agents SDK directly. After investigation, I found the root cause.

Root Cause

Gemini Live API sends two signals when a user starts speaking:

1. server_content.interrupted = True - fires instantly when VAD detects speech
2. server_content.input_transcription - fires after speech is transcribed (500-2000ms later)

Currently, GeminiLiveLLMService only handles input_transcription for interruption detection (via emulated VAD). The instant interrupted signal is ignored.

Current behavior in _connection_task_handler:

# Handled:
✅ message.server_content.model_turn
✅ message.server_content.turn_complete
✅ message.server_content.input_transcription

# Not handled:
❌ message.server_content.interrupted

Impact

Signal Used	Interruption Latency	User Experience
interrupted	~50-100ms	Natural, snappy
input_transcription	~500-2000ms	Bot talks over user

The 1-2 second delay makes conversations feel unnatural. Users have to finish their phrase before the bot stops talking.

Proposed Fix

Add handling for server_content.interrupted in the message loop:

async for message in turn:
    # Check interrupted signal FIRST - instant VAD detection
    if message.server_content and getattr(message.server_content, 'interrupted', False):
        logger.debug("Gemini VAD: interrupted signal received")
        await self.push_frame(StartInterruptionFrame())
        continue
    
    # ... rest of existing handlers

Reference

LiveKit Agents SDK handles this signal directly, which is why it feels snappier:

if server_content.interrupted:
    self.speaker_audio_buffer.clear()

Environment

• pipecat-ai version: 0.0.90
• Model: gemini-2.5-flash-native-audio-preview-12-2025
• Transport: LiveKit

[markbackman]

Thanks for reporting. Can you clarify your finding? In my testing, I see interruptions work reliably and fast. Though, I'm running Silero as a local VAD, which triggers and InterruptionFrame and stops output when a user speaks. You can test this using main with:
https://github.com/pipecat-ai/pipecat/blob/main/examples/foundational/26a-gemini-live-transcription.py

Can you say more about the configuration you're using? I'm assuming there's no local VAD, but perhaps you can confirm.

FYI @kompfner who has worked most recently on this service. Thoughts on this one?

[Punchy25]

Thanks for the quick response!

My configuration: No local VAD (no Silero). Using only Gemini's server-side VAD for turn detection. This is intentional - with Gemini Live native audio, I wanted to rely solely on Gemini's VAD rather than adding a local VAD layer.

The issue: Without local VAD, the only interruption signal comes from enable_emulated_vad_interruptions=True, which triggers on input_transcription. This works, but there's a 700-1100ms delay because it waits for Gemini to transcribe the speech before triggering the interrupt.

What I found: Gemini sends server_content.interrupted = True instantly when its VAD detects speech - way before transcription arrives. But Pipecat doesn't handle this signal.

My workaround: I extended GeminiLiveLLMService to intercept server_content.interrupted:

class GeminiLiveWithInstantInterrupt(GeminiLiveLLMService):
    async def _handle_session_ready(self, session):
        await super()._handle_session_ready(session)
        original_receive = self._session.receive
        self._session.receive = lambda: self._intercept_messages(original_receive())

    async def _intercept_messages(self, message_iterator):
        async for message in message_iterator:
            if message.server_content and message.server_content.interrupted:
                logger.info("INSTANT INTERRUPT: Gemini detected user speech")
                await self._handle_interruption()
                await self.push_frame(InterruptionFrame())
            yield message

Results:

23:18:01.542 | INSTANT INTERRUPT: Gemini detected user speech
23:18:01.544 | Bot stopped speaking (+2ms)
23:18:02.234 | Transcription arrives (+692ms later)

~700-1100ms faster interruptions by using the instant signal instead of waiting for transcription.

Suggestion: For users who want to use only Gemini's native VAD (no Silero), it would be helpful if GeminiLiveLLMService handled server_content.interrupted natively. This would give instant interruptions without needing a local VAD.

[lukepayyapilli]

I've submitted a PR with handling for server_content.interrupted: #3429

@Punchy25 would you be able to try it out and let me know if it works for your use case?

Thanks!