[Feature request] compatibility with real-time endpoints

[guillaumeboniface]

I'm playing with your real-time demo of gemini2.0 and finding myself wishing it'd be compatible with Pipecat Flows! I'm wondering whether that's on the roadmap or whether that's something you could use some help on?

[markbackman]

This is on my priority list. We have some work to do with the GeminiMultimodalLiveLLMService in Pipecat first before this will be possible.

[gururise]

I too tried getting gemini 2.0 realtime to work with pipecat flows, but was unsuccessful.

[gururise]

Just curious if there is any movement on this? Would love to integrate flows with Gemini Flash 2.0 multimodal.

[markbackman]

@gururise not yet. This is work we plan to do in the next week or two.

[igorrendulic]

I just wanted to check in and see if there has been any progress on integrating real-time endpoints with Pipecat Flows. I saw that it was on the roadmap a while back and was curious if there are any updates.

Thanks for your time, and I appreciate all the work you’re putting into this!

[markbackman]

@igorrendulic not yet, but we have plans to support them soon. We're coordinating with OpenAI and Google on improving the flexibility of their APIs, so that they can support runtime updates.

[JosephDahan]

Any updates?

[markbackman]

None yet. We're hoping that the real-time APIs support context updates mid-session. Otherwise, we'll have to break the websocket connection to do this, which is not a feasible solution.

[batwood-1]

Hi, thanks for all of your work and your updates. Do you see any major issues with offloading context management to the realtime server to deal with? Obviously it is less granular control than some use cases would demand, but it could work in some scenarios?

[nischalj10]

Hey team. I think the realtime api now supports mid session system message updates. Is the flow support now possible?