Add auto_mode support for inworld plugin

[ianbbqzy]

Inworld auto_mode to improve first token latency.

[markbackman]

Tagging @cshape for review.

@ianbbqzy can you link to the docs for this? I don't see this feature listed.

[cshape]

Tagging @cshape for review.

@ianbbqzy can you link to the docs for this? I don't see this feature listed.

It's a new feature which will be going out next week. @ianbbqzy and I will link the docs, review/test and ping you once we're ready then. Thanks!

[markbackman]

It's a new feature which will be going out next week. @ianbbqzy and I will link the docs, review/test and ping you once we're ready then. Thanks!

Sounds good!

[cshape]

Hi @markbackman -

Chatted with @ianbbqzy and this feature (basically a smart auto-flushing mechanism to get optimal latency and prosody from our TTS with websockets) is good to merge now from our perspective.

It's currently slated to be rolled out on our side later this week, but will be safely ignored if the Pipecat SDK includes it before then. Our preference is to merge it into the Pipecat SDK ASAP so users can benefit as soon as our release goes out.

Thanks!

[markbackman]

Thanks for confirming, @cshape!

Can you please add a changelog to this PR and then it'll be ready to go:
https://github.com/pipecat-ai/pipecat/blob/main/CONTRIBUTING.md#changelog-entries

[codecov]

Codecov Report

❌ Patch coverage is 0% with 23 lines in your changes missing coverage. Please review.

Files with missing lines	Patch %	Lines
src/pipecat/services/inworld/tts.py	0.00%	23 Missing ⚠️

Files with missing lines	Coverage Δ
src/pipecat/services/inworld/tts.py	0.00% <0.00%> (ø)

... and 21 files with indirect coverage changes

🚀 New features to boost your workflow:

• ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

[ianbbqzy]

Can you please add a changelog to this PR and then it'll be ready to go:
@markbackman Done! thanks!

[ianbbqzy]

@markbackman sorry, I realized that this auto_mode could exacerbate the timestamps resetting issue. I will work on a fix and report back

[ianbbqzy]

Implemented timestamp cumulation using flushCompleted response message as the indication that timestamps will reset to 0.

[markbackman]

Thanks for patiently educating me on this. This does sound like a nice improvement. In running the example, I see really solid TTFB performance now. Kudos!

Before this is ready, just two small things:

1. Lint the code (either uv run scripts/fix-ruff.sh or install pre-commit hook: uv run pre-commit install)
2. Update the changelog per my comment.

[ianbbqzy]

Thanks for patiently educating me on this. This does sound like a nice improvement. In running the example, I see really solid TTFB performance now. Kudos!

No problem at all and thanks for your team's and your personal insights into Voice AI orchestration in general. There might still be some rough edges with the Inworld API, but we will keep improving it to truly show the potentials of our model.

solid TTFB performance now

Thanks! but we also know it's not there yet, we have an even bigger latency improvement regarding timestamps coming soon next week, stay tuned!