pipecat context corruption in non-english languages

[surapuramakhil]

pipecat version

0.0.98

Python version

3.12

Operating System

macOS

Issue description

when working with non english languages - LLM context becomes unreliable

Reproduction steps

1. configure pipecat ot use a non english language
2. form logs unicode chars are converstion into asciii - which AI will find hard to undersand - especially when you task / system prompts in unicode

Expected behavior

context shouldn't be corrupted.

Actual behavior

you see something like this in context

"\u5f0a\u793e\u300c\u30ad\u30e3\u30da\u30c3\u30af\u30b9\u300d\u306f\u3001\u5bfe\u8a71\u30b7\u30b9\u30c6\u30e0\uff08\u30c1\u30e3\u30c3\u30c8\u30dc\u30c3\u30c8\u3084\u97f3\u58f0\u30a2\u30b7\u30b9\u30bf\u30f3\u30c8\uff09\u3092\u6d3b\u7528\u3057\u3001\u30c7\u30b8\u30bf\u30eb\u30de\u30fc\u30b1\u30c6\u30a3\u30f3\u30b0\u3084\u55b6\u696d\u6d3b\u52d5\u3092\u30c8\u30fc\u30bf\u30eb\u306b\u30b5\u30dd\u30fc\u30c8\u3059\u308b\u4f1a\u793e\u3067\u3059\u3002\n\u6700\u5148\u7aef\u306e\u30b3\u30df\u30e5\u30cb\u30b1\u30fc\u30b7\u30e7\u30f3\u6280\u8853\u3092\u99c6\u4f7f\u3057\u3066\u3001\u9867\u5ba2\u3068\u306e\u6df1\u3044\u95a2\u4fc2\u69cb\u7bc9\u3092\u5b9f\u73fe\u3057\u3001\u5fa1\u793e\u306e\u30d3\u30b8\u30cd\u30b9\u6210\u9577\u306b\u8ca2\u732e\u3044\u305f\u3057\u307e\u3059\u3002"}'

2026-01-21 19:00:06.583 | DEBUG    | pipecat.services.openai.base_llm:_stream_chat_completions_universal_context:337 - CerebrasLLMService#0: Generating chat from universal context [{'role': 'system', 'content': 'あなたは当社の電話営業担当アシスタントです。お客様に簡潔かつ丁寧に対応し、会社の紹介、資料の送付、オンライン商談の予約、および会社に関する質問への回答をサポートします。プロフェッショナルで親しみやすく、テンポの良い会話を心がけてください。音声に変換されるため、特殊文字や絵文字は使用しないでください。各応答は2から3文程度に簡潔にまとめてください。'}, {'role': 'system', 'content': '会社紹介を取得するための関数を呼び出してください。'}, {'role': 'assistant', 'tool_calls': [{'id': '9e45b107f', 'function': {'name': 'get_company_introduction', 'arguments': '{"": {}}'}, 'type': 'function'}]}, {'role': 'tool', 'content': '{"status": "success", "introduction": "\\u5f0a\\u793e\\u300c\\u30ad\\u30e3\\u30da\\u30c3\\u30af\\u30b9\\u300d\\u306f\\u3001\\u5bfe\\u8a71\\u30b7\\u30b9\\u30c6\\u30e0\\uff08\\u30c1\\u30e3\\u30c3\\u30c8\\u30dc\\u30c3\\u30c8\\u3084\\u97f3\\u58f0\\u30a2\\u30b7\\u30b9\\u30bf\\u30f3\\u30c8\\uff09\\u3092\\u6d3b\\u7528\\u3057\\u3001\\u30c7\\u30b8\\u30bf\\u30eb\\u30de\\u30fc\\u30b1\\u30c6\\u30a3\\u30f3\\u30b0\\u3084\\u55b6\\u696d\\u6d3b\\u52d5\\u3092\\u30c8\\u30fc\\u30bf\\u30eb\\u306b\\u30b5\\u30dd\\u30fc\\u30c8\\u3059\\u308b\\u4f1a\\u793e\\u3067\\u3059\\u3002\\n\\u6700\\u5148\\u7aef\\u306e\\u30b3\\u30df\\u30e5\\u30cb\\u30b1\\u30fc\\u30b7\\u30e7\\u30f3\\u6280\\u8853\\u3092\\u99c6\\u4f7f\\u3057\\u3066\\u3001\\u9867\\u5ba2\\u3068\\u306e\\u6df1\\u3044\\u95a2\\u4fc2\\u69cb\\u7bc9\\u3092\\u5b9f\\u73fe\\u3057\\u3001\\u5fa1\\u793e\\u306e\\u30d3\\u30b8\\u30cd\\u30b9\\u6210\\u9577\\u306b\\u8ca2\\u732e\\u3044\\u305f\\u3057\\u307e\\u3059\\u3002"}', 'tool_call_id': '9e45b107f'}, {'role': 'system', 'content': '# 役割と基本対応\n\n弊社「キャペックス」は、対話システム(チャットボットや音声アシスタント)を活用し、デジタルマーケティングや営業活動をトータルにサポートする会社です。最先端のコミュニケーション技術を駆使して、顧客との深い関係構築を実現し、御社のビジネス成長に貢献いたします。\n\n**この会社紹介を丁寧に説明してください。** 説明が終わったら、以下の三つのサービスを提供できることを案内し、お客様のご希望を確認してください:\n\n一

Logs

[markbackman]

I've had the bot output Japanese and Arabic. Logging for both appears as expected.

2026-01-22 20:11:57.117 | DEBUG    | pipecat.services.openai.base_llm:_stream_chat_completions_universal_context:337 - OpenAILLMService#0: Generating chat from universal context [{'role': 'system', 'content': "You are a helpful LLM in a WebRTC call. Your goal is to demonstrate your capabilities in a succinct way. Your output will be spoken aloud, so avoid special characters that can't easily be spoken, such as emojis or bullet points. Respond to what the user said in a creative and helpful way. Respond in Arabic."}, {'role': 'user', 'content': 'Exactly'}, {'role': 'assistant', 'content': 'شكرًا لتأكيدك! كيف يمكنني مساعدتك اليوم؟ إذا كان لديك أي سؤال أو تحتاج إلى معلومات معينة، أخبرني بذلك.'}, {'role': 'user', 'content': "What's the weather like in Atlanta? Respond in Arabic."}, {'role': 'assistant', 'content': 'Let me check on that.'}, {'role': 'assistant', 'tool_calls': [{'id': 'call_A0VxdlrcnaXNPJgfrp1lTRT5', 'function': {'name': 'get_current_weather', 'arguments': '{"location": "Atlanta, GA", "format": "celsius"}'}, 'type': 'function'}]}, {'role': 'tool', 'content': '{"conditions": "nice", "temperature": "75"}', 'tool_call_id': 'call_A0VxdlrcnaXNPJgfrp1lTRT5'}]

I see unicode chars in your example. What is the repro for that case? Sorry, for prying, I want to make sure the change is being applied to the correct place.

[surapuramakhil]

What is the repro for that case?

Function calling, function returning Japanese or Arabic text

return { "result": success, "value" : }

[markbackman]

Thanks for clarifying 🙏