fix(telegram): fix 4 streaming truncation bugs — safe_limit, cursor cleanup, metadata forwarding, overflow chunking#5270
Conversation
…leanup, metadata forwarding, overflow chunking Problem 1 — MDv2 escape inflation blows past 4096 Raw safe_limit of ~3994 chars doesn't account for MDv2 backslash escaping. Every special char (. - ( ) _ etc.) adds 1 byte; a 3994-char message can inflate to 4200+ after format_message(). Fix: use 75% of MAX_MESSAGE_LENGTH as the safe ceiling (3072 chars), leaving real headroom for escaping. Problem 2 — Cursor stuck in split messages When an overflow split is triggered, the first chunk's cursor-removal edit can set _edit_supported=False on failure. All subsequent edits including the final cursor-removal are then silently skipped — both messages keep ▉. Fix: bypass _edit_supported for cursor-removal operations; call adapter.edit_message() directly with a single retry. Problem 3 — metadata not forwarded in edit path _send_or_edit called adapter.edit_message() without metadata=self.metadata, so thread_id was unavailable. Overflow chunks were sent to the main chat instead of the correct supergroup topic. Fixed at all edit_message call sites in stream_consumer. Problem 4 — edit_message overflow truncates instead of chunking The message_too_long handler in edit_message() truncated to 4076 chars + '…' and lost the rest of the content. Fix: split into 80%-limit chunks via truncate_message(), edit first chunk in place, send remaining chunks as new messages with message_thread_id=effective_thread_id to preserve topic routing. Bonus: normalize \r\n and bare \r line endings in format_message(). API responses from Close.com, GHL, etc. frequently use Windows line endings; Telegram renders \r as a visible character producing garbled output.
|
Four genuinely distinct bugs, each with a clear repro and a targeted fix. Bug 1 (MDv2 escape inflation blowing past 4096) is a great catch — byte-count vs. rendered-length divergence is exactly the thing that slips through. The |
|
Closing — the upstream refactor of |
Summary
Four bugs that cause messages to be cut off, garbled, or land in the wrong Telegram topic during streaming. All four affect anyone using supergroup forum topics, which is a growing use case since
group_topicsskill binding merged.Bug 1: MDv2 escape inflation blows past 4096 (
stream_consumer.py)The old safe_limit was
_raw_limit - len(cursor) - 100at roughly 3994 chars. MDv2 escaping adds a backslash before every special character (.-()_etc.), inflating formatted length 10-20% above raw. A 3994-char raw chunk becomes 4200+ afterformat_message(), triggering a Telegram 400 on the edit.Fix:
_safe_limit = max(500, int(_raw_limit * 0.75))at 3072 chars, leaving real headroom for escape inflation.Bug 2: Cursor stuck in split messages (
stream_consumer.py)When an overflow split fires, the first chunk cursor-removal edit can fail and set
_edit_supported = False. All subsequent edits including the final cursor-removal on the second chunk are then silently skipped. Both messages keep the cursor artifact permanently.Fix: bypass
_edit_supportedfor cursor-removal operations. Calladapter.edit_message()directly with a single retry instead of routing through_send_or_edit().Bug 3: metadata not forwarded in edit path (
stream_consumer.py)_send_or_editcalledadapter.edit_message()withoutmetadata=self.metadata, sothread_idwas unavailable. Overflow chunks landed in the main chat instead of the correct supergroup topic.Fix: pass
metadata=self.metadataat alledit_messagecall sites instream_consumer.Bug 4:
edit_messageoverflow truncates instead of chunking (telegram.py)The
message_too_longhandler inedit_message()cut content to 4076 chars plus...and dropped the rest silently. This loses content on any long streaming response.Fix: split content into 80%-limit chunks via
truncate_message(). Edit the first chunk in place, then send remaining chunks as new messages withmessage_thread_id=effective_thread_idto keep them in the correct topic. Theedit_messagesignature gains an optionalmetadataparam to makethread_idavailable.Bonus: normalize
\r\nline endings informat_message()(telegram.py)API responses from Close.com, GHL, and other external systems frequently use Windows (
\r\n) or old Mac (\r) line endings. Telegram renders\ras a visible character, producing garbled output.Fix: one-liner normalize at the top of
format_message().Testing
All four bugs were reproduced and fixed in a live Falcon HQ supergroup with forum topics enabled. The safe_limit change is conservative at 3072 chars, which gives consistent clean splits without MDv2 inflate errors.