Skip to content

fix(telegram): fix 4 streaming truncation bugs — safe_limit, cursor cleanup, metadata forwarding, overflow chunking#5270

Closed
ZK-Snarky wants to merge 1 commit into
NousResearch:mainfrom
ZK-Snarky:seb/telegram-truncation-fixes
Closed

fix(telegram): fix 4 streaming truncation bugs — safe_limit, cursor cleanup, metadata forwarding, overflow chunking#5270
ZK-Snarky wants to merge 1 commit into
NousResearch:mainfrom
ZK-Snarky:seb/telegram-truncation-fixes

Conversation

@ZK-Snarky
Copy link
Copy Markdown

@ZK-Snarky ZK-Snarky commented Apr 5, 2026

Summary

Four bugs that cause messages to be cut off, garbled, or land in the wrong Telegram topic during streaming. All four affect anyone using supergroup forum topics, which is a growing use case since group_topics skill binding merged.


Bug 1: MDv2 escape inflation blows past 4096 (stream_consumer.py)

The old safe_limit was _raw_limit - len(cursor) - 100 at roughly 3994 chars. MDv2 escaping adds a backslash before every special character (. - ( ) _ etc.), inflating formatted length 10-20% above raw. A 3994-char raw chunk becomes 4200+ after format_message(), triggering a Telegram 400 on the edit.

Fix: _safe_limit = max(500, int(_raw_limit * 0.75)) at 3072 chars, leaving real headroom for escape inflation.


Bug 2: Cursor stuck in split messages (stream_consumer.py)

When an overflow split fires, the first chunk cursor-removal edit can fail and set _edit_supported = False. All subsequent edits including the final cursor-removal on the second chunk are then silently skipped. Both messages keep the cursor artifact permanently.

Fix: bypass _edit_supported for cursor-removal operations. Call adapter.edit_message() directly with a single retry instead of routing through _send_or_edit().


Bug 3: metadata not forwarded in edit path (stream_consumer.py)

_send_or_edit called adapter.edit_message() without metadata=self.metadata, so thread_id was unavailable. Overflow chunks landed in the main chat instead of the correct supergroup topic.

Fix: pass metadata=self.metadata at all edit_message call sites in stream_consumer.


Bug 4: edit_message overflow truncates instead of chunking (telegram.py)

The message_too_long handler in edit_message() cut content to 4076 chars plus ... and dropped the rest silently. This loses content on any long streaming response.

Fix: split content into 80%-limit chunks via truncate_message(). Edit the first chunk in place, then send remaining chunks as new messages with message_thread_id=effective_thread_id to keep them in the correct topic. The edit_message signature gains an optional metadata param to make thread_id available.


Bonus: normalize \r\n line endings in format_message() (telegram.py)

API responses from Close.com, GHL, and other external systems frequently use Windows (\r\n) or old Mac (\r) line endings. Telegram renders \r as a visible character, producing garbled output.

Fix: one-liner normalize at the top of format_message().


Testing

All four bugs were reproduced and fixed in a live Falcon HQ supergroup with forum topics enabled. The safe_limit change is conservative at 3072 chars, which gives consistent clean splits without MDv2 inflate errors.

…leanup, metadata forwarding, overflow chunking

Problem 1 — MDv2 escape inflation blows past 4096
Raw safe_limit of ~3994 chars doesn't account for MDv2 backslash escaping.
Every special char (. - ( ) _ etc.) adds 1 byte; a 3994-char message can
inflate to 4200+ after format_message(). Fix: use 75% of MAX_MESSAGE_LENGTH
as the safe ceiling (3072 chars), leaving real headroom for escaping.

Problem 2 — Cursor stuck in split messages
When an overflow split is triggered, the first chunk's cursor-removal edit
can set _edit_supported=False on failure. All subsequent edits including
the final cursor-removal are then silently skipped — both messages keep ▉.
Fix: bypass _edit_supported for cursor-removal operations; call
adapter.edit_message() directly with a single retry.

Problem 3 — metadata not forwarded in edit path
_send_or_edit called adapter.edit_message() without metadata=self.metadata,
so thread_id was unavailable. Overflow chunks were sent to the main chat
instead of the correct supergroup topic. Fixed at all edit_message call sites
in stream_consumer.

Problem 4 — edit_message overflow truncates instead of chunking
The message_too_long handler in edit_message() truncated to 4076 chars + '…'
and lost the rest of the content. Fix: split into 80%-limit chunks via
truncate_message(), edit first chunk in place, send remaining chunks as new
messages with message_thread_id=effective_thread_id to preserve topic routing.

Bonus: normalize \r\n and bare \r line endings in format_message().
API responses from Close.com, GHL, etc. frequently use Windows line endings;
Telegram renders \r as a visible character producing garbled output.
@trevorgordon981
Copy link
Copy Markdown

Four genuinely distinct bugs, each with a clear repro and a targeted fix. Bug 1 (MDv2 escape inflation blowing past 4096) is a great catch — byte-count vs. rendered-length divergence is exactly the thing that slips through. The 0.75 × raw_limit safety factor is conservative-correct. Bug 2 (cursor stuck after first edit sets _edit_supported=False) is a nasty state-leak. Only live testing called out — would be worth a unit test asserting a 3800-char payload with heavy MDv2-escape chars gets split before the 4096 ceiling, to lock in the safe_limit regression. Proceed.

@alt-glitch alt-glitch added type/bug Something isn't working P1 High — major feature broken, no workaround comp/gateway Gateway runner, session dispatch, delivery platform/telegram Telegram bot adapter labels May 1, 2026
@ZK-Snarky
Copy link
Copy Markdown
Author

Closing — the upstream refactor of gateway/stream_consumer.py (the new _send_or_edit abstraction) absorbed bugs #2 (cursor in split messages) and #4 (overflow chunking). The remaining bugs (#1 safe_limit MDv2 inflation, #3 metadata forwarding in edit path) no longer warrant a 134-line PR on their own. Thanks to @trevorgordon981 for the original review.

@ZK-Snarky ZK-Snarky closed this May 9, 2026
@ZK-Snarky ZK-Snarky deleted the seb/telegram-truncation-fixes branch May 9, 2026 22:37
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

comp/gateway Gateway runner, session dispatch, delivery P1 High — major feature broken, no workaround platform/telegram Telegram bot adapter type/bug Something isn't working

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants