Skip to content

🏭 feat: Build Chat Messages From Live and Replayed Events - #18190

Merged
danny-avila merged 4 commits into
devfrom
danny-avila/chat-stream-events
Nov 17, 2026
Merged

danny-avila merged 4 commits into
devfrom
danny-avila/chat-stream-events

Conversation

@danny-avila

@danny-avila danny-avila commented Nov 15, 2026 •

Copy link
Copy Markdown
Collaborator

Summary

A Chat in @librechat/chat only understands one frame. Its default applyEvent merges a final frame's request and response and ignores everything else, so a Chat attached to a running agent generation shows nothing until the turn ends, and a Chat that reattaches mid-run cannot rebuild what was already streamed from the sync snapshot. Main chat applies those frames through the hooks session (useStepHandler, handleSync), which needs the app's hosts and cannot be handed to a Chat.

This adds createEventApplier, a reducer to pass as ChatInit.applyEvent. It folds created, sync, step and final frames into the message list with the same code main chat runs: reduceStep for the graph's step events, and the resume snapshot rebuilt and placed by the function handleSync now also calls. For the same frames, a Chat constructed with it holds the messages main chat holds, and a chat that joins late holds what one that watched from the first frame holds.

Third in a stack, on #18188 (the transport that delivers these frames). The child-thread pane is the next PR and is the first consumer of both.

How it works

The snapshot rebuild was inline in handleSync. It moves to placeResumedResponse in core/resumable/placement.ts, and both paths call it, so the rule that decides whether a snapshot replaces loaded content exists once:

core/resumable/sync.ts   handleSync (hooks session)  ─┐
                                                      ├─> placeResumedResponse  -> mergeResumeMessages
core/apply.ts            createEventApplier (Chat)   ─┘

What the applier does with each frame:

created   name the turn (user message + the reply the turn owns); replay steps held before it
sync      build the submission from the snapshot, replay its run steps, place the response,
          then apply the step events in replayEvents and pendingEvents
step      reduceStep; token writes are held for `schedule` (one write per frame) or written at once
final     drop any held write, merge the final frame, forget the turn
abort | error   write what is held, so a stopped turn keeps every token it streamed

A created frame never rewrites a row the list already holds. A regeneration reuses a saved user turn, and a compaction's frame names the leaf it summarizes up to, with that leaf's author and no text; such a leaf is recognized as an anchor and the turn is applied the way main chat applies a compaction, so the summary starts empty under the leaf. After a snapshot is placed, the placed list is the baseline a regeneration's later steps rebuild from, so the user turn it reconstructed stays. A turn that made no user row settles its response alone at final, as main chat's final handler does, so the frame's request does not blank the row it names.

A created frame carries on only from a reply the turn owns: one under the id the frame names, or under the padded id (<userMessageId>_) a turn draws its reply at before the server names it. Any other reply already under that user message is a saved answer, so the turn is applied as a regeneration, which starts empty beside the saved answer and leaves it in place. A host that draws its optimistic reply under some other id cannot say so yet: applyEvent is not given the turn's identity. Nothing drives a Chat through this applier today, and the followed chat only observes, so that is recorded as a gap rather than patched here. For the same reason a continued or edited reply cannot be told from a regeneration at created (both name an existing user turn with a saved reply), so an observer shows it as a new reply until its final frame; closing that needs the server to name the turn's shape in created, as it does in a resume snapshot.

Two small changes to Chat support it. ChatEventContext.messages now reads the list as it stands, so a write made earlier in the same event (or a deferred token flush) is seen rather than a copy taken when the event began. mergeFinalFrame is exported so the applier settles a turn exactly as Chat does alone.

Frames that are not message content (usage, titles, attachments, steers, activity and reasoning labels, pauses, pending MCP OAuth prompts) are not applied; they still reach onEvent. Step side channels that are not list writes (sandbox start, nested subagent progress, announcements) are reported through onEffect.

Type of change

  • Feature
  • Refactor
  • Tests / tooling / CI

Testing

Tested environments/configuration:

Unit level. Nothing in the app constructs the applier yet; the pane in the next PR does, and is where it is verified against a running server.

Automated tests:

  • Added packages/chat/src/core/__tests__/apply.spec.ts (22 cases): a snapshot onto an empty list, tokens continuing a snapshot, a late joiner ending equal to a first-frame watcher (before and after final), a snapshot replacing a stale copy, an empty snapshot keeping loaded content, replayed and pending step events, a second snapshot after a dropped connection, steps held until created, the optimistic reply replaced in place, a regeneration started beside the saved answer, a compaction's leaf (assistant and user) left as it was, a regeneration snapshot on an empty list keeping its user turn, a compaction settling without blanking its leaf, final settling and clearing the turn, scheduled writes (one write per burst, never over a final frame, flushed on abort and error, dropped on reset), onEffect, and a real Chat following a generation through createGenerationTransport.
  • packages/chat: npx jest src/core (66 suites, 1655 tests) and npx tsc --noEmit.
  • client, against the rebuilt package: npx jest src/hooks/SSE passes (13 suites, 479 tests, run at the top of the stack).
  • npm run static-checks -- --against origin/dev.

Screenshots / recordings

No user-facing change.

Risk / compatibility

handleSync is on main chat's reattach path. The change there is a move, not a rewrite: the same indexes, the same preserve-loaded-content rule and the same editPrefixCleared outcome, now computed in placeResumedResponse. Two debug log lines were folded into one.

ChatEventContext.messages changing from a copy to a live read is observable only to an applyEvent that writes and then reads within one event, which previously saw the list from before its own write.

@danny-avila
danny-avila added this pull request to stack #18259 November 17, 2026 14:36
@danny-avila
danny-avila force-pushed the danny-avila/chat-stream-events branch 2 times, most recently from 713bdb5 to bc50377 Compare November 17, 2026 15:05
@lia-by-librechat lia-by-librechat Bot changed the title 🧮 feat: Apply Stream Events in the Chat Class the Way Main Chat Does 🏭 feat: Build Chat Messages From Live and Replayed Events Nov 17, 2026
@danny-avila

Copy link
Copy Markdown
Collaborator Author

@codex review

@chatgpt-codex-connector

chatgpt-codex-connector Bot commented Nov 17, 2026 •

Copy link
Copy Markdown
Codex Review Summary: ✅ Completed

This comment shows the latest Codex review activity on this pull request.

Review Status Commit Review trigger
📝 Code Review ✅ Completed 2026-11-17T15:56:32.504403Z d9b07a3 Manual request
ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review" or "@codex security review".

Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings.

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: bc50377776

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread packages/chat/src/core/apply.ts Outdated
Comment on lines +63 to +64
if (!message.isCreatedByUser && message.parentMessageId === userMessage.messageId) {
placeholder = message;

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Badge Avoid treating persisted replies as optimistic placeholders

When a live created frame names a user turn that already has a saved answer, such as when observing a regeneration, this parent-only check selects that saved answer as the placeholder. The returned submission also sets isRegenerate: false, so reduceStep seeds the new response from the existing trailing assistant message and shouldRemoveInitialResponse removes the saved sibling. Consequently, new text deltas append to the previous answer's content instead of starting a fresh response, and the displayed answer remains incorrect until an authoritative snapshot or final frame arrives. Only adopt a generation-owned optimistic placeholder; a shared user parent does not establish that ownership.

Useful? React with 👍 / 👎.

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Valid. Fixed in b3450a9c56. A created frame now carries on only from a reply the turn owns: one under the id the frame names, or under the padded id (<userMessageId>_) a turn draws its reply at before the server names it. Any other reply already under that user message is a saved answer, so the turn is applied as a regeneration: it starts empty beside the saved answer and leaves it in place. Pinned by starts a regeneration beside the saved answer instead of carrying on from it in apply.spec.ts, which fails against the previous code (1 failed, 17 passed) and passes now (1081 in core/__tests__).

One limit, stated in the PR's gap list rather than patched here: a host that draws its own optimistic reply under some other id has no way to say so yet, because applyEvent is not given the turn's identity. Nothing drives a Chat through this applier today; the followed chat only observes.

@danny-avila
danny-avila force-pushed the danny-avila/chat-stream-events branch from bc50377 to b3450a9 Compare November 17, 2026 15:33
@danny-avila

Copy link
Copy Markdown
Collaborator Author

@codex review

Head b3450a9c56.

@chatgpt-codex-connector

Copy link
Copy Markdown

Codex Review: Didn't find any major issues. More of your lovely PRs please.

Reviewed commit: b3450a9c56

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Base automatically changed from danny-avila/chat-reattach-transport to dev November 17, 2026 15:37
@danny-avila
danny-avila force-pushed the danny-avila/chat-stream-events branch from b3450a9 to 457a694 Compare November 17, 2026 15:38
@danny-avila

Copy link
Copy Markdown
Collaborator Author

@codex review

Head 457a694 (rebased onto dev after #18188 merged; same change as b3450a9c56).

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 457a694380

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread packages/chat/src/core/apply.ts Outdated
*/
function createdSubmission(data: ChatCreatedFrame, messages: TMessage[]): EventSubmission {
const loaded = messages.find((message) => message.messageId === data.message.messageId);
const userMessage = { ...loaded, ...data.message, isCreatedByUser: true } as TMessage;

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Badge Preserve the existing anchor when applying a compaction

When observing a live compaction, the created frame's message is the existing branch leaf, not a new user turn: BaseClient.getCompactionAnchor() returns that leaf's identity with empty text and its original author. This merge overwrites the loaded leaf's text and unconditionally marks it as user-authored, and created() immediately writes the altered row. An assistant leaf therefore becomes a user message while its content still contains the saved answer; a user leaf loses its text. Preserve the loaded anchor and carry the compaction semantics rather than treating every created message as a fresh user turn.

Useful? React with 👍 / 👎.

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Valid. Fixed in d00077c. A created frame no longer rewrites a row the list already holds: a regeneration reuses a saved user turn, and a compaction's frame names the leaf it summarizes up to, with that leaf's author and no text. A named leaf is recognized as an anchor (it is not user-authored, or the frame carries no text for it), and the turn is then applied the way main chat applies a compaction: regenerate-shaped with compact, so the summary starts empty under the leaf and the leaf stays as it was. Pinned by leaves the leaf a compaction names as it is and starts the summary under it and keeps the text of a user leaf a compaction names; both fail against the previous code.

Comment on lines +234 to +235
ctx.write(placement.messages);
settle(syncStreamMessage(stream, placement.response));

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Badge Retain the snapshot's user turn in regeneration writes

When a chat joins a regeneration with an empty message list, a snapshot with isRegenerate: true initially places both the reconstructed user message and response here, but next.messages remains the empty baseline returned by buildSubmissionFromResumeState(). On the next text delta, getEventMessages() selects that baseline for regenerations, and flushResponses() writes only the response because ensureUserMessagePresent() explicitly skips regenerations. The user turn consequently disappears and the streamed response has a missing parent until the final frame arrives. Include the reconstructed user turn in the regeneration baseline used by subsequent steps.

Useful? React with 👍 / 👎.

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Valid. Fixed in d00077c: after a snapshot is placed, the placed list becomes the submission's baseline, so a regeneration's later steps, which rebuild from that baseline, keep the user turn the snapshot reconstructed. Pinned by keeps the user turn a regeneration snapshot placed on an empty list, which fails against the previous code.

@danny-avila

Copy link
Copy Markdown
Collaborator Author

@codex review

Head d00077c.

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: d00077c88e

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread packages/chat/src/core/apply.ts Outdated
} else if (event.type === 'final') {
/** The final frame is authoritative; a held copy of the stream must not land on top of it. */
drop();
mergeFinalFrame(event.data, ctx);

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Badge Preserve the compaction anchor when merging the final frame

When a compaction runs on a user-authored leaf, its final frame still erases that leaf's text. BaseClient.getCompactionAnchor() supplies the leaf's identity with text: '', and api/server/controllers/agents/request.js sends that anchor as the final requestMessage. Although createdSubmission() now preserves the loaded anchor, this unconditional mergeFinalFrame() spreads the empty request text onto it when compaction finishes. Main chat avoids this through its regeneration-shaped final handling, which merges only the response. For a compact submission, preserve the existing anchor here too and merge only the summary response.

Useful? React with 👍 / 👎.

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Valid. Fixed in d9b07a3. A turn that made no user row settles its response alone, as main chat's final handler does: the request a final frame names is then merged onto nothing. That covers a regeneration (its saved turn) and a compaction (its anchor), known from the turn the applier built; a chat that saw only the final frame recognizes the anchor from the frame, by the same rule the created path uses. Pinned by settles a compaction on a user leaf without blanking the leaf, which covers both and fails with either half removed.

I also read the applier for the rest of this class, a frame's identity-only message written onto a row the list holds. The four places a frame meets the list are created (fixed in d00077c), sync (the placement and submission code main chat runs, which already recognize an anchor), steps (main chat's reducer) and final (this commit). One case is not decidable from the frames: a continued or edited reply names an existing user turn and an existing reply, and nothing in created tells it from a regeneration, so an observer shows it as a new reply until its final frame. That needs the server to name the turn's shape in created, as it does in a resume snapshot; it is in the PR description's limits and not patched here, since nothing observes such a turn through this applier today.

@danny-avila

Copy link
Copy Markdown
Collaborator Author

@codex review

Head d9b07a3.

@chatgpt-codex-connector

Copy link
Copy Markdown

Codex Review: Didn't find any major issues. Breezy!

Reviewed commit: d9b07a35d5

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

@danny-avila
danny-avila marked this pull request as ready for review November 17, 2026 16:10
@danny-avila
danny-avila merged commit 2d6e455 into dev Nov 17, 2026
59 of 60 checks passed
@danny-avila
danny-avila deleted the danny-avila/chat-stream-events branch November 17, 2026 16:10
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant