Skip to content

📈 fix: Keep the Context Gauge Counting While a Reply Streams - #18487

Draft
danny-avila wants to merge 1 commit into
devfrom
danny-avila/gauge-follows-reply
Draft

danny-avila wants to merge 1 commit into
devfrom
danny-avila/gauge-follows-reply

Conversation

@danny-avila

Copy link
Copy Markdown
Collaborator

Summary

The context gauge stands still for the whole of a reply and jumps to its final figure when the reply ends. Send a long reply and watch the gauge: it takes the count it had when the turn was sent, does not move while hundreds of tokens arrive, and lands on the total at the end.

The gauge only adds the output still arriving, and only uses the reading the server sends as a reply starts, while it considers the turn in progress: the response its usage is bound to has to be the one at the tail of the thread. A turn draws its reply under a placeholder id, and the first run step renames that row to the id the server stores it under. Usage stays bound to the placeholder, because every usage handler binds from the submission, which keeps the placeholder for the handlers that key on it. From the first run step on, no row has the bound id, the turn no longer counts as in progress, and the live figures are ignored until the final frame.

With this change the usage writers are told the id a run step gives the reply and follow the row to it. The reply being written stays the one in progress, so the gauge counts it as it arrives.

Measured on the mock e2e lane through one slow reply (gauge's used count at five points, then after it ended):

before   24   24   24   24   24   -> 417
after    73  126  223  321  418   -> 417

How it works

on_run_step (runId = the reply's stored id)
  nameStepResponse(usage, event, submission)        # core/resumable/events.ts
    usage.nameResponse(runId, submission)
      named: placeholder id -> runId                 # react/stream/usage/handlers.ts
      bindResponse(submission)                       # binds runId, moves the turn's snapshot to it

every later frame
  bindResponse(submission)  ->  named.get(placeholder) ?? placeholder

nameStepResponse is called wherever a run step is handed on: main chat's stream session, for a live step and for the steps replayed once created has named the turn, and the event applier a core Chat runs, so a conversation driven that way keeps the same usage for the same frames. A step that reuses the preliminary id (USE_PRELIM_RESPONSE_MESSAGE_ID) or names none renames nothing.

The submission is left as it is. The final handler, the regenerate merge and the abort paths find the turn's reply by the placeholder id it carries, so the id the row took is kept beside it, in the usage writers, rather than written over it. bindResponse already moved a turn's snapshot when the response id it was given changed; it now sees that change.

Type of change

  • Bug fix

Testing

Reproduced in a real browser before the change, on a conversation just used and on one opened fresh: the gauge read the same number at chunks 5, 20, 60, 100 and 140 of a 160-chunk reply, then the settled total. After the change it rises at each point and ends on the same total.

The new scenario compares two readings taken while the reply is still running (the Stop control is on screen for both) and fails against a build without the change (Expected: > 38, Received: 38). A first version of it waited for the higher reading, which let the end of the reply satisfy it; it passed without the fix, so I replaced it.

The existing usage scenarios (e2e/specs/mock/usage.spec.ts, seven) pass unchanged.

Tested environments/configuration:

  • Chromium via Playwright, mock e2e lane (in-memory MongoDB, fake model), desktop 1440×900.

Automated tests:

  • packages/chat: npx jest (248 suites, 4008 tests), tsc --noEmit. New cases in react/stream/usage/__tests__/handlers.spec.ts (usage follows the reply to the id it is named, through later frames; another turn's placeholder is not affected) and new core/__tests__/nameStepResponse.spec.ts. The handler case fails with the alias removed.
  • client: npx jest src/hooks/SSE src/hooks/Chat/__tests__/useTokenUsage.spec.tsx (12 suites, 496 tests).
  • New scenario @scenario:context-gauge-follows-a-streaming-reply in e2e/specs/mock/scenarios/gauge-follows-reply.spec.ts, run three times.

@danny-avila

Copy link
Copy Markdown
Collaborator Author

@codex review

Head: c79dcda

@chatgpt-codex-connector

chatgpt-codex-connector Bot commented Nov 19, 2026 •

Copy link
Copy Markdown
Codex Review Summary: ✅ Completed

This comment shows the latest Codex review activity on this pull request.

Review Status Commit Review trigger
📝 Code Review ✅ Completed 2026-11-19T03:44:42.504208Z c79dcda Manual request
ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review" or "@codex security review".

Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings.

@chatgpt-codex-connector

Copy link
Copy Markdown

Codex Review: Didn't find any major issues. Nice work!

Reviewed commit: c79dcdad27

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant