Skip to content

fix(anthropic): fall back to message_start usage when message_delta omits fields - #41743

Open
Hunter Lovell (hntrl) wants to merge 2 commits into
masterfrom
hunter/anthropic-message-start-usage-fallback
Open

Hunter Lovell (hntrl) wants to merge 2 commits into
masterfrom
hunter/anthropic-message-start-usage-fallback

Conversation

@hntrl

Copy link
Copy Markdown
Member

When streaming, ChatAnthropic built usage only from message_delta, so any input or cache field the delta left out reported as 0. Some providers (e.g. some Bedrock deployments) put cache counts only on message_start, and Anthropic's docs show output-only deltas. Streams now keep the message_start usage and fill in any field the delta leaves null; fields the delta does report (e.g. cumulative counts after server tools) still win. message_start still emits no usage_metadata. Mirrors langchain-ai/langchainjs#12145.

Release note

Streamed usage_metadata from ChatAnthropic no longer reports 0 input/cache tokens when message_delta omits those fields.

Tested with new sync and async unit tests (output-only delta, delta without cache fields, full cumulative delta); make lint and unit tests pass.

@github-actions github-actions Bot added anthropic `langchain-anthropic` package issues & PRs fix For PRs that implement a fix integration PR made that is related to a provider partner package integration internal size: S 50-199 LOC labels Nov 7, 2026

@robin-agent robin-agent Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Robin Review found 1 potential issue.

Open in Web

@byhow Baihao Wang (byhow) left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

non-blocking nit, up to you

if message_start_usage is not None:
# Some providers (e.g. Bedrock) omit input/cache fields from
# `message_delta`; fall back to `message_start` for those.
update = {k: v for k, v in event.usage if v is not None}

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

is there a better native way in py? like a js map or something

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

anthropic `langchain-anthropic` package issues & PRs fix For PRs that implement a fix integration PR made that is related to a provider partner package integration internal size: S 50-199 LOC

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants