Skip to content

Anthropic streaming usage drops input/cache tokens from message_delta #2290

Description

@CodePrometheus

Description:

Anthropic Messages streaming usage extraction drops input/cache token fields when they are reported on message_delta.

Some Anthropic-compatible streaming backends report the final usage only on the message_delta event, for example:

{
  "type": "message_delta",
  "usage": {
    "input_tokens": 4522,
    "cache_creation_input_tokens": 4511,
    "output_tokens": 5
  }
}

Current AI Gateway streaming handling reads message_start.usage for input/cache tokens, but for message_delta.usage it only updates output_tokens.

As a result, AI Gateway records usage like:

input_tokens = 0
cache_creation_input_tokens = 0
cache_read_input_tokens = 0
output_tokens = 5
total_tokens = 5

The expected behavior is to treat message_delta.usage as a valid source of final usage fields and merge the explicit fields into token usage:

raw input_tokens = 4522
cache_creation_input_tokens = 4511
output_tokens = 5
input total = 9033
total_tokens = 9038

This affects internal token usage accounting, metrics, and any downstream cost or quota logic that depends on AI Gateway token usage.

Repro steps:

Use an Anthropic Messages streaming response where usage is emitted on message_delta.

Minimal sanitized SSE response body:

event: message_delta
data: {"type":"message_delta","delta":{"stop_reason":"end_turn","stop_sequence":null},"usage":
{"input_tokens":4522,"output_tokens":5,"cache_creation_input_tokens":4511}}

event: message_stop
data: {"type":"message_stop"}

A cache-read variant:

event: message_delta
data: {"type":"message_delta","delta":{"stop_reason":"end_turn","stop_sequence":null},"usage":
{"input_tokens":4522,"output_tokens":5,"cache_read_input_tokens":4511}}

event: message_stop
data: {"type":"message_stop"}

Expected result:

input/cache fields from message_delta.usage are included in TokenUsage.

Actual result:

Only output_tokens from message_delta.usage is included.
input_tokens and cache_*_input_tokens are ignored.

Environment:

AI Gateway: main branch
Envoy: not Envoy-specific
Request format: Anthropic Messages API, streaming response

Logs:

Sanitized response stream:

event: message_delta
data: {"type":"message_delta","delta":{"stop_reason":"end_turn","stop_sequence":null},"usage":
{"input_tokens":4522,"output_tokens":5,"cache_creation_input_tokens":4511}}

event: message_stop
data: {"type":"message_stop"}

Metadata

Metadata

Assignees

No one assigned

    Labels

    bugSomething isn't working

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions