Description:
Anthropic Messages streaming usage extraction drops input/cache token fields when they are reported on message_delta.
Some Anthropic-compatible streaming backends report the final usage only on the message_delta event, for example:
{
"type": "message_delta",
"usage": {
"input_tokens": 4522,
"cache_creation_input_tokens": 4511,
"output_tokens": 5
}
}
Current AI Gateway streaming handling reads message_start.usage for input/cache tokens, but for message_delta.usage it only updates output_tokens.
As a result, AI Gateway records usage like:
input_tokens = 0
cache_creation_input_tokens = 0
cache_read_input_tokens = 0
output_tokens = 5
total_tokens = 5
The expected behavior is to treat message_delta.usage as a valid source of final usage fields and merge the explicit fields into token usage:
raw input_tokens = 4522
cache_creation_input_tokens = 4511
output_tokens = 5
input total = 9033
total_tokens = 9038
This affects internal token usage accounting, metrics, and any downstream cost or quota logic that depends on AI Gateway token usage.
Repro steps:
Use an Anthropic Messages streaming response where usage is emitted on message_delta.
Minimal sanitized SSE response body:
event: message_delta
data: {"type":"message_delta","delta":{"stop_reason":"end_turn","stop_sequence":null},"usage":
{"input_tokens":4522,"output_tokens":5,"cache_creation_input_tokens":4511}}
event: message_stop
data: {"type":"message_stop"}
A cache-read variant:
event: message_delta
data: {"type":"message_delta","delta":{"stop_reason":"end_turn","stop_sequence":null},"usage":
{"input_tokens":4522,"output_tokens":5,"cache_read_input_tokens":4511}}
event: message_stop
data: {"type":"message_stop"}
Expected result:
input/cache fields from message_delta.usage are included in TokenUsage.
Actual result:
Only output_tokens from message_delta.usage is included.
input_tokens and cache_*_input_tokens are ignored.
Environment:
AI Gateway: main branch
Envoy: not Envoy-specific
Request format: Anthropic Messages API, streaming response
Logs:
Sanitized response stream:
event: message_delta
data: {"type":"message_delta","delta":{"stop_reason":"end_turn","stop_sequence":null},"usage":
{"input_tokens":4522,"output_tokens":5,"cache_creation_input_tokens":4511}}
event: message_stop
data: {"type":"message_stop"}
Description:
Anthropic Messages streaming usage extraction drops input/cache token fields when they are reported on
message_delta.Some Anthropic-compatible streaming backends report the final usage only on the
message_deltaevent, for example:{ "type": "message_delta", "usage": { "input_tokens": 4522, "cache_creation_input_tokens": 4511, "output_tokens": 5 } }Current AI Gateway streaming handling reads
message_start.usagefor input/cache tokens, but formessage_delta.usageit only updatesoutput_tokens.As a result, AI Gateway records usage like:
The expected behavior is to treat
message_delta.usageas a valid source of final usage fields and merge the explicit fields into token usage:This affects internal token usage accounting, metrics, and any downstream cost or quota logic that depends on AI Gateway token usage.
Repro steps:
Use an Anthropic Messages streaming response where usage is emitted on
message_delta.Minimal sanitized SSE response body:
A cache-read variant:
Expected result:
Actual result:
Environment:
Logs:
Sanitized response stream: