Skip to content

fix(open_responses): tolerate streaming events missing index fields#243

Merged
davidmigloz merged 3 commits into
mainfrom
open-responses-242
Jun 2, 2026
Merged

fix(open_responses): tolerate streaming events missing index fields#243
davidmigloz merged 3 commits into
mainfrom
open-responses-242

Conversation

@davidmigloz

@davidmigloz davidmigloz commented Jun 2, 2026

Copy link
Copy Markdown
Owner

Summary

  • Fixes a crash when streaming from llama.cpp (and other minimal OpenAI-compatible servers): StreamingEvent.fromJson threw type 'Null' is not a subtype of type 'int', killing the stream.
  • llama.cpp's Responses implementation omits the positional index fields the OpenAPI spec marks as requiredoutput_index, content_index, annotation_index, summary_index — which our hand-written factories were hard-casting as non-nullable int.
  • These fields now default to 0 when absent, so streaming works against servers that don't emit them while remaining unchanged for compliant servers (OpenAI, etc.).
  • Adds llama.cpp to the Supported providers table in the package README.

Details

The fromJson factories in streaming_event.dart previously did:

outputIndex: json['output_index'] as int,   // 💥 throws when the field is null/absent
contentIndex: json['content_index'] as int, // 💥

This now mirrors the handling already in place for sequence_number (which is also spec-required and also omitted by llama.cpp):

outputIndex: json['output_index'] as int? ?? 0,
contentIndex: json['content_index'] as int? ?? 0,

Why default-to-0 instead of making the fields nullable (int?):

  • Non-breakingevent.outputIndex stays int; no consumer null-checks required (nullable would be a major-version change).
  • Consistent — matches the existing sequence_number convention, the most analogous "ordering metadata some providers omit" field.
  • Semantically correct — a single-output server genuinely is at index 0.
  • Minimal — only the fromJson factories change; toJson / copyWith / == / hashCode / toString are untouched because the field types are unchanged, so we still emit spec-conformant output.

Applied across all 27 affected casts (15× output_index, 9× content_index, 1× annotation_index, 2× summary_index). Verified safe for downstream consumers: streaming_event_accumulator, streaming_extensions, and streaming_resource correlate deltas by item_id / event type — never by these indices — so defaulting to 0 has no behavioral side effects.

item_id (required String) is intentionally left as-is: llama.cpp does emit it, and keeping it required preserves type safety where the data is present.

This is a client-robustness fix; the OpenAPI spec is correct and unchanged.

References

  • llama.cpp#18486 — initial /v1/responses support in llama.cpp server (the implementation users are hitting).
  • llama.cpp#19138 — open tracking issue for full Responses-endpoint support.

Test Plan

  • Unit tests pass for open_responses (dart test test/unit/ — 438 passed, 3 env-skipped)
  • New regression tests added: output_text.delta, content_part.added, and output_item.added parsed from llama.cpp-shaped JSON (no sequence_number/output_index/content_index) default to 0 instead of throwing
  • dart format clean; dart analyze introduces no new issues
  • toJson round-trip still verified by existing tests

Closes #242

llama.cpp's Responses implementation omits the spec-required positional
index fields (output_index, content_index, annotation_index,
summary_index), so StreamingEvent.fromJson threw "Null is not a subtype
of int" and the stream died.

Default these fields to 0 in the fromJson factories, mirroring the
existing sequence_number handling. Fields stay non-nullable (int), so
this is non-breaking.

Closes #242
Copilot AI review requested due to automatic review settings June 2, 2026 07:51
@davidmigloz davidmigloz added t:bug Bug report - something isn't working p:open_responses_dart open_responses_dart package labels Jun 2, 2026
@davidmigloz davidmigloz self-assigned this Jun 2, 2026

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Improves open_responses streaming robustness by making StreamingEvent.fromJson tolerate OpenAI-compatible servers (e.g., llama.cpp) that omit spec-required positional index fields, preventing runtime type-cast crashes during streaming.

Changes:

  • Default missing output_index, content_index, annotation_index, and summary_index fields to 0 in streaming event fromJson factories (instead of hard-casting to non-nullable int).
  • Add/adjust inline documentation on affected index fields to clarify defaulting behavior for non-conformant providers.
  • Add regression tests to ensure minimal/llama.cpp-shaped streaming JSON parses successfully and defaults indices/sequence numbers to 0.

Reviewed changes

Copilot reviewed 2 out of 2 changed files in this pull request and generated no comments.

File Description
packages/open_responses/lib/src/models/streaming/streaming_event.dart Makes index parsing null-tolerant by defaulting missing positional indices to 0 to avoid stream crashes.
packages/open_responses/test/unit/models/streaming_event_test.dart Adds regression coverage for minimal-server events missing index fields.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

llama.cpp's llama-server exposes the OpenAI-compatible Responses surface
under /v1; with the missing-index-field fix in this PR the client now
works against it. List it alongside the other local runtimes.

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 3 out of 3 changed files in this pull request and generated 1 comment.

Comment thread packages/open_responses/test/unit/models/streaming_event_test.dart
The fix also made annotation_index (output_text.annotation.added) and
summary_index (reasoning_summary_part.added) default to 0, but those
paths lacked direct regression coverage. Add tests parsing both events
with all index fields omitted (PR review by @Copilot).

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 3 out of 3 changed files in this pull request and generated no new comments.

@davidmigloz
davidmigloz merged commit 5d296e8 into main Jun 2, 2026
10 checks passed
@davidmigloz
davidmigloz deleted the open-responses-242 branch June 2, 2026 10:14
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

p:open_responses_dart open_responses_dart package t:bug Bug report - something isn't working

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Null values for output_index and content_index

2 participants