Prevent NPE when delta is null in Azure OpenAI streaming #1721
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Azure OpenAI sends streaming chunks without delta content for end-of-stream markers and content filter responses. Added null check to prevent NullPointerException in streaming handler.
geoand
added
the
triage/backport-1.1
This comment has been minimized.
This comment has been minimized.
Status for workflow
|
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Description
Fixes a NullPointerException that occurs in
AzureOpenAiStreamingChatModelwhen Azure OpenAI sends streaming chunks without delta content.Problem
Azure OpenAI streaming responses sometimes include chunks without a delta field. The current code assumes delta is always present, causing NPE at line 237: Cannot invoke "dev.langchain4j.model.openai.internal.chat.Delta.content()" because "delta" is null
Solution
Added null check for
deltaobject before accessing its contentTesting
Tested with streaming requests that previously crashed