Lacks support for `annotations` field in OpenAI's `chat completion object`

[minhrongcon2000]

Please do a quick search on GitHub issues first, there might be already a duplicate issue for the one you are about to create.
If the bug is trivial, just go ahead and create the issue. Otherwise, please take a few moments and fill in the following sections:

Bug description
Lacks support for annotations field in OpenAI's chat completion object (under chat completion object > message > annotations)
https://platform.openai.com/docs/api-reference/chat/object

Environment
org.springframework.ai:spring-ai-bom:1.0.0-M6

Steps to reproduce

• Probably using web search feature in Chat completion API (https://platform.openai.com/docs/guides/tools-web-search?api-mode=chat)

Expected behavior

Caused by: com.fasterxml.jackson.databind.exc.UnrecognizedPropertyException: Unrecognized field "annotations" (class org.springframework.ai.openai.api.OpenAiApi$ChatCompletionMessage), not marked as ignorable (7 known properties: "tool_call_id", "role", "content", "name", "refusal", "tool_calls", "audio"])
 at [Source: REDACTED (`StreamReadFeature.INCLUDE_SOURCE_IN_LOCATION` disabled); line: 14, column: 8] (through reference chain: org.springframework.ai.openai.api.OpenAiApi$ChatCompletion["choices"]->java.util.ArrayList[0]->org.springframework.ai.openai.api.OpenAiApi$ChatCompletion$Choice["message"]->org.springframework.ai.openai.api.OpenAiApi$ChatCompletionMessage["annotations"])
	at com.fasterxml.jackson.databind.exc.UnrecognizedPropertyException.from(UnrecognizedPropertyException.java:61) ~[jackson-databind-2.18.2.jar!/:2.18.2]
	at com.fasterxml.jackson.databind.DeserializationContext.handleUnknownProperty(DeserializationContext.java:1153) ~[jackson-databind-2.18.2.jar!/:2.18.2]
	at com.fasterxml.jackson.databind.deser.std.StdDeserializer.handleUnknownProperty(StdDeserializer.java:2241) ~[jackson-databind-2.18.2.jar!/:2.18.2]
	at com.fasterxml.jackson.databind.deser.BeanDeserializerBase.handleUnknownProperty(BeanDeserializerBase.java:1821) ~[jackson-databind-2.18.2.jar!/:2.18.2]
	at com.fasterxml.jackson.databind.deser.BeanDeserializerBase.handleUnknownProperties(BeanDeserializerBase.java:1771) ~[jackson-databind-2.18.2.jar!/:2.18.2]
	at com.fasterxml.jackson.databind.deser.BeanDeserializer._deserializeUsingPropertyBased(BeanDeserializer.java:561) ~[jackson-databind-2.18.2.jar!/:2.18.2]
	at com.fasterxml.jackson.databind.deser.BeanDeserializerBase.deserializeFromObjectUsingNonDefault(BeanDeserializerBase.java:1497) ~[jackson-databind-2.18.2.jar!/:2.18.2]
	at com.fasterxml.jackson.databind.deser.BeanDeserializer.deserializeFromObject(BeanDeserializer.java:348) ~[jackson-databind-2.18.2.jar!/:2.18.2]
	at com.fasterxml.jackson.databind.deser.BeanDeserializer.deserialize(BeanDeserializer.java:185) ~[jackson-databind-2.18.2.jar!/:2.18.2]
	at com.fasterxml.jackson.databind.deser.SettableBeanProperty.deserialize(SettableBeanProperty.java:543) ~[jackson-databind-2.18.2.jar!/:2.18.2]
	at com.fasterxml.jackson.databind.deser.BeanDeserializer._deserializeWithErrorWrapping(BeanDeserializer.java:585) ~[jackson-databind-2.18.2.jar!/:2.18.2]
	at com.fasterxml.jackson.databind.deser.BeanDeserializer._deserializeUsingPropertyBased(BeanDeserializer.java:447) ~[jackson-databind-2.18.2.jar!/:2.18.2]
	at com.fasterxml.jackson.databind.deser.BeanDeserializerBase.deserializeFromObjectUsingNonDefault(BeanDeserializerBase.java:1497) ~[jackson-databind-2.18.2.jar!/:2.18.2]
	at com.fasterxml.jackson.databind.deser.BeanDeserializer.deserializeFromObject(BeanDeserializer.java:348) ~[jackson-databind-2.18.2.jar!/:2.18.2]
	at com.fasterxml.jackson.databind.deser.BeanDeserializer.deserialize(BeanDeserializer.java:185) ~[jackson-databind-2.18.2.jar!/:2.18.2]
	at com.fasterxml.jackson.databind.deser.std.CollectionDeserializer._deserializeFromArray(CollectionDeserializer.java:361) ~[jackson-databind-2.18.2.jar!/:2.18.2]
	at com.fasterxml.jackson.databind.deser.std.CollectionDeserializer.deserialize(CollectionDeserializer.java:246) ~[jackson-databind-2.18.2.jar!/:2.18.2]
	at com.fasterxml.jackson.databind.deser.std.CollectionDeserializer.deserialize(CollectionDeserializer.java:30) ~[jackson-databind-2.18.2.jar!/:2.18.2]
	at com.fasterxml.jackson.databind.deser.SettableBeanProperty.deserialize(SettableBeanProperty.java:543) ~[jackson-databind-2.18.2.jar!/:2.18.2]
	at com.fasterxml.jackson.databind.deser.BeanDeserializer._deserializeWithErrorWrapping(BeanDeserializer.java:585) ~[jackson-databind-2.18.2.jar!/:2.18.2]
	at com.fasterxml.jackson.databind.deser.BeanDeserializer._deserializeUsingPropertyBased(BeanDeserializer.java:447) ~[jackson-databind-2.18.2.jar!/:2.18.2]
	at com.fasterxml.jackson.databind.deser.BeanDeserializerBase.deserializeFromObjectUsingNonDefault(BeanDeserializerBase.java:1497) ~[jackson-databind-2.18.2.jar!/:2.18.2]
	at com.fasterxml.jackson.databind.deser.BeanDeserializer.deserializeFromObject(BeanDeserializer.java:348) ~[jackson-databind-2.18.2.jar!/:2.18.2]
	at com.fasterxml.jackson.databind.deser.BeanDeserializer.deserialize(BeanDeserializer.java:185) ~[jackson-databind-2.18.2.jar!/:2.18.2]
	at com.fasterxml.jackson.databind.deser.DefaultDeserializationContext.readRootValue(DefaultDeserializationContext.java:342) ~[jackson-databind-2.18.2.jar!/:2.18.2]
	at com.fasterxml.jackson.databind.ObjectReader._bindAndClose(ObjectReader.java:2125) ~[jackson-databind-2.18.2.jar!/:2.18.2]
	at com.fasterxml.jackson.databind.ObjectReader.readValue(ObjectReader.java:1501) ~[jackson-databind-2.18.2.jar!/:2.18.2]
	at org.springframework.http.converter.json.AbstractJackson2HttpMessageConverter.readJavaType(AbstractJackson2HttpMessageConverter.java:398) ~[spring-web-6.2.2.jar!/:6.2.2]
	... 85 common frames omitted

Minimal Complete Reproducible example
N/A

Ref:

• spring-ai/models/spring-ai-openai/src/main/java/org/springframework/ai/openai/api/OpenAiApi.java

  Lines 1054 to 1063 in 66e8d88

  	public record ChatCompletionMessage(// @formatter:off
  	@JsonProperty("content") Object rawContent,
  	@JsonProperty("role") Role role,
  	@JsonProperty("name") String name,
  	@JsonProperty("tool_call_id") String toolCallId,
  	@JsonProperty("tool_calls")
  	@JsonFormat(with = JsonFormat.Feature.ACCEPT_SINGLE_VALUE_AS_ARRAY) List<ToolCall> toolCalls,
  	@JsonProperty("refusal") String refusal,
  	@JsonProperty("audio") AudioOutput audioOutput) { // @formatter:on

• https://platform.openai.com/docs/api-reference/chat/object

[zmwei666]

The problem is very serious.

[zainlateef]

As a temporary workaround, add this objectMapper to your main class

@SpringBootApplication
public class Application {
    public static void main(String[] args) {
        SpringApplication.run(Application.class, args);
    }

    @Bean
    public ObjectMapper objectMapper() {
        ObjectMapper mapper = new ObjectMapper();
        mapper.configure(DeserializationFeature.FAIL_ON_UNKNOWN_PROPERTIES, false);
        return mapper;
    }
}

[minhrongcon2000]

@zainlateef yeah! I have done that in my project, but I just happened to see it, so I reported it here 😄

[markpollack]

Thanks, just getting to this now. Sorry for the delay.

[markpollack]

Need to see how the annotations field propagates up to the user when using ChatClient

[markpollack]