[Feature]: Support for Anthropic Citations API #7970
Description
The Feature
There is a new citations API: https://www.anthropic.com/news/introducing-citations-api.
Unfortunately, it requires a richer message structure, such as this (content, text, type, ...):
messages=[
{
"role": "user",
"content": [
{
"type": "document",
"source": {
"type": "text",
"media_type": "text/plain",
"data": "The grass is green. The sky is blue.",
},
"title": "My Document",
"context": "This is a trustworthy document.",
"citations": {"enabled": True},
},
{"type": "text", "text": "What color is the grass and sky?"},
],
}
]
If you try that with latest litellm, e.g.:
from litellm import completion
completion(
model="vertex_ai/claude-3-5-sonnet-v2@20241022",
messages=[
{
"role": "user",
"content": [
{
"type": "document",
"source": {
"type": "text",
"media_type": "text/plain",
"data": "The grass is green. The sky is blue.",
},
"title": "My Document",
"context": "This is a trustworthy document.",
"citations": {"enabled": True},
},
{"type": "text", "text": "What color is the grass and sky?"},
],
}
],
)
you will get:
Exception: Invalid user message={'role': 'user', 'content': [{'type': 'document', 'source': {'type': 'text', 'media_type': 'text/plain', 'data': 'The grass is green. The sky is blue.'}, 'title': 'My Document', 'context': 'This is a trustworthy document.', 'citations': {'enabled': True}}, {'type': 'text', 'text': 'What color is the grass and sky?'}]} at index 0. Please ensure all user messages are valid OpenAI chat completion messages.
Motivation, pitch
Eh, it's in the post!
Are you a ML Ops Team?
No
Twitter / LinkedIn details
No response