-
Notifications
You must be signed in to change notification settings - Fork 86
Open
Description
i wanna deploy qwen3-vl-reranker-2b model, and i tried 3 methods:
- transformers
- vllm local method: https://github.com/QwenLM/Qwen3-VL-Embedding/blob/main/examples/reranker_vllm.ipynb
- vllm openai method: https://github.com/vllm-project/vllm/blob/main/examples/pooling/score/vision_rerank_api_online.py
take method 1 as standard answer, i found that method 2 have little difference, but the result of method 3 is exactly wired:all around 0.5X score. anyone can give me some advice?
my query is image, and documents are images
here is my code:
import requests
import json
def rerank(query_image_url, document_image_urls):
documents = {
"content": []
}
for url in document_image_urls:
documents["content"].append({
"type": "image_url",
"image_url": {"url": url}
})
query = {
"content": [
{
"type": "image_url",
"image_url": {"url": query_image_url}
}
]
}
payload = {
"model": "Qwen3-VL-Reranker-2B",
# "instruction": "Retrieval relevant image or text with user's query",
"query": query,
"documents": documents,
}
# print(payload)
response = requests.post(
"http://localhost:8000/v2/rerank",
json=payload,
headers={"Content-Type": "application/json"}
)
return response.json()
def score(query, documents):
query_vllm = {
"content": [
query
]
}
documents_vllm = {
"content": documents
}
payload = {
"model": "Qwen3-VL-Reranker-2B",
"instruction": "Retrieval relevant image or text with user's query",
"queries": query_vllm,
"documents": documents_vllm,
}
response = requests.post(
"http://localhost:8000/score",
json=payload,
headers={"Content-Type": "application/json"}
)
return response.json()
if __name__ == "__main__":
query_image = ""
doc_images = []
result = rerank(query_image, doc_images)
print("Result :", result)
Reactions are currently unavailable
Metadata
Metadata
Assignees
Labels
No labels