Skip to content

qwen3-vl-reranker-2b deploy issue #66

@newbecoder

Description

@newbecoder

i wanna deploy qwen3-vl-reranker-2b model, and i tried 3 methods:

  1. transformers
  2. vllm local method: https://github.com/QwenLM/Qwen3-VL-Embedding/blob/main/examples/reranker_vllm.ipynb
  3. vllm openai method: https://github.com/vllm-project/vllm/blob/main/examples/pooling/score/vision_rerank_api_online.py

take method 1 as standard answer, i found that method 2 have little difference, but the result of method 3 is exactly wired:all around 0.5X score. anyone can give me some advice?

my query is image, and documents are images

here is my code:

import requests
import json

def rerank(query_image_url, document_image_urls):
    
    documents = {
        "content": []
    }
    
    for url in document_image_urls:
        documents["content"].append({
            "type": "image_url",
            "image_url": {"url": url}
        })
    
    query = {
        "content": [
            {
                "type": "image_url",
                "image_url": {"url": query_image_url}
            }
        ]
    }
    
    payload = {
        "model": "Qwen3-VL-Reranker-2B",
        # "instruction": "Retrieval relevant image or text with user's query",
        "query": query,
        "documents": documents,
    }
    # print(payload)
    
    response = requests.post(
        "http://localhost:8000/v2/rerank",  
        json=payload,
        headers={"Content-Type": "application/json"}
    )
    
    return response.json()

def score(query, documents):
    query_vllm = {
        "content": [
            query
        ]
    }
    documents_vllm = {
        "content": documents
    }

    payload = {
        "model": "Qwen3-VL-Reranker-2B",
        "instruction": "Retrieval relevant image or text with user's query",
        "queries": query_vllm,
        "documents": documents_vllm,
    }
    
    response = requests.post(
        "http://localhost:8000/score",  
        json=payload,
        headers={"Content-Type": "application/json"}
    )
    
    return response.json()

if __name__ == "__main__":
    query_image =  ""
    doc_images = []
    
    result = rerank(query_image, doc_images)
    print("Result :", result)

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions