Adding reranking API from vLLM endpoints (coherere rank client and jinaai rerank client)

[Swipe4057]

Validations

• I believe this is a way to improve. I'll try to join the Continue Discord for questions
• I'm not able to find an open issue that requests the same enhancement

Problem

The vLLM API now allows deploying different reranking models while supporting endpoints in the format of both Cohere API and Jina AI API. However, when specifying the model name in "reranker": { "name": "cohere", "params": { "model": model_name, the configuration has fixed model names that are allowed for the Cohere provider, but the user can choose any name and any model for inference in vLLM.

It is necessary to remove the model name check or create another reranking provider - vLLM, which would support the current capabilities.

Additional links:

https://docs.vllm.ai/en/stable/getting_started/examples/cohere_rerank_client.html
https://docs.vllm.ai/en/stable/getting_started/examples/jinaai_rerank_client.html
vllm-project/vllm#12376 [Frontend] Rerank API (Jina- and Cohere-compatible API)

Solution

No response

[alice-cohere]

Hi team, I'd like to +1 the issue raised by @Swipe4057 above. The model name check is in place for reranker but also more generally when specifying models to use for chat (in the models section for JSON config files). With the cohere provider, it's currently possible to call only "command-r" and "command-r-plus" there, even if other model names would be valid to call directly via the API.

Could you please remove the model name check for Cohere models? Thank you