Skip to content

Feature Request: support for XLMRobertaModel embeddings #8789

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
4 tasks done
IzzyHibbert opened this issue Jul 31, 2024 · 2 comments
Closed
4 tasks done

Feature Request: support for XLMRobertaModel embeddings #8789

IzzyHibbert opened this issue Jul 31, 2024 · 2 comments
Labels
enhancement New feature or request

Comments

@IzzyHibbert
Copy link

Prerequisites

  • I am running the latest code. Mention the version if possible as well.
  • I carefully followed the README.md.
  • I searched using keywords relevant to my issue to make sure that I am creating a new issue that is not already open (or closed).
  • I reviewed the Discussions, and have a new and useful enhancement to share.

Feature Description

At the moment the project does not support embeddings based on XLMRobertaModel architecture.
Could be very good to extend the support to those.

Motivation

I believe that adding support could be great. Models like BAAI/bge-m3 are showing promising results against proprietary models like OpenAI, see here :
https://towardsdatascience.com/openai-vs-open-source-multilingual-embedding-models-e5ccb7c90f05

Also, they support higher volume of tokens (bge--m3 for instance up to 8192 tokens).

Possible Implementation

No response

@IzzyHibbert IzzyHibbert added the enhancement New feature or request label Jul 31, 2024
@ExtReMLapin
Copy link
Contributor

#8658

@IzzyHibbert
Copy link
Author

Closed as duplicate of #8658. (apologies, did not find it when searching..)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request
Projects
None yet
Development

No branches or pull requests

2 participants