Repository navigation
model: add perplexity-ai/pplx-embed-v2-late - #5606
Conversation
|
Cc @NohTow |
There was a problem hiding this comment.
How did you generate lock? We have an issue that it's not worked with current uv version #5482. I use compiled version from draft ai pr astral-sh/uv-dev#2107 that's what created in my issue astral-sh/uv#21954
There was a problem hiding this comment.
I ran a plain uv lock (no flags, no cache clearing) with the stock releaseuv 0.12.0 (x86_64-unknown-linux-gnu), after adding the extra and its entry in [tool.uv] conflicts.
There was a problem hiding this comment.
Hm, strange. We had an issue with endless lock
There was a problem hiding this comment.
It took about 20 minutes. As noted in #5482, that was with the existing uv.lock in place, not a cold resolution, which is probably why it finished.
|
@whybe-choi or @NohTow can you reproduce results of this configuration? |
|
I've already posted the reproduction results in the PR description above |
|
Sure, I will submit the results! I've also already finished evaluating the models on MIRACL-VISION and other Korean VDR tasks alongside ViDoRe v3, so I will submit those together as well. |
|
A small note. During the investigation from the Perplexity team it turned out that the difference in scores comes from using the FastPlaid index. Results for 0.6b model:
Brobably more diff comes from fp32 search |
Closes #5602
Checklists
mteb.get_model(model_name, revision)andmteb.get_model_meta(model_name, revision)Results