docker部署结果与sentence-transformers结果相差太大了

[ucas010]

以0.6B向量生成模型为例：
参考：https://modelscope.cn/models/Qwen/Qwen3-Embedding-0.6B/summary
docker run -p 8080:80 -v hf_cache:/data --pull always ghcr.io/huggingface/text-embeddings-inference:1.7.2 --model-id Qwen/Qwen3-Embedding-0.6B

inputs为"hello"
结果相差太大了，
Docker：
[-0.0214787364773,-0.32268136214761,-0.00709057082745,0.0681453923416,.....]
sentence-transformers:
[
-0.0187803003937006,
-0.02243664301931858,
-0.01279719639569521,
-0.041881732642650604,
....]
@g0t4 @pedramr @hzhwcmhf @erjanmx @huybery
辛苦帮忙看下，谢谢

[ucas010]

@afalf @0russwest0 @tastelikefeet
could u pls help me ?

[ucas010]

均是 document_embeddings

[afalf]

看起来sentence-transformers版本的结果是正确的，TEI的问题建议去TEI的github仓库提个issue问一下

[g0t4]

a few things to check first

are you certain both use cases are pulling the same model size, i.e. 0.6B? different model size will produce different embeddings and thus diff cosine similarity results.

my next guess would be there's a difference in how embeddings are extracted... have you tried using the examples in the HF repo for 0.6B? https://huggingface.co/Qwen/Qwen3-Embedding-0.6B#transformers-usage

They list multiple ways to use the model with working code and a sample query+docs and expected output

my own RAG code if it helps

I have a working embeddings/reranker server using 0.6B models... I am beyond happy with the results in real work I throw at it every day...

Here is how I compute embeddings (note I linked to the line for the function w.r.t. pooling which is a candidate area of problems with embeddings extraction)... I derived this code from the HF repo linked above: https://github.com/g0t4/ask-openai.nvim/blob/feat-predictions/lua/ask-openai/rag/lsp/inference/server/qwen3_embeddings.py#L62

Toward the bottom of the embeddings code, I have verification of cosine similarity in the example given in the HF repos for Qwen3-Embedding-0.6B, I always run that on startup just to be sure I didn't change something and break embeddings extraction (or otherwise)... here is the verification scenario (straight from HF repo): https://github.com/g0t4/ask-openai.nvim/blob/feat-predictions/lua/ask-openai/rag/lsp/inference/qwen3/known.py

You'll also find code that uses Qwen3-Reranker-0.6B right next to this:
https://github.com/g0t4/ask-openai.nvim/blob/feat-predictions/lua/ask-openai/rag/lsp/inference/server/qwen3_rerank.py

I can attest my code is producing remarkable results when both are used in combination to index entire codebases (including codebases I know nothing about) and then run semantic queries and find exactly what I want, nearly every time...

I hope this helps! Qwen3-Embeddings/Reranker are amazing models

[ucas010]

thx all
tei said use the of 1.8.3 or later will solve the problem
see the url huggingface/text-embeddings-inference#774