Feature Request: Support external API embedding providers (OpenAI/Ollama compatible)

[jsty409]

Summary

QMD currently only supports local GGUF models via node-llama-cpp for embeddings. It would be great to support external API-based embedding providers (OpenAI-compatible, Ollama, etc.) as an alternative.

Use Case

Many users already have powerful embedding models running externally:

• Ollama hosting bge-m3 (1024-dim) on a NAS/server
• OpenAI text-embedding-3-small/large
• Self-hosted TEI (Text Embeddings Inference) endpoints

These often provide better embedding quality than the local 300M/0.6B GGUF models, especially for non-English content (CJK languages).

Current Workaround

Users who need high-quality embeddings + reranking currently cannot use QMD because:

1. Local embeddinggemma-300M has limited CJK coverage
2. Qwen3-Embedding-0.6B is better but still weaker than bge-m3 (1024-dim)
3. No way to point QMD at an external embedding API

Proposed Solution

Add an environment variable or config option like:

# Use Ollama OpenAI-compatible API for embeddings
export QMD_EMBED_PROVIDER="openai"
export QMD_EMBED_API_URL="http://192.168.1.13:11434/v1"
export QMD_EMBED_MODEL="bge-m3"
export QMD_EMBED_DIMENSIONS=1024

This would allow QMD to call external embedding APIs while keeping reranking and query expansion local (which is a great design choice).

Environment

• QMD version: 2.1.0
• OS: Ubuntu 24.04 (x64)
• Node: 22.22.0

Thanks for the great tool! The hybrid search + query expansion + reranking pipeline is excellent.

[dickinsc]

Looks like it's being looked at already
#489

[liuhao6741]

Hi — thanks for writing this up; the use case (external OpenAI-compatible embeddings, better CJK, TEI/Ollama on a NAS, etc.) is very clear.

I’m not a qmd maintainer, but for anyone who needs API-based embeddings today while upstream evaluates native support, there is a related project that takes that trade-off by design: seekx — it targets OpenAI-compatible embedding APIs (SiliconFlow, Jina, Ollama /v1, OpenAI, etc.), no local GGUF / GPU requirement for embeddings, hybrid BM25 + vector (RRF), optional rerank/query expansion/HyDE, and Jieba-based tokenization for CJK full-text search.

It’s a different architecture than qmd’s current local node-llama-cpp stack, so it’s not a drop-in replacement — but it may be useful as a workaround or comparison point for this issue’s requirements (dimensions, remote models, non-English corpora) until qmd can expose a QMD_EMBED_* provider mode as you sketched.

Hope this helps the discussion.

[Fengqx2022]

Great feature request! I'd like to add another use case:

SiliconFlow (硅基流动) - Free Embedding API for Chinese Users

SiliconFlow provides free embedding models via OpenAI-compatible API:

• Model: BAAI/bge-m3 (1024-dim, 8192 context)
• Price: Completely FREE (no monthly limits)
• Reranker: bge-reranker-v2-m3 also FREE
• Performance: Better CJK support than embeddinggemma-300M
• API Format: OpenAI-compatible (/v1/embeddings)

Example Config

export QMD_EMBED_PROVIDER="openai"
export QMD_EMBED_API_URL="https://api.siliconflow.cn/v1"
export QMD_EMBED_API_KEY="sk-xxx"
export QMD_EMBED_MODEL="BAAI/bge-m3"
export QMD_EMBED_DIMENSIONS=1024

Why This Matters

• Users in China get faster latency (domestic API vs OpenAI/OpenRouter)
• Better Chinese/Korean/Japanese embedding quality
• Free + unlimited vs OpenAI's paid tier
• No GPU required for embedding - perfect for low-power machines

This would be a huge win for Chinese users who want high-quality embeddings without local GPU resources.

Implementation Note

The existing LLM interface in src/llm.ts is well-designed for this. Just need a new implementation class that calls external APIs instead of node-llama-cpp.

Thanks for considering!

[BigMoonTech]

I would just like to say, I would REALLY love this. Like really really love this.

[mcr-ksh]

+1

[tobi]

Closing as duplicate of #620, which is the better umbrella for OpenAI-compatible generation/embedding/reranking backends.