Skip to content

Commit 3ad7006

Browse files
committed
feat: address PR #116 feedback — base_url, expansion_model, env rename, embed fix
Changes based on PR comments: 1. Configurable base_url for OpenAI-compatible APIs (Ollama, vLLM, Azure) - collections.ts: EmbeddingProviderConfig already has base_url field - qmd.ts: now passes base_url and expansion_model from YAML to setEmbeddingConfig - openai-llm.ts: constructor accepts baseURL config 2. Env var rename: QMD_OPENAI_API_KEY takes priority over OPENAI_API_KEY - Avoids conflict with official openai-node SDK (per @alexleach) - Falls back to OPENAI_API_KEY for backwards compatibility 3. generateEmbeddings bypasses LlamaCpp when using OpenAI (per @viniciushsantana) - OpenAI path calls API directly, no local model session needed - Refactored to shared runEmbedding() with pluggable embed/embedBatch fns 4. expandQuery now actually calls OpenAI for query expansion - Was previously returning lex-only fallback when isUsingOpenAI() - Now uses gpt-4o-mini via openaiLLM.expandQuery() 5. README updated with base_url, expansion_model docs Addresses: @alexleach (env naming, base_url), @viniciushsantana (embed fix, expansion_model, base_url YAML wiring)
1 parent 5791764 commit 3ad7006

5 files changed

Lines changed: 100 additions & 47 deletions

File tree

README.md

Lines changed: 5 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -524,17 +524,20 @@ As an alternative to local embedding models, you can use OpenAI's API for faster
524524
embedding:
525525
provider: openai
526526
openai:
527-
api_key: sk-... # Optional, falls back to OPENAI_API_KEY env var
527+
api_key: sk-... # Optional, falls back to QMD_OPENAI_API_KEY or OPENAI_API_KEY env var
528528
model: text-embedding-3-small # Optional, this is the default
529+
expansion_model: gpt-4o-mini # Optional, model for query expansion/reranking
530+
base_url: https://api.openai.com/v1 # Optional, for OpenAI-compatible APIs (Ollama, vLLM, etc.)
529531
```
530532
531533
Benefits:
532534
- **~10x faster** than local CPU inference
533535
- **No GPU required** - works on any machine
534536
- **More reliable** - no local model loading issues
535537
- **Cost:** ~$0.02 per 1M tokens (very cheap)
538+
- **OpenAI-compatible** - works with Ollama, vLLM, Azure, and other compatible APIs via `base_url`
536539

537-
When using OpenAI embeddings, query expansion and reranking are skipped to avoid loading local models.
540+
When using OpenAI embeddings, query expansion and reranking use the OpenAI API instead of local models.
538541

539542
## Installation
540543

bun.lock

Lines changed: 41 additions & 22 deletions
Some generated files are not rendered by default. Learn more about customizing how changed files appear on GitHub.

src/cli/qmd.ts

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -2826,6 +2826,8 @@ if (isMain) {
28262826
openai: {
28272827
apiKey: embeddingYamlConfig.openai?.api_key,
28282828
embedModel: embeddingYamlConfig.openai?.model,
2829+
expansionModel: embeddingYamlConfig.openai?.expansion_model,
2830+
baseURL: embeddingYamlConfig.openai?.base_url,
28292831
},
28302832
});
28312833
}

src/openai-llm.ts

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -106,7 +106,7 @@ export class OpenAIEmbedding implements LLM {
106106

107107
constructor(config: OpenAIConfig = {}) {
108108
this.client = new OpenAI({
109-
apiKey: config.apiKey || process.env.OPENAI_API_KEY,
109+
apiKey: config.apiKey || process.env.QMD_OPENAI_API_KEY || process.env.OPENAI_API_KEY,
110110
baseURL: config.baseURL,
111111
});
112112
this.embedModel = config.embedModel || DEFAULT_EMBED_MODEL;

0 commit comments

Comments
 (0)