feat: remote endpoint support for QMD (embed, expand, rerank, generate)

[unithejerk]

feat: remote endpoint support for QMD (embed, expand, rerank, generate)

Summary

This PR adds remote endpoint support across QMD's embed, expand, rerank, and generate paths.

It introduces role-specific endpoint configuration plus protocol adapters for OpenAI-compatible, Cohere, Anthropic, Ollama, and vLLM-style APIs. Remote mode activates only when endpoints are configured, so existing local-only behavior remains the default.

Current branch shape:

• 18 commits
• 41 files changed
• dedicated remote endpoint docs
• remote/runtime, retrieval-gate, local-config, and CLI regression coverage

What it adds

Remote runtime

• src/remote/ module with per-role endpoint configuration for embed, expand, rerank, and generate
• protocol-specific handlers for:
  • OpenAI chat completions
  • OpenAI completions
  • OpenAI responses
  • Anthropic messages
  • Cohere embed / rerank
  • Ollama embed / chat / generate
  • vLLM pooling / score
• circuit breakers, health probing, retry/timeout handling, and HTTP transport helpers
• remote tokenizer support for chunking when local tokenization is unavailable

QMD integration

• remote wiring in src/store.ts, src/llm.ts, src/index.ts, and src/cli/qmd.ts
• qmd status / qmd doctor model and connection reporting updates
• programmatic createStore({ llm }) support preserved for remote/custom LLM injection
• remote-only setups avoid instantiating local LlamaCpp during tokenizer detection

Docs and tests

• docs/remote-endpoints.md with provider roles, formats, examples, tokenizer notes, and troubleshooting
• one-line README.md pointer to the dedicated remote docs
• test/remote.test.ts for adapter/runtime behavior
• test/retrieval-gate.test.ts for retrieval-quality regression gating
• local config and CLI regression coverage for status output and collection-add behavior

Small CLI fix included

• qmd collection add with no path argument now exits with usage guidance instead of implicitly indexing CWD
• fixes #684

Related work

This overlaps with earlier remote-endpoint efforts, but differs in scope and current integration point:

• #705: focuses on OpenAI-compatible endpoints; this PR adds multiple protocol families
• #629: provides earlier scaffolding; this PR carries that forward into adapter/runtime integration
• #603: earlier closed remote-endpoint PR; this PR is rebased onto current main
• #517: adds OpenAI-compatible remote embedding/reranking with a hybrid local/remote split; this PR extends remote support across all four roles

Architecture

flowchart TD
    Q["qmd embed / query"] --> R["RemoteLLM"]
    R --> E["embed.ts → /v1/embeddings, /v2/embed, /pooling, /api/embed"]
    R --> EX["expand.ts → chat / completions / responses / messages / Ollama"]
    R --> RK["rerank.ts → /rerank or /score"]
    R --> G["generate.ts → chat / completions / responses / messages / Ollama"]
    R --> P["probe.ts → metadata / startup checks"]
    R --> T["transport.ts → Node http/https"]
    R --> CB["circuit-breaker.ts → per-role breaker"]
    R --> A["adapters/ → protocol handlers"]

Loading

Validation

Ran on this branch:

• npm run test:types
• node ./node_modules/vitest/vitest.mjs run --reporter=verbose --testTimeout 60000 test/remote.test.ts test/retrieval-gate.test.ts test/local-config.test.ts
• node ./node_modules/vitest/vitest.mjs run --reporter=verbose --testTimeout 60000 test/cli.test.ts test/local-config.test.ts

Most recent focused results:

• 198/198 passed for remote + retrieval-gate + local-config
• 148/148 passed for cli + local-config

Manual / live validation performed during development:

• vLLM embedding endpoint with Qwen3-Embedding-0.6B
• OpenRouter-based expand path
• Cohere rerank via OpenRouter
• local-only QMD behavior checked as unchanged when no remote endpoints are configured

Configuration

Remote mode activates only when endpoint config is present.

YAML (~/.config/qmd/index.yml)

models:
  embed_api_url: http://embed-host:8000/v1
  embed_api_model: Qwen3-Embedding-0.6B
  embed_api_key: sk-...
  embed_api_format: vllm_pooling

  expand_api_url: https://openrouter.ai/api/v1
  expand_api_model: google/gemini-2.0-flash-lite-001
  expand_api_key: sk-or-...
  expand_api_format: openai_chat_completions

  rerank_api_url: https://openrouter.ai/api/v1
  rerank_api_model: cohere/rerank-v3.5
  rerank_api_key: sk-or-...
  rerank_api_format: cohere_v2_rerank

Environment variables

QMD_EMBED_BASE_URL=http://embed-host:8000/v1
QMD_EMBED_MODEL=Qwen3-Embedding-0.6B
QMD_EXPAND_BASE_URL=https://openrouter.ai/api/v1
QMD_EXPAND_MODEL=google/gemini-2.0-flash-lite-001
# ... etc

Precedence:

• env vars
• YAML config
• backward-compat fallbacks such as OPENAI_BASE_URL for embed

See docs/remote-endpoints.md for provider roles, protocol formats, tokenizer behavior, and example setups.

Backward compatibility

• no *_api_url configured: QMD stays local-only
• existing qmd collection add <path> behavior is unchanged
• programmatic consumers can inject a custom llm without losing direct expandQuery() behavior

Out of scope

• broader CLI UX changes such as interactive confirmation / --dry-run for collection add
• additional batching-policy work beyond the current remote embedding support