Skip to content

feat(providers): add openai-compatible embeddings support#2624

Closed
badgerbees wants to merge 1 commit into
sipeed:mainfrom
badgerbees:feature/vllm-embeddings
Closed

feat(providers): add openai-compatible embeddings support#2624
badgerbees wants to merge 1 commit into
sipeed:mainfrom
badgerbees:feature/vllm-embeddings

Conversation

@badgerbees

Copy link
Copy Markdown
Contributor

📝 Description

Add OpenAI-compatible embeddings support to the provider layer so vLLM-style endpoints can serve embeddings without forwarding dimensions upstream. The new provider path sends model and input, truncates vectors locally when a smaller width is requested, and preserves existing extra_body settings except for dimensions.

🗣️ Type of Change

  • 🐞 Bug fix (non-breaking change which fixes an issue)
  • ✨ New feature (non-breaking change which adds functionality)
  • 📖 Documentation update
  • ⚡ Code refactoring (no functional changes, no api changes)

🤖 AI Code Generation

  • 🤖 Fully AI-generated (100% AI, 0% Human)
  • 🛠️ Mostly AI-generated (AI draft, Human verified/modified)
  • 👨‍💻 Mostly Human-written (Human lead, AI assisted or none)

🔗 Related Issue

N/A

📚 Technical Context (Skip for Docs)

  • Reference URL: N/A
  • Reasoning: The provider stack already handled chat and streaming for OpenAI-compatible backends, but it had no embeddings seam. This change adds EmbeddingProvider, wires it through the HTTP provider facade, and implements /embeddings calls in the shared OpenAI-compatible client.

🧪 Test Environment

  • Hardware: PC
  • OS: Windows
  • Model/Provider: OpenAI-compatible provider tests (httptest)
  • Channels: N/A

📸 Evidence (Optional)

Click to view Logs/Screenshots

Validation passed:

  • go test ./pkg/providers/...

☑️ Checklist

  • My code/docs follow the style of this project.
  • I have performed a self-review of my own changes.
  • I have updated the documentation accordingly.

Copilot AI review requested due to automatic review settings April 22, 2026 13:56
@CLAassistant

CLAassistant commented Apr 22, 2026

Copy link
Copy Markdown

CLA assistant check
All committers have signed the CLA.

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Adds OpenAI-compatible embeddings support to the provider layer, enabling OpenAI-compat backends (e.g., vLLM-style endpoints) to serve embeddings without forwarding dimensions upstream, while still allowing callers to request a smaller output width via local truncation.

Changes:

  • Introduces a new EmbeddingProvider interface in pkg/providers.
  • Implements /embeddings calls (query + batch) in openai_compat.Provider, omitting dimensions from the upstream request and truncating vectors client-side when requested.
  • Wires embedding methods through the httpapi HTTP provider facade and adds httptest coverage for the new behavior.

Reviewed changes

Copilot reviewed 5 out of 5 changed files in this pull request and generated 1 comment.

Show a summary per file
File Description
pkg/providers/types.go Adds EmbeddingProvider interface for embeddings query/batch.
pkg/providers/openai_compat/embeddings.go Implements embeddings request/response handling and local truncation helper.
pkg/providers/openai_compat/embeddings_test.go Adds tests for omitting upstream dimensions and truncation/error behavior.
pkg/providers/httpapi_facade.go Adds compile-time assertion that HTTPProvider satisfies EmbeddingProvider.
pkg/providers/httpapi/http_provider.go Delegates EmbedQuery/EmbedBatch to the OpenAI-compat provider implementation.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Comment thread pkg/providers/openai_compat/embeddings_test.go
@sipeed-bot sipeed-bot Bot added type: enhancement New feature or request domain: provider go Pull requests that update go code labels Apr 22, 2026
@github-actions

Copy link
Copy Markdown

This PR has had no activity for 7 days and has been marked as stale. If you are still working on it, please push an update or leave a comment; otherwise it will be closed automatically in 7 days.

@github-actions

github-actions Bot commented May 6, 2026

Copy link
Copy Markdown

This PR has been closed after 14 days of inactivity. If you would like to continue, feel free to reopen it or submit a new PR.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

domain: provider go Pull requests that update go code stale type: enhancement New feature or request

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants