Skip to content

vLLM Chat Completions model Responses API - OpenRouter workflow #157

Closed

Description

@bxyu-nvidia

Use cases, pain points, and background
OpenRouter is a very popular inference endpoint provider and we should explicitly support this workflow

Description:
There shouldn't be any major changes necessary to use OpenRouter Chat Completions endpoints with VLLMModel. We need to test it out ourselves and write a tutorial on how to do it.

Design:
No code changes necessary, should just be able to point the config to an OpenRouter endpoint.

Out of scope:

Acceptance Criteria:

  • Create an OpenRouter API key
  • Find a model that is free to query on OpenRouter
  • Try using it by collecting rollouts using the MultiNeedle resources server. This will test tool calling and multi step interactions.
  • Tutorial on how to get an OpenRouter API key, how to find a model that is free to query, and steps to collect rollouts using the MultiNeedle resources server

Activity

  1. added
    modelsinfra for different models and inference providers; middleware to standardize to OAI responses
    on Oct 23, 2025
  2. cwing-nvidia commented on May 25, 2026

    @cwing-nvidia
    Contributor

    closing in favor of #1285

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    documentationImprovements to documentationmodelsinfra for different models and inference providers; middleware to standardize to OAI responses

    Type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions