Skip to content

feat: add OrcaRouter as a named inference provider - #2553

Draft
Marc-oss-hub wants to merge 1 commit into
NVIDIA-NeMo:mainfrom
Marc-oss-hub:feat/orcarouter-provider
Draft

Marc-oss-hub wants to merge 1 commit into
NVIDIA-NeMo:mainfrom
Marc-oss-hub:feat/orcarouter-provider

Conversation

@Marc-oss-hub

Copy link
Copy Markdown

Summary

Adds OrcaRouter as a named provider to the
inference_provider model server, wired exactly like the existing OpenRouter
entry: a ready-made configs/orcarouter.yaml plus rows in the README and Fern
docs provider tables.

I'm an engineer on the OrcaRouter team.

What does this PR do?

  • responses_api_models/inference_provider/configs/orcarouter.yaml (new):
    named config pointing at https://api.orcarouter.ai/v1, mirroring
    openrouter.yaml. Auto-discovered by nemo_gym/model_registry.py, so it's
    selectable via --model-type inference_provider/orcarouter with just
    policy_api_key and policy_model_name set in env.yaml.
  • responses_api_models/inference_provider/README.md: adds OrcaRouter to
    the Supported Providers table.
  • fern/versions/latest/pages/model-server/inference-providers.mdx: adds
    OrcaRouter (base URL + config) to the Supported Providers table.

OrcaRouter is an OpenAI-compatible routing gateway, so the server uses the same
/v1/chat/completions transport as the other inference_provider variants; no
application code changes. The generic inference_provider.yaml remains
available for any unlisted provider.

Checklist

  • I have read the contributing guidelines.
  • The change is focused; no unrelated drive-by edits.
  • Tests N/A for this docs/config-only change (existing model-server tests
    and tests/unit_tests/test_model_registry.py cover discovery mechanics
    and are unaffected).
  • Pre-commit checks pass locally (pre-commit run --all-files): the
    YAML/README/mdx files added are outside the .py-only hooks; ruff,
    end-of-file-fixer, and trailing-whitespace report no findings.
  • Commit has DCO sign-off (git commit -s).

Verification

  • New config parses as valid YAML and resolves to the correct base URL.
  • nemo_gym/model_registry._discover_models_in_dir walks
    configs/*.yaml, so inference_provider/orcarouter is discoverable with no
    registry changes.
  • Live API (same OpenAI-compatible transport the server uses, against the real
    gateway): GET /v1/models returns 200 with 193 models (incl.
    openai/gpt-5.5); POST /v1/chat/completions with an OrcaRouter API key
    returns 200 and the expected completion.
  • Because the provider is a base-URL config, any pipeline that uses it also
    inherits OrcaRouter's gateway-level, zero-trust security controls for AI
    agents, with no application code changes.

Add a ready-made `orcarouter.yaml` config to the `inference_provider`
model server (mirroring the OpenRouter config), plus rows in the
inference-provider README and Fern docs tables.

OrcaRouter is an OpenAI-compatible routing gateway
(https://api.orcarouter.ai/v1). Registering it as a named provider makes
it selectable via `--model-type inference_provider/orcarouter` with just
`policy_api_key` and `policy_model_name` in env.yaml; the generic
`inference_provider.yaml` remains available for unlisted providers.

Verified against the live gateway: /v1/models returns 193 models
(including openai/gpt-5.5), and a /v1/chat/completions call through the
same OpenAI-compatible transport the server uses returned 200.

Co-Authored-By: Claude <noreply@anthropic.com>
Signed-off-by: Marc-oss-hub <sjh668899@outlook.com>
@copy-pr-bot

copy-pr-bot Bot commented Aug 14, 2026

Copy link
Copy Markdown

This pull request requires additional validation before any workflows can run on NVIDIA's runners.

Pull request vetters can view their responsibilities here.

Contributors can view more details about this message here.

@nemo-automation-bot nemo-automation-bot Bot added the community-request Issue reported or requested by someone from the community label Aug 14, 2026
@svcnvidia-nemo-ci svcnvidia-nemo-ci added the waiting-on-maintainers Waiting on maintainers to respond label Aug 16, 2026
@github-actions github-actions Bot added the sla:triage-overdue Review assignment is over the one-business-day SLA label Aug 17, 2026
@svcnvidia-nemo-ci svcnvidia-nemo-ci added area:model Model servers, inference providers, and model adapters feature New capabilities, enhancements, or enablement work labels Oct 20, 2026
@yaoyu-33
yaoyu-33 marked this pull request as draft November 18, 2026 02:15
@yaoyu-33

Copy link
Copy Markdown
Contributor

We’re doing some cleanup of the open PR list. This PR has had no activity for over a month, so I’ve moved it back to draft.

If this change is still needed, please resolve any merge conflicts and mark the PR ready for review again when it’s ready to merge. Thanks!

@github-actions github-actions Bot removed the sla:triage-overdue Review assignment is over the one-business-day SLA label Nov 18, 2026
@svcnvidia-nemo-ci svcnvidia-nemo-ci removed the waiting-on-maintainers Waiting on maintainers to respond label Nov 18, 2026

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

area:model Model servers, inference providers, and model adapters community-request Issue reported or requested by someone from the community feature New capabilities, enhancements, or enablement work

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants