Skip to content

feat(litellm): block compromised proxy versions on startup - #1692

Closed
kajalj22 wants to merge 3 commits into
mainfrom
feat/litellm-proxy-version-check
Closed

kajalj22 wants to merge 3 commits into
mainfrom
feat/litellm-proxy-version-check

Conversation

@kajalj22

Copy link
Copy Markdown
Contributor

Summary

Adds a startup version check to LiteLLMModelServer that blocks known-compromised LiteLLM proxy versions.

  • On startup, calls GET /health on the configured proxy and checks litellm_version
  • Hard-fails if v1.82.7 or v1.82.8 is detected (supply chain compromise, March 2026)
  • Logs a non-blocking warning if the proxy doesn't expose version info

Context

Companion to #990. LiteLLM v1.82.7 and v1.82.8 contained credential-harvesting malware injected via a PyPI supply chain attack (~40 min window, March 24 2026). The malware exfiltrated env vars, SSH keys, and cloud credentials from the host running the proxy. See: https://docs.litellm.ai/blog/security-update-march-2026

Since LiteLLMModelServer makes real HTTP calls to a LiteLLM proxy at runtime, we have no control over which version the proxy operator runs. This check gives an early, loud failure rather than silently routing traffic through a compromised proxy.

Test plan

  • Existing tests still pass (mocked, no real proxy needed)
  • Add test: compromised version → startup raises RuntimeError
  • Add test: safe version → startup proceeds normally
  • Add test: unreachable health endpoint → warning logged, startup proceeds

🤖 Generated with Claude Code

imxj and others added 3 commits April 6, 2026 18:52
LiteLLM proxies may return chat.completion format or hybrid response
objects when called via /v1/responses. This normalizes those responses
to the expected Responses API shape so downstream NeMoGymResponse
validation succeeds. Also fixes reasoning.effort="none" validation
error for native Responses API responses.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Signed-off-by: Jin Xu <jinx@nvidia.com>
Revert openai_model/app.py to its clean state and move the LiteLLM
response normalization logic into a new responses_api_models/litellm_model
that extends SimpleModelServer. This keeps proxy-specific workarounds
(reasoning.effort="none" fix, chat.completion->response normalization)
isolated from the native OpenAI model server.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Signed-off-by: Jin Xu <jinx@nvidia.com>
Adds a startup HTTP check against the LiteLLM proxy's /health endpoint
to detect and hard-fail on v1.82.7 and v1.82.8, which contained
credential-harvesting malware (supply chain incident, March 2026).

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Signed-off-by: Kajal Jain <kajalj@nvidia.com>
@copy-pr-bot

copy-pr-bot Bot commented Jun 23, 2026

Copy link
Copy Markdown

This pull request requires additional validation before any workflows can run on NVIDIA's runners.

Pull request vetters can view their responsibilities here.

Contributors can view more details about this message here.

@kajalj22 kajalj22 closed this Jun 23, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants