Skip to content

test(vllm): cover native LoRA wire compatibility - #18643

Closed
DiweiSun wants to merge 1 commit into
ai-dynamo:mainfrom
DiweiSun:diweis/test-vllm-lora-protocol
Closed

DiweiSun wants to merge 1 commit into
ai-dynamo:mainfrom
DiweiSun:diweis/test-vllm-lora-protocol

Conversation

@DiweiSun

Copy link
Copy Markdown
Contributor

Overview

Backfill regression coverage for native runtime LoRA loading fixed by #17862 and its 1.7.0 cherry-pick #18271. The purpose is to catch an incompatible Rust/Python protocol pair in the vLLM image we ship when updating backend dependencies, before running GPU deployment tests.

The underlying fix is vllm-project/vllm#63295. Python had removed tensorizer_config_dict from the positional MessagePack LoRARequest, while vllm-rs 0.32.0 still serialized that slot. The extra null occupied Python's load_inplace: bool position, so runtime adapter loading failed with Expected bool, got None at $[4]. vLLM 0.33.0 restores the matching layout.

Key Changes

Production logic added: 0 lines. One new test file; no workflow or dependency changes.

  • test_vllm_lora_protocol.py starts the installed vllm-rs executable against a local CPU EngineCore protocol peer. The peer decodes the actual add_lora argument bytes using the installed Python LoRARequest definition, rather than a copied schema or a fabricated request.
  • test_native_lora_matches_installed_python_schema checks successful load, adapter visibility, and unload, along with the decoded name, ID, path, and boolean fields. A negative control verifies that the historical extra-null layout is rejected by the same decoder. Assertions use response data and decoded fields; logs are diagnostic only.
  • Reuses ManagedProcess and the shared port allocator. pre_merge, vllm, integration, and gpu_0 markers select the case in the existing vLLM CPU test job (pre_merge and vllm and gpu_0 and not e2e), with no new job or Rust test binary to distribute.

Existing Python-worker LoRA tests exercise the Python path; Dynamo's fake-gRPC LoRA lifecycle tests return before the native Rust-to-Python serialization boundary. This case covers that missing boundary using the packaged producer and consumer together.

Validation

Ran the same pytest file, using the repository's actual conftests, default strict configuration, and the existing CPU-job marker expression:

Runtime Result Observation
Dynamo 1.7.0-rc.1 / vLLM 0.32.0 Fails as expected The native load endpoint returns HTTP 500 after the installed Python schema rejects the real add_lora payload at $[4].
Dynamo 1.7.0-rc.2 / vLLM 0.33.0 Passes Load/list/unload succeeds; the decoded identity, path, and boolean fields match the request.

Both runs used amd64 CPU containers with networking disabled, no GPU, no model downloads, and no model weights. The verification images add pytest plugins already declared in container/deps/requirements.test.txt; the backend and protocol packages remain at their respective RC versions. The passing case takes approximately 1.3 seconds (approximately 5.5 seconds including pytest startup and collection).

python3 -m pytest tests/serve/test_vllm_lora_protocol.py \
  -m 'pre_merge and vllm and gpu_0 and not e2e' -vv

The peer replaces weight loading only after validating the real wire request. This is a protocol-compatibility test; GPU adapter computation remains covered by serving E2E tests. Upstream CI has not yet run for this branch.

Issues and Ongoing Work

Pull Request Checklist
  • I used Dynamo's /pr-checklist skill.

Backfill the installed-package boundary fixed by vLLM 0.33 and
Dynamo ai-dynamo#17862 (ai-dynamo#18271 on release/1.7.0). Decode native add_lora
messages with the installed Python schema before stubbing weight
loading, and check the load/list/unload lifecycle without a GPU.

Signed-off-by: Molly Sun <diweis@nvidia.com>
@copy-pr-bot

copy-pr-bot Bot commented Nov 12, 2026

Copy link
Copy Markdown

This pull request requires additional validation before any workflows can run on NVIDIA's runners.

Pull request vetters can view their responsibilities here.

Contributors can view more details about this message here.

@DiweiSun
DiweiSun temporarily deployed to external_collaborator November 12, 2026 08:08 — with GitHub Actions Inactive
@DiweiSun
DiweiSun temporarily deployed to external_collaborator November 12, 2026 08:08 — with GitHub Actions Inactive
@github-actions github-actions Bot added the test label Nov 12, 2026
@dynamo-ops

Copy link
Copy Markdown
Contributor

Unsigned Commits Detected

The following commits are not GPG-signed and must be signed before CI can run:

Commit Reason
8006e90 unsigned

Please sign your commits and push again. See the GitHub docs on commit signature verification for help.

@github-actions

Copy link
Copy Markdown
Contributor

👋 Hi DiweiSun! Thank you for contributing to ai-dynamo/dynamo.

Just a reminder: The NVIDIA Test Github Validation CI runs an essential subset of the testing framework to quickly catch errors.Your PR reviewers may elect to test the changes comprehensively before approving your changes.

🚀

@github-actions github-actions Bot added the external-contribution Pull request is from an external contributor label Nov 12, 2026
@DiweiSun DiweiSun closed this Nov 12, 2026

This branch was previously deployed

1 inactive deployment
external_collaborator — 8006e907 Deployed Nov 12, 2026 by DiweiSun via ok-to-test #50411
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

external-contribution Pull request is from an external contributor size/L test

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants