Skip to content

test(sidecar): isolate vLLM unit contracts and fix regressions - #15089

Closed
JulienDarve wants to merge 12 commits into
jdarve/sidecar-shared-unit-harnessfrom
jdarve/sidecar-isolated-units
Closed

JulienDarve wants to merge 12 commits into
jdarve/sidecar-shared-unit-harnessfrom
jdarve/sidecar-isolated-units

Conversation

@JulienDarve

@JulienDarve JulienDarve commented Sep 19, 2026 •

Copy link
Copy Markdown
Contributor

Summary

Add isolated coverage for shared sidecar configuration, deadlines and typed errors, plus vLLM conversion, discovery metadata, LoRA identity and Worker configuration. Shared production code runs once; new backend units are vLLM-only.

Keep the three supported behavior fixes with their regressions: hide system-only stop reasons, prevent successful handoff from aborted prefill, and consume engine-supplied effective attention block size with legacy fallback.

Second PR in the stack: #14879 → #15089 → #15091. Preserve #14879’s SGLang coverage and all five legacy wire tests until their replacements arrive in #15091. Only vLLM’s before-start cleanup assertions move into isolated units.

Coverage, retained assertions, deviations and historical evidence are recorded in COVERAGE.md, UNITS.md and DEVIATIONS.md.

Validation

At b3ab163, on refreshed foundation 286d6fd:

  • Network-isolated CPU container: all 62 collected units passed (11 common, 51 vLLM), zero failed or ignored.
  • All 102 common/vLLM library cases, 8 shared conformance cases and 8 retained Mocker cases passed. The 62 units are included in the 102 library cases.
  • Targeted Clippy with warnings denied, formatting, ownership checks and commit hooks passed.

Current-head GitHub CI is pending. Stock vLLM 0.29’s effective-block-size producer limitation remains upstream; this PR adds no native-engine tests.

Linear: https://linear.app/nvidia/issue/DIS-2942

Signed-off-by: Julien Darve <jdarve@NVIDIA.com>
Signed-off-by: Julien Darve <jdarve@NVIDIA.com>
Signed-off-by: Julien Darve <jdarve@NVIDIA.com>
Signed-off-by: Julien Darve <jdarve@NVIDIA.com>
Signed-off-by: Julien Darve <jdarve@NVIDIA.com>
Signed-off-by: Julien Darve <jdarve@NVIDIA.com>
Signed-off-by: Julien Darve <jdarve@NVIDIA.com>
Signed-off-by: Julien Darve <jdarve@NVIDIA.com>
Signed-off-by: Julien Darve <jdarve@NVIDIA.com>
Signed-off-by: Julien Darve <jdarve@NVIDIA.com>
@copy-pr-bot

copy-pr-bot Bot commented Sep 22, 2026

Copy link
Copy Markdown

This pull request requires additional validation before any workflows can run on NVIDIA's runners.

Pull request vetters can view their responsibilities here.

Contributors can view more details about this message here.

@github-actions github-actions Bot added backend::vllm Relates to the vllm backend deployment::k8s Relates to dynamo deployment in kubernetes backend::sglang Relates to the sglang backend backend::trtllm Relates to the trtllm backend frontend `python -m dynamo.frontend` and `dynamo-run in=http|text|grpc` multimodal router Relates to routing, KV-aware routing, etc. container xpu labels Sep 22, 2026
@JulienDarve
JulienDarve changed the base branch from jdarve/sidecar-cpu-wire to jdarve/sidecar-shared-unit-harness September 22, 2026 22:23

@JulienDarve JulienDarve left a comment

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@dcoapp recheck

@JulienDarve

Copy link
Copy Markdown
Contributor Author

/ok to test b3ab163

Signed-off-by: Julien Darve <jdarve@NVIDIA.com>
Signed-off-by: Julien Darve <jdarve@NVIDIA.com>
@rmccorm4 rmccorm4 mentioned this pull request Sep 27, 2026
@JulienDarve
JulienDarve removed this pull request from stack #15192 September 28, 2026 22:45
@JulienDarve
JulienDarve force-pushed the jdarve/sidecar-shared-unit-harness branch 2 times, most recently from f6f9063 to 6d572dd Compare September 29, 2026 22:11
@JulienDarve

Copy link
Copy Markdown
Contributor Author

Implemented here instead: #15243

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

actions backend::sglang Relates to the sglang backend backend::trtllm Relates to the trtllm backend backend::vllm Relates to the vllm backend container deployment::k8s Relates to dynamo deployment in kubernetes documentation Improvements or additions to documentation frontend `python -m dynamo.frontend` and `dynamo-run in=http|text|grpc` multimodal router Relates to routing, KV-aware routing, etc. size/XXL test xpu

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant