Repository navigation
fix(router): write SGLang wide image pads in MM routing tokens - #17048
KrishnanPrash wants to merge 1 commit into
Conversation
Signed-off-by: Krishnan Prashanth <kprashanth@nvidia.com>
| ) | ||
| .unwrap(); | ||
|
|
||
| // SGLang `padding_sequence`: p0 p1 p2 p3 p0 ..., from p0 on each run. |
There was a problem hiding this comment.
This comment merely narrates the sequence shown by the immediately following expected array, while the test name already states that pads restart for each image run. It adds no lasting constraint beyond the executable assertion.
🤖 AI Fix
Remove this redundant comment.
❌ Dynamo PR CI failed — run 39679650390 (attempt 1) on
|
| Framework | 1-GPU arm64 |
|---|---|
| vLLM | ❌ 1 |
Failure details
1 job failed: the vLLM arm64 CPU-only pytest stage was cut off midway by a self-hosted runner container-exec error. No test failed, so this looks like infrastructure.
❌ vllm-runtime / Test cuda13.0, arm64: runner "custom container implementation failed" during pytest
Job: vllm-runtime / Test cuda13.0, arm64 · Failed step: Run CPU-only tests (parallelized) · Logs: gh run view --job 119509197266 -R ai-dynamo/dynamo --log-failed
[gw3] [ 46%] PASSED components/src/dynamo/vllm/tests/test_vllm_unit.py::TestBenchmarkConfig::test_benchmark_config_kwargs_unpack
components/src/dynamo/vllm/tests/test_vllm_unit.py::TestBenchmarkConfig::test_benchmark_operational_controls_reach_scheduler_config
##[error]Executing the custom container implementation failed. Please contact your self hosted runner administrator.
...
❌ pre_merge_cpu: step outcome 'failure'
✅ pre_merge_cpu_e2e: pytest exited 0
⏭️ pre_merge_gpu_parallel: did not run
⏭️ pre_merge_gpu: did not run
##[error]A pytest stage did not record a passing result. See this step's log for the stage.
The pre_merge_cpu pytest run stopped at 46% with every test before that passing. The runner's Kubernetes container hook then reported Executing the custom container implementation failed, so the stage never recorded a result. The later pre_merge_cpu_e2e stage passed, and the GPU stages did not run. No pytest failure or assertion appears, which points to a runner/infrastructure failure. Rerunning the failed jobs will likely be enough.
For agents
{"pr": 17048, "run_id": 39679650390, "run_attempt": 1, "head_sha": "f6235add03388297e8d68843d4a6fbc6776b7d35",
"failures": [{"job": "vllm-runtime / Test cuda13.0, arm64", "job_id": 119509197266, "failed_step": "Run CPU-only tests (parallelized)", "signature": "##[error]Executing the custom container implementation failed (runner container exec aborted pre_merge_cpu pytest at 46%)", "tests": [], "log_cmd": "gh run view --job 119509197266 -R ai-dynamo/dynamo --log-failed"}]}Posted automatically by Devin for run 39679650390. Updated on every full-CI run of this PR.
|
Closing in favor of the DEP #17044 work in #17126, #17082 and sgl-project/sglang#45737. |
Summary
On SGLang main, the KV router never sees the cache of a repeated image, so it can send the repeat to a worker that must compute it again. SGLang now derives four wide pads from the hash that Dynamo forwards and writes them over each image placeholder run (sgl-project/sglang#43043). The frontend still writes one constant pad, so request block hashes never match KV-event block hashes. The SGLang worker now publishes the pad scheme
wide_v1when its engine derives pads from forwarded hashes, and the frontend writes the same four pads, folded as the KV events fold them. A worker that publishes no scheme, such as one on the pinned SGLang 0.5.22, keeps the single pad.Validation
Two
python -m dynamo.sglangworkers (Qwen3-VL-2B) with the KV router on SGLang main5e79333c95, same client script on the parent PR and on this PR:dynamo-llm with mm-routing: 3571 passed. The SGLang protocol test passes on 0.5.22 and on SGLang main.
Related Issues
wide_v1.