Repository navigation
[CI] test: Give TestLlavaServer's llava preprocessing more time - #48619
Merged
nvpohanh merged 1 commit intoNov 6, 2026
Merged
Conversation
TestLlavaServer (test_vision_openai_server_a.py) fails on h100-computable-* 1-gpu-large runners: the server's startup warmup image request hits llava's 10 s REQUEST_TIMEOUT in _process_single_image, warmup returns 500, and the relaunch fails with EADDRINUSE. Launch its server with REQUEST_TIMEOUT=60, as sgl-project#48443 did for test_vision_chunked_prefill, through a new extra_env hook on TestOpenAIMLLMServerBase. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
[by Claude Code]
Motivation
test/registered/vlm/test_vision_openai_server_a.py::TestLlavaServer(lmms-lab/llava-onevision-qwen2-0.5b-ov) fails insetUpClasson1-gpu-largerunners from theh100-computable-*pool. This is the same failure that #48443 fixed fortest_vision_chunked_prefill.py:Seen in two consecutive attempts of the same run on #47493: job 126709198594 (
h100-computable-e-gpu-2) and job 126989270916 (h100-computable-d-gpu-7).In both jobs, and in the scheduled
mainfailure oftest_vision_chunked_prefill.py(job 126928003215), the timeout fires on the server's own startup warmup image request. It comes about 10 s after Uvicorn starts and before any test request. Other VLM servers in the same partition and on the same runner (Gemma-3, InternVL, Kimi-VL, LFM2.5) start fine, because only llava's preprocessor bounds the CPU pool call withREQUEST_TIMEOUT.The test now runs in
extra-a(#47619), so it no longer blocksPR Test Base, but it still fails inPR Test Extraon those runners.Modifications
python/sglang/test/vlm_utils.py:TestOpenAIMLLMServerBasegets anextra_envclass attribute (default{}), passed topopen_launch_server(env=...).popen_launch_servermerges it intoos.environ, so the default leaves every existing test unchanged.test/registered/vlm/test_vision_openai_server_a.py:TestLlavaServersetsextra_env = {"REQUEST_TIMEOUT": "60"}. This is the same value [CI] test: Give test_vision_chunked_prefill's llava preprocessing more time #48443 uses.Test-only change; no runtime code changes. The cause is the fork-context CPU preprocess pool, which starts all
os.cpu_count()workers inside its firstsubmit(); see the measurements below and in this comment on #48372. #48372 caps that pool at 16 workers, which fixes the cause for every llava test; if it lands, this override can be dropped.Accuracy Tests
N/A (test harness only). I checked locally with
popen_launch_serverstubbed:TestLlavaServer.setUpClassnow passesenv={"REQUEST_TIMEOUT": "60"}, andTestLfm2VlServerstill passesenv={}. I have not run the GPU test here; it needs a1-gpu-largeCI runner.Speed Tests and Profiling
No runtime change. Root-cause measurements, on
mainf3532057e1,lmms-lab/llava-onevision-qwen2-0.5b-ov, an AMD EPYC 9554 host withos.cpu_count()= 256, and CPython 3.12.16:REQUEST_TIMEOUTsubmit)SGLANG_CPU_WORKERS=16submitOnce the pool is up, per-image preprocessing takes about 0.57 s. Forking costs about 28-30 ms per worker. With this PR, the first request on a large host still takes about 8-11 s, but it fits within the 60 s timeout.
Checklist
🤖 Generated with Claude Code
CI States
Latest PR Test (Base): ❌ Run #42001182193
Latest PR Test (Extra): ❌ Run #42001182269
Latest PR Test (AMD ROCm 10): ❌ Run #42001182153
Tip
Base CI only covers the core cases, and its workload may not exercise your change. Use
/rerun-testto run the tests your change touches, or copy this prompt for an agent: