Make sglang compat with vllm 0.5.1

[M0gician]

vLLM just updated to version 0.5.1 and its changes in Vision Language Model implementation break current SGLang from initializing any model correctly due to the removal of VisionLanguageConfig

• Vision Language Model adding support for Phi3-Vision, dynamic image size, and a registry for processing model inputs ([Model] Initialize Phi-3-vision support vllm-project/vllm#4986, [Core] Dynamic image size support for VLMs vllm-project/vllm#5276, [Core] Registry for processing model inputs vllm-project/vllm#5214)
  • Notably, it has a breaking change that all VLM specific arguments are now removed from engine APIs so you no longer need to set it globally via CLI. However, you now only need to pass in into the prompt instead of complicated prompt formatting. See more here
  • There is also a new guide on adding VLMs! We would love your contribution for new models!

This minimal patch at least makes vLLM happy and prevent SRT crashes. I've only tested this patch non-vision model (Qwen2) as I don't have any vision models locally.

[merrymercy]

@M0gician It is merged. Thanks for the contribution!