[Feature]: Adding Lora on visual blocks of multimodal models #26422
Description
🚀 The feature, motivation and pitch
Hello, I have trained Qwen 2.5 VL 7B on all multimodal layers with Lora but when I try to load my adapter with vLLM I have :
Regarding multimodal models, vLLM currently only supports adding LoRA to language model
Do you know when LoRA will be added to visual layers also ?
Alternatives
No response
Additional context
No response
Before submitting a new issue...
- Make sure you already searched for relevant issues, and asked the chatbot living at the bottom right corner of the documentation page, which can answer lots of frequently asked questions.