[Version: 4.49.0] Qwen2.5-VL is not supported in vLLM because of transformers #36292

usun1997 · 2025-02-20T01:44:22Z

System Info

error: Model architectures Qwen2 5 VLForConditionalGeneration' failed to be inspected. Please check the logs for more details.

right now people say that they are using methods like:

pip install --upgrade git+https://github.com/huggingface/transformers.git@336dc69d63d56f232a183a3e7f52790429b871ef ([Bug]: Qwen2.5-VL broke due to transformers upstream changes vllm-project/vllm#13285)
pip install --force-reinstall git+https://github.com/huggingface/transformers.git@9985d06add07a4cc691dc54a7e34f54205c04d40 ([Bug] ValueError: Model architectures ['Qwen2_5_VLForConditionalGeneration'] failed to be inspected. Please check the logs for more details. vllm-project/vllm#12932)
There is a breaking change in transformers dev. You need to update vLLM to latest dev and also redownload the HF model repo. ([Bug] ValueError: Model architectures ['Qwen2_5_VLForConditionalGeneration'] failed to be inspected. Please check the logs for more details. vllm-project/vllm#12932)

I mean. This is not supposed to be like this. I can't connect to your github without a vpn, and with a vpn, I can't connect to my workspace. Could transformers team just fix the problem instead of letting people solve it by some weird method. Thanks!

Who can help?

No response

Information

The official example scripts
My own modified scripts

Tasks

An officially supported task in the examples folder (such as GLUE/SQuAD, ...)
My own task or dataset (give details below)

Reproduction

use Xinference, download the newest vLLM so that you get transformers 4.49.0. You download Qwen2.5 VL and deploy in vLLM and get the error:

Expected behavior

NO MORE ERROR DURING DEPLOYMENT OF QWEN2.5 VL 7B IN VLLM

The text was updated successfully, but these errors were encountered:

glamourzc · 2025-02-20T09:00:55Z

me too.

hayreenlee · 2025-02-20T10:58:45Z

me too.

yynickyeh · 2025-02-20T12:11:22Z

me too.

ppkliu · 2025-02-20T12:13:35Z

mee too

Rocketknight1 · 2025-02-20T13:58:00Z

cc @zucchini-nlp, but it would help a lot if someone could post the logs/error to help us figure out what's going on here!

zucchini-nlp · 2025-02-20T14:10:16Z

Hey all! The issue is being fixed on vllm side with vllm-project/vllm#13592 afaik, the team will check compatibility with v4.49 release

cc @Isotr0py

Isotr0py · 2025-02-20T14:58:27Z

That's because the qwen2.5-vl implementation in vllm 0.7.2 (the latest release, not latest commit) is still trying import Qwen2_5_VLImageProcessor which has been removed in transformers 4.49.0 release.

vLLM team is planning to make a new release including the corresponding fix (vllm-project/vllm#13286), perhaps today or tomorrow.

ywang96 · 2025-02-20T19:46:24Z

Hello! We have released vllm==0.7.3 - Although vllm-project/vllm#13602 didn't make it to the release, we have confirmed that our docker image and PyPI package both have transformers==4.49.0 installed and should be compatible with it!

igorpereirabr1 · 2025-02-20T20:56:08Z

Still not working after upgrate to vllm==0.7.3

ywang96 · 2025-02-20T21:06:38Z

Still not working after upgrate to vllm==0.7.3

Can you try specifying the huggingface ID (e.g, Qwen/Qwen2.5-VL-3B-Instruct) directly?

igorpereirabr1 · 2025-02-20T21:55:07Z

Still not working after upgrate to vllm==0.7.3

Can you try specifying the huggingface ID (e.g, Qwen/Qwen2.5-VL-3B-Instruct) directly?

Unfortunately, I can't.. my GPU cluster do not have access to huggingface.. but I've the latest version of the model stored in this path.. .

usun1997 · 2025-02-21T01:02:15Z

Updates:

I managed to use the mentioned "weird" method to solve the case by using:

pip install --force-reinstall git+https://github.com/huggingface/transformers.git@9985d06add07a4cc691dc54a7e34f54205c04d40

Do you know why it can be solved? Transformers 4.49.0dev version is helpful. Here, the version 4.49.0 dev can support Qwen2.5-VL-7B, but "pip install transformers --upgrade" to get 4.49.0 version does not help.

usun1997 · 2025-02-21T01:05:28Z

Hey all! The issue is being fixed on vllm side with vllm-project/vllm#13592 afaik, the team will check compatibility with v4.49 release

cc @Isotr0py

GOOD NEWS

gouqi666 · 2025-02-21T03:49:22Z

@usun1997 , hello , it still doesn't work, could u share your version?
my library version:
transformers 4.49.0.dev0
transformers-stream-generator 0.0.5
triton 3.1.0
trl 0.15.1
typeguard 4.4.1
typer 0.15.1
typing_extensions 4.12.2
tyro 0.9.13
tzdata 2025.1
uc-micro-py 1.0.3
urllib3 2.3.0
uvicorn 0.34.0
uvloop 0.21.0
virtualenv 20.29.1
vllm 0.7.3
watchfiles 1.0.4
websockets 14.2
Werkzeug 3.1.3
wheel 0.45.1
wrapt 1.17.2
xattr 1.1.4
xformers 0.0.28.post3
xgrammar 0.1.11
xxhash 3.5.0
yarl 1.18.3
zipp 3.21.0
zstandard 0.23.0

balachandarsv · 2025-02-21T09:51:46Z

Check your config.json file, and change the processor name in preprocessor_config.json

"image_processor_type": "Qwen2VLImageProcessor",

This should work for most cases if you have latest transformers

igorpereirabr1 · 2025-02-21T13:35:13Z

pip install --force-reinstall git+https://github.com/huggingface/transformers.git@9985d06add07a4cc691dc54a7e34f54205c04d40

still not working for me.. Same error: ValueError: Model architectures ['Qwen2_5_VLForConditionalGeneration'] failed to be inspected. Please check the logs for more details.

PocketBrain · 2025-02-27T08:14:28Z

Has it been fixed? I still have the error.

koleshjr · 2025-02-27T16:04:02Z

@gouqi666 we meet again :) , have you found a fix for this issue?

igorpereirabr1 · 2025-02-27T19:46:54Z

Has it been fixed? I still have the error.

Still not working for me...

gouqi666 · 2025-02-28T08:26:12Z

haha,  change  Qwen2_5_VL to Qwen2_VL in the config.json

…

---Original--- From: "Kolesh ***@***.***> Date: Fri, Feb 28, 2025 00:04 AM To: ***@***.***>; Cc: ***@***.******@***.***>; Subject: Re: [huggingface/transformers] [Version: 4.49.0] Qwen2.5-VL is notsupported in vLLM because of transformers (Issue #36292) @gouqi666 we meet again :) , have you found a fix for this issue? — Reply to this email directly, view it on GitHub, or unsubscribe. You are receiving this because you were mentioned.Message ID: ***@***.***> koleshjr left a comment (huggingface/transformers#36292) @gouqi666 we meet again :) , have you found a fix for this issue? — Reply to this email directly, view it on GitHub, or unsubscribe. You are receiving this because you were mentioned.Message ID: ***@***.***>

koleshjr · 2025-02-28T11:02:45Z

haha, change Qwen2_5_VL to Qwen2_VL in the config.json
…

I think the huggingface model has already been modified. For example for Qwen2.5-vl-7b:

    <html>
    <body>
    <!--StartFragment-->
      | {
    -- | --
      | "min_pixels": 3136,
      | "max_pixels": 12845056,
      | "patch_size": 14,
      | "temporal_patch_size": 2,
      | "merge_size": 2,
      | "image_mean": [
      | 0.48145466,
      | 0.4578275,
      | 0.40821073
      | ],
      | "image_std": [
      | 0.26862954,
      | 0.26130258,
      | 0.27577711
      | ],
      | "image_processor_type": "Qwen2VLImageProcessor",
      | "processor_class": "Qwen2_5_VLProcessor"
      | }
    
    <!--EndFragment-->
    </body>
    </html>

but still I am facing this issue

gouqi666 · 2025-02-28T11:06:06Z

have you updatef your local directory ？

…

---Original--- From: "Kolesh ***@***.***> Date: Fri, Feb 28, 2025 19:03 PM To: ***@***.***>; Cc: ***@***.******@***.***>; Subject: Re: [huggingface/transformers] [Version: 4.49.0] Qwen2.5-VL is notsupported in vLLM because of transformers (Issue #36292) haha,  change  Qwen2_5_VL to Qwen2_VL in the config.json … I think the huggingface model has already been modified. For example for Qwen2.5-vl-7b: <html> <body>    | { -- | --   | "min_pixels": 3136,   | "max_pixels": 12845056,   | "patch_size": 14,   | "temporal_patch_size": 2,   | "merge_size": 2,   | "image_mean": [   | 0.48145466,   | 0.4578275,   | 0.40821073   | ],   | "image_std": [   | 0.26862954,   | 0.26130258,   | 0.27577711   | ],   | "image_processor_type": "Qwen2VLImageProcessor",   | "processor_class": "Qwen2_5_VLProcessor"   | }  </body> </html> but still I am facing this issue — Reply to this email directly, view it on GitHub, or unsubscribe. You are receiving this because you were mentioned.Message ID: ***@***.***> koleshjr left a comment (huggingface/transformers#36292) haha,  change  Qwen2_5_VL to Qwen2_VL in the config.json … I think the huggingface model has already been modified. For example for Qwen2.5-vl-7b: <html> <body>    | { -- | --   | "min_pixels": 3136,   | "max_pixels": 12845056,   | "patch_size": 14,   | "temporal_patch_size": 2,   | "merge_size": 2,   | "image_mean": [   | 0.48145466,   | 0.4578275,   | 0.40821073   | ],   | "image_std": [   | 0.26862954,   | 0.26130258,   | 0.27577711   | ],   | "image_processor_type": "Qwen2VLImageProcessor",   | "processor_class": "Qwen2_5_VLProcessor"   | }  </body> </html> but still I am facing this issue — Reply to this email directly, view it on GitHub, or unsubscribe. You are receiving this because you were mentioned.Message ID: ***@***.***>

igorpereirabr1 · 2025-02-28T21:05:21Z

have you updatef your local directory ？
…

Yes... same error.. even after updating the model localy.

gouqi666 · 2025-03-01T01:14:48Z

change the transformer version  to 4.49 and vllm to 0.7.3

…

---Original--- From: "Igor L ***@***.***> Date: Sat, Mar 1, 2025 05:05 AM To: ***@***.***>; Cc: ***@***.******@***.***>; Subject: Re: [huggingface/transformers] [Version: 4.49.0] Qwen2.5-VL is notsupported in vLLM because of transformers (Issue #36292) have you updatef your local directory ？ … Yes... same error.. even after updating the model localy. — Reply to this email directly, view it on GitHub, or unsubscribe. You are receiving this because you were mentioned.Message ID: ***@***.***> igorpereirabr1 left a comment (huggingface/transformers#36292) have you updatef your local directory ？ … Yes... same error.. even after updating the model localy. — Reply to this email directly, view it on GitHub, or unsubscribe. You are receiving this because you were mentioned.Message ID: ***@***.***>

philipwan · 2025-03-05T11:59:57Z

same error with vll==0.7.3 and transformers==4.49.0

yifei-reducto · 2025-03-11T23:00:45Z

Try using python 3.12. I had the same issue with 3.11 but now everything works fine.

philipwan · 2025-03-12T01:50:54Z

same error with vll==0.7.3 and transformers==4.49.0

I had solved this problem, It's my source code error，but vllm send a puzzling message

ywang96 · 2025-03-12T02:16:08Z

but vllm send a puzzling message

@philipwan what message is it? I can take it back to see if we can update on the vllm side so that we don't confuse users!

philipwan · 2025-03-12T12:06:15Z

but vllm send a puzzling message

@philipwan what message is it? I can take it back to see if we can update on the vllm side so that we don't confuse users!

"ValueError: Model architectures ['Qwen2_5_VLForConditionalGeneration'] failed to be inspected. Please check the logs for more details."
These is error message, real reason for the error stored in vllm.log. When I was a beginner, I mistakenly thought that all the error messages displayed on the screen were

RodriMora · 2025-03-31T14:45:48Z

I'm having the same problem with:

transformers 4.49.0.dev0
vllm 0.8.2

python3.12.9

downloaded the latest model from HF from today

vllm serve /mnt/llms/models/Qwen2.5-VL-32B-Instruct --max-model-len 36000 --host 0.0.0.0 --port 5000 -tp 4

Traceback (most recent call last):
  File "/home/ubuntuai/vllm/.venv/bin/vllm", line 8, in <module>
    sys.exit(main())
             ^^^^^^
  File "/home/ubuntuai/vllm/.venv/lib/python3.12/site-packages/vllm/entrypoints/cli/main.py", line 75, in main
    args.dispatch_function(args)
  File "/home/ubuntuai/vllm/.venv/lib/python3.12/site-packages/vllm/entrypoints/cli/serve.py", line 33, in cmd
    uvloop.run(run_server(args))
  File "/home/ubuntuai/vllm/.venv/lib/python3.12/site-packages/uvloop/__init__.py", line 109, in run
    return __asyncio.run(
           ^^^^^^^^^^^^^^
  File "/usr/lib/python3.12/asyncio/runners.py", line 195, in run
    return runner.run(main)
           ^^^^^^^^^^^^^^^^
  File "/usr/lib/python3.12/asyncio/runners.py", line 118, in run
    return self._loop.run_until_complete(task)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "uvloop/loop.pyx", line 1518, in uvloop.loop.Loop.run_until_complete
  File "/home/ubuntuai/vllm/.venv/lib/python3.12/site-packages/uvloop/__init__.py", line 61, in wrapper
    return await main
           ^^^^^^^^^^
  File "/home/ubuntuai/vllm/.venv/lib/python3.12/site-packages/vllm/entrypoints/openai/api_server.py", line 1016, in run_server
    async with build_async_engine_client(args) as engine_client:
               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3.12/contextlib.py", line 210, in __aenter__
    return await anext(self.gen)
           ^^^^^^^^^^^^^^^^^^^^^
  File "/home/ubuntuai/vllm/.venv/lib/python3.12/site-packages/vllm/entrypoints/openai/api_server.py", line 141, in build_async_engine_client
    async with build_async_engine_client_from_engine_args(
               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3.12/contextlib.py", line 210, in __aenter__
    return await anext(self.gen)
           ^^^^^^^^^^^^^^^^^^^^^
  File "/home/ubuntuai/vllm/.venv/lib/python3.12/site-packages/vllm/entrypoints/openai/api_server.py", line 161, in build_async_engine_client_from_engine_args
    vllm_config = engine_args.create_engine_config(usage_context=usage_context)
                  ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/ubuntuai/vllm/.venv/lib/python3.12/site-packages/vllm/engine/arg_utils.py", line 1296, in create_engine_config
    model_config = self.create_model_config()
                   ^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/ubuntuai/vllm/.venv/lib/python3.12/site-packages/vllm/engine/arg_utils.py", line 1141, in create_model_config
    return ModelConfig(
           ^^^^^^^^^^^^
  File "/home/ubuntuai/vllm/.venv/lib/python3.12/site-packages/vllm/config.py", line 399, in __init__
    self.multimodal_config = self._init_multimodal_config(
                             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/ubuntuai/vllm/.venv/lib/python3.12/site-packages/vllm/config.py", line 467, in _init_multimodal_config
    if self.registry.is_multimodal_model(self.architectures):
       ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/ubuntuai/vllm/.venv/lib/python3.12/site-packages/vllm/model_executor/models/registry.py", line 482, in is_multimodal_model
    model_cls, _ = self.inspect_model_cls(architectures)
                   ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/ubuntuai/vllm/.venv/lib/python3.12/site-packages/vllm/model_executor/models/registry.py", line 442, in inspect_model_cls
    return self._raise_for_unsupported(architectures)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/ubuntuai/vllm/.venv/lib/python3.12/site-packages/vllm/model_executor/models/registry.py", line 392, in _raise_for_unsupported
    raise ValueError(
ValueError: Model architectures ['Qwen2_5_VLForConditionalGeneration'] failed to be inspected. Please check the logs for more details

Sean-Wick · 2025-04-09T02:26:53Z

I'm having the same problem with:
transformers 4.49.0.dev0
vllm 0.8.2
ValueError: Model architectures ['Qwen2_5_VLForConditionalGeneration'] failed to be inspected. Please check the logs for more details

for me

transformers 4.50.3
vllm 0.8.2
ValueError: Model architectures ['Qwen2_5_VLForConditionalGeneration'] failed to be inspected. Please check the logs for more details

it works, but don't know the reason, damn

from gevent import monkey
monkey.patch_all()

usun1997 added the bug label Feb 20, 2025

usun1997 closed this as completed Feb 21, 2025

vladislavdonchev mentioned this issue Feb 22, 2025

Feature Request: Qwen 2.5 VL ggml-org/llama.cpp#11483

Open

4 tasks

zhaohm14 mentioned this issue Feb 25, 2025

Model became broken, huggingface QwenLM/Qwen2.5-VL#840

Closed

tulerfeng mentioned this issue Apr 12, 2025

Unable to run inference tulerfeng/Video-R1#30

Open

[Version: 4.49.0] Qwen2.5-VL is not supported in vLLM because of transformers #36292

[Version: 4.49.0] Qwen2.5-VL is not supported in vLLM because of transformers #36292

Comments

usun1997 commented Feb 20, 2025

System Info

Who can help?

Information

Tasks

Reproduction

Expected behavior

glamourzc commented Feb 20, 2025

Uh oh!

hayreenlee commented Feb 20, 2025

Uh oh!

yynickyeh commented Feb 20, 2025

Uh oh!

ppkliu commented Feb 20, 2025

Uh oh!

Rocketknight1 commented Feb 20, 2025

Uh oh!

zucchini-nlp commented Feb 20, 2025

Uh oh!

Isotr0py commented Feb 20, 2025

Uh oh!

ywang96 commented Feb 20, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

igorpereirabr1 commented Feb 20, 2025

Uh oh!

ywang96 commented Feb 20, 2025

Uh oh!

igorpereirabr1 commented Feb 20, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

usun1997 commented Feb 21, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

usun1997 commented Feb 21, 2025

Uh oh!

gouqi666 commented Feb 21, 2025

Uh oh!

balachandarsv commented Feb 21, 2025

Uh oh!

igorpereirabr1 commented Feb 21, 2025

Uh oh!

PocketBrain commented Feb 27, 2025

Uh oh!

koleshjr commented Feb 27, 2025

Uh oh!

igorpereirabr1 commented Feb 27, 2025

Uh oh!

gouqi666 commented Feb 28, 2025 via email

Uh oh!

koleshjr commented Feb 28, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

gouqi666 commented Feb 28, 2025 via email

Uh oh!

igorpereirabr1 commented Feb 28, 2025

Uh oh!

gouqi666 commented Mar 1, 2025 via email

Uh oh!

philipwan commented Mar 5, 2025

Uh oh!

yifei-reducto commented Mar 11, 2025

Uh oh!

philipwan commented Mar 12, 2025

Uh oh!

ywang96 commented Mar 12, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

philipwan commented Mar 12, 2025

Uh oh!

RodriMora commented Mar 31, 2025

Uh oh!

Sean-Wick commented Apr 9, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

ywang96 commented Feb 20, 2025 •

edited

Loading

igorpereirabr1 commented Feb 20, 2025 •

edited

Loading

usun1997 commented Feb 21, 2025 •

edited

Loading

koleshjr commented Feb 28, 2025 •

edited

Loading

ywang96 commented Mar 12, 2025 •

edited

Loading

Sean-Wick commented Apr 9, 2025 •

edited

Loading