AttributeError: 'Qwen2VLConfig' object has no attribute 'hidden_size'

[Tcc0403]

System Info

• transformers version: 4.53.0.dev0
• Platform: Linux-5.15.167.4-microsoft-standard-WSL2-x86_64-with-glibc2.35
• Python version: 3.13.1
• Huggingface_hub version: 0.32.0
• Safetensors version: 0.5.3
• Accelerate version: 1.7.0
• Accelerate config: not found
• DeepSpeed version: not installed
• PyTorch version (accelerator?): 2.6.0+cu124 (CUDA)
• Tensorflow version (GPU?): not installed (NA)
• Flax version (CPU?/GPU?/TPU?): not installed (NA)
• Jax version: not installed
• JaxLib version: not installed
• Using distributed or parallel set-up in script?: no
• Using GPU in script?: no
• GPU type: NVIDIA GeForce RTX 3080

Who can help?

@zucchini-nlp

Information

• The official example scripts
• My own modified scripts

Tasks

• An officially supported task in the examples folder (such as GLUE/SQuAD, ...)
• My own task or dataset (give details below)

Reproduction

Running the follwoing code gives an error:

from transformers.models.qwen2_vl import Qwen2VLConfig, Qwen2VLForConditionalGeneration


# Lower model size to speed up initialization
config = Qwen2VLConfig(text_config={"hidden_size": 1024, "intermediate_size": 2048, "num_hidden_layers": 4})
model = Qwen2VLForConditionalGeneration(config)

Error message:

Traceback (most recent call last):
  File "/home/tcc/transformers/qwen2_vl_config_bug.py", line 6, in <module>
    model = Qwen2VLForConditionalGeneration(config)
  File "/home/tcc/transformers/.venv/lib/python3.13/site-packages/transformers/models/qwen2_vl/modeling_qwen2_vl.py", line 1744, in __init__
    self.lm_head = nn.Linear(config.hidden_size, config.vocab_size, bias=False)
                             ^^^^^^^^^^^^^^^^^^
  File "/home/tcc/transformers/.venv/lib/python3.13/site-packages/transformers/configuration_utils.py", line 211, in __getattribute__
    return super().__getattribute__(key)
           ~~~~~~~~~~~~~~~~~~~~~~~~^^^^^
AttributeError: 'Qwen2VLConfig' object has no attribute 'hidden_size'

Expected behavior

The current lm_head is initialized with config.vocab_size and config.hidden_size.

transformers/src/transformers/models/qwen2_vl/modeling_qwen2_vl.py

Lines 1741 to 1744 in 1ed1936

	def __init__(self, config):
	super().__init__(config)
	self.model = Qwen2VLModel(config)
	self.lm_head = nn.Linear(config.hidden_size, config.vocab_size, bias=False)

Since Qwen2VLConfig is passed to Qwen2VLForConditionalGeneration, perhaps we need to read from config.text_config instead.

More context

#37268 introduced standardize config to qwen vlm family and they are lm_head was properly initialized with config.text_config.hidden_size/vocab_size. However, in the follow-up PR #37033, the initialization of lm_head fell back to config.hidden_size/vocab_size.

Afaik Qwen2_VL and Qwen2_5_VL are only two models that have the above issues.

[zucchini-nlp]

Thanks for reporting! Indeed for the correctness it should look in the text config, though the models wouldn't fail using the official checkpoints. I recently updated config to contain a text_config but it is totally BC and the tests are passing on our end. The script fails as it uses custom config...

I will fix it so that models copying from Qwen will have the correct version and the model can load with custom new-format config

[Tcc0403]

Thanks for the reply! I think the tests only passed becasue all arguments are passed in as **kwargs and default config wasn't covered.

Even if the config is created without any arguments, i.e. default config, it will still throw an error when trying to access hidden_size under default config.

Script:

from transformers.models.qwen2_vl import Qwen2VLConfig

# default config
config = Qwen2VLConfig()
print(f"{config.hidden_size=}")

Error message:

❯ python3 qwen2_vl_config_bug.py
Traceback (most recent call last):
  File "/home/tcc/transformers/qwen2_vl_config_bug.py", line 6, in <module>
    print(f"{config.hidden_size=}")
             ^^^^^^^^^^^^^^^^^^
  File "/home/tcc/transformers/.venv/lib/python3.13/site-packages/transformers/configuration_utils.py", line 211, in __getattribute__
    return super().__getattribute__(key)
           ~~~~~~~~~~~~~~~~~~~~~~~~^^^^^
AttributeError: 'Qwen2VLConfig' object has no attribute 'hidden_size'

[zucchini-nlp]

Yes, that is done on purpose for BC and the reason why tests pass using the official checkpoints. It fails only when a new config is created manually, following the expected nested format