Fixes hqq by following a new path for bias parameter in pre_quantized models

[MekkCyber]

What does this PR do?

Since HQQ overrides the load_state_dict method for HQQLinear, it directly loads both the weight and bias parameters. This differs from our approach, where we iterate through the parameters one by one and load the bias separately from the weights.

This PR updates the behavior to simply ignore the bias parameter, assuming it was already loaded alongside the weights in the case of pre-quantized models.

[github-actions]

Hi 👋, thank you for opening this pull request! The pull request is converted to draft by default. The CI will be paused while the PR is in draft mode. When it is ready for review, please click the Ready for review button (at the bottom of the PR page). This will assign reviewers and trigger CI.

[SunMarc]

SGTM ! Let's add a small test if this isn't tested

[mobicham]

Thank you @MekkCyber. Can you also change this:
https://github.com/huggingface/transformers/blob/main/tests/quantization/hqq/test_hqq.py#L85
to facebook/opt-125m
The current hqq test use a model without a bias.

[MekkCyber]

we already have this test : https://github.com/huggingface/transformers/blob/main/tests/quantization/hqq/test_hqq.py#L151, we just need to add the pre_quantized case

[HuggingFaceDocBuilderDev]

The docs for this PR live here. All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.

[mobicham]

we already have this test : https://github.com/huggingface/transformers/blob/main/tests/quantization/hqq/test_hqq.py#L151, we just need to add the pre_quantized case

It doesn't check for serialization of a model with a bias. If it did, the tests would have failed actually

[mobicham]

Can you please run this script:
https://gist.github.com/mobicham/701dd564c52590203ee09631425ad797

If it doesn't throw an error, the fix works as expected.

[MekkCyber]

https://github.com/huggingface/transformers/blob/main/tests/quantization/hqq/test_hqq.py#L151, we just need to add the pre_quantized case

It doesn't check for serialization of a model with a bias. If it did, the tests would have failed actually

yep that's what i meant, we only need to add the case of pre_quantized (serialized) models

[MekkCyber]

The snippet works well

[mobicham]

Awesome, thank you again @MekkCyber 🙏

[mobicham]

@MekkCyber unfortunately it seems that it's not fully resolved. For example, when I tried to load a quantized Qwen model that has a bias:

import torch
from transformers import Qwen2_5_VLForConditionalGeneration, AutoProcessor

model_id = "mobiuslabsgmbh/Qwen2.5-VL-3B-Instruct_4bitgs64_hqq_hf"
model = Qwen2_5_VLForConditionalGeneration.from_pretrained(model_id, torch_dtype=torch.float16, device_map="cuda:0")

File /opt/conda/lib/python3.11/site-packages/accelerate/utils/modeling.py:283, in set_module_tensor_to_device(module, tensor_name, device, value, dtype, fp16_statistics, tied_params_map)
    280     return
    282 if old_value.device == torch.device("meta") and device not in ["meta", torch.device("meta")] and value is None:
--> 283     raise ValueError(f"{tensor_name} is on the meta device, we need a `value` to put in on {device}.")
    285 param = module._parameters[tensor_name] if tensor_name in module._parameters else None
    286 param_cls = type(param)

ValueError: bias is on the meta device, we need a `value` to put in on cuda:0.

[MekkCyber]

Thanks for reporting @mobicham, will take a look