Skip to content

Fix: propagate quantization_config to text sub-config for composite models in AutoModelForCausalLM #45494

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
SunMarc merged 2 commits into from
Apr 21, 2026
Merged

Fix: propagate quantization_config to text sub-config for composite models in AutoModelForCausalLM #45494

Changes from all commits
Commits
Show all changes
2 commits
Select commit Hold shift + click to select a range
3aa823f
Fix: propagate quantization_config to text sub-config for composite m…
lvliang-intel Apr 17, 2026
c9981a2
fix comments
lvliang-intel Apr 21, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
7 changes: 7 additions & 0 deletions src/transformers/models/auto/auto_factory.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -383,7 +383,14 @@ def from_pretrained(cls, pretrained_model_name_or_path: str | os.PathLike[str],
elif has_local_code:
model_class = _get_model_class(config, cls._model_mapping)
if model_class.config_class == config.sub_configs.get("text_config", None):
# TODO: Validate that copying the parent quantization config to the text sub-config preserves
# modules_to_not_convert and skip-module matching when composite-model module prefixes differ.
parent_config = config
config = config.get_text_config()
# Propagate quantization_config from the composite parent config so that
# `get_hf_quantizer` can correctly detect the model as pre-quantized.
if hasattr(parent_config, "quantization_config"):
config.quantization_config = parent_config.quantization_config
Comment thread
lvliang-intel marked this conversation as resolved.
Resolved
return model_class.from_pretrained(
pretrained_model_name_or_path, *model_args, config=config, **hub_kwargs, **kwargs
)
Expand Down
Loading