fix(models): Forward timm model kwargs to timm.create_model for OmDet-Turbo by harshaljanjani · Pull Request #44611 · huggingface/transformers

harshaljanjani · 2026-03-11T20:02:14Z

What does this PR do?

The following issue was identified and fixed in this PR:

→ This PR (🚨 Delete duplicate code in backbone utils) structured config loading to use BackboneMixin.consolidate_backbone_kwargs_to_config. For the DETR-family; the current state works correctly because timm_default_kwargs only contains keys that map to TimmBackboneConfig.init. There might be others; but OmDet-Turbo is a model that passes kwargs meant for timm.create_model itself, and which are not TimmBackboneConfig params and were dropped.
→ From the prev PR's diff, before the refactor, the implementation forwarded these params via **kwargs to timm.create_model and it worked before, but after the refactor they were stored as attributes on PreTrainedConfig and never forwarded, and parameters like img_size are ignored.

Fixes #44610.

cc: @zucchini-nlp

CI Failures

Before the fix (feel free to cross-check; these errors are reproducible):

After the fix (feel free to cross-check):

Before submitting

This PR fixes a typo or improves the docs (you can dismiss the other checks if that's the case).
Did you read the contributor guideline,
Pull Request section?
Was this discussed/approved via a Github issue or the forum? Please add a link
to it if that's the case.
Did you make sure to update the documentation with your changes? Here are the
documentation guidelines, and
here are tips on formatting docstrings.
Did you fix any necessary existing tests?

Note

Medium Risk
Changes timm backbone construction and argument forwarding; while scoped to an optional timm_model_kwargs dict, it can affect any model relying on TimmBackbone if those kwargs are set or collide with existing **kwargs.

Overview
Fixes OmDet-Turbo’s timm backbone initialization by storing timm-only parameters (e.g. img_size, always_partition) under backbone_config.timm_model_kwargs instead of top-level config fields.

Updates TimmBackbone to forward config.timm_model_kwargs into timm.create_model, and adds a backward-compatibility shim to migrate older hub configs that had img_size/always_partition as direct attributes.

^{Written by Cursor Bugbot for commit 414ee30. This will update automatically on new commits. Configure here.}

zucchini-nlp · 2026-03-12T11:09:01Z

run-slow: omdet_turbo

github-actions · 2026-03-12T11:10:18Z

Workflow Run ⚙️

This comment contains run-slow, running the specified jobs:

models: ["models/omdet_turbo"]
quantizations: []

zucchini-nlp · 2026-03-12T11:12:33Z

src/transformers/models/timm_backbone/configuration_timm_backbone.py

        out_indices=None,
        freeze_batch_norm_2d=False,
        output_stride=None,
+        timm_model_kwargs=None,


not really what we want for now. The above fix you did with passing it as timm_kwargs should be enough and consolidate_backbone_kwargs_to_config will put it inside timm config

That makes sense; reverted :)

zucchini-nlp · 2026-03-12T11:12:54Z

src/transformers/models/omdet_turbo/configuration_omdet_turbo.py

+        if getattr(backbone_config, "model_type", None) == "timm_backbone" and not getattr(
+            backbone_config, "timm_model_kwargs", None
+        ):
+            timm_extra = {}
+            for attr in ("img_size", "always_partition"):
+                if hasattr(backbone_config, attr):
+                    timm_extra[attr] = getattr(backbone_config, attr)
+            if timm_extra:
+                backbone_config.timm_model_kwargs = timm_extra
+


same here, the consolidate_backbone_kwargs_to_config utility should work ootb. If not, can you check why?

Actually, the omdet-turbo-swin-tiny-hf/config.json has backbone_config:null and backbone_kwargs: {"img_size": 640, "always_partition": true, ...}.

→ consolidate_backbone_kwargs_to_config takes this path bec backbone_kwargs is non-empty, the timm_default_kwargs path is skipped entirely, so img_size/always_partition end up as direct attrs with no timm_model_kwargs. I've added a corresponding comment in the code as well on why the block is required (happy to know if there's a better direction). So consolidate_backbone_kwargs_to_config works for fresh configs OOTB, but omdet-turbo-swin-tiny-hf seems to require the BC check :)
→ I thought about whether to change consolidate_backbone_kwargs_to_config or write the BC block, but writing a check of TimmBackboneConfig params vs timm.create_model params might be fragile, like, DETR passes use_pretrained_backbone in timm_default_kwargs, which is neither a TimmBackboneConfig param nor a valid timm.create_model kwarg, so it would get incorrectly forwarded to timm.create_model and break.

ahhh right, TimmBackbone doesn't accept arbitrary kwargs like TimmWrapper does so it gets popped, even if we pass it when creating the config

Good catch!

Actually I am working on subsequent PR here (#44252). It is blocked currently by a few other required fixes, so until then let's patch OmdetTurbo model code

We can change:

transformers/src/transformers/models/omdet_turbo/modeling_omdet_turbo.py

Line 282 in aad13b8

self.vision_backbone = load_backbone(config)

to smth like:

backbone = AutoBackbone.from_config(config=config.backbone_config, **config.timm_kwargs)

And make sure that timm_kwargs isn't saved when serializing the config. We don't want it in hub checkpoints, it'll cause more headache

after unification we'll have more freedom to pass any timm kwargs when creating the model by config.model_args

Thank you for your time @zucchini-nlp; attempted to resolve it accordingly 🤗

HuggingFaceDocBuilderDev · 2026-03-12T11:18:50Z

The docs for this PR live here. All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.

github-actions · 2026-03-12T11:19:54Z

CI Results

Workflow Run ⚙️

Commit Info

Context	Commit	Description
RUN	ba3c4441	workflow commit (merge commit)
PR	357a5cfb	branch commit (from PR)
main	3027515c	base commit (on `main`)

✅ No failing test specific to this PR 🎉 👏 !

cursor

Cursor Bugbot has reviewed your changes and found 1 potential issue.

^{Bugbot Autofix is OFF. To automatically fix reported issues with cloud agents, enable autofix in the Cursor dashboard.}

cursor · 2026-03-12T12:40:57Z

src/transformers/models/timm_backbone/modeling_timm_backbone.py

        out_indices = config.out_indices if getattr(config, "out_indices", None) is not None else (-1,)
        pretrained = kwargs.pop("pretrained", False)
        in_chans = kwargs.pop("in_chans", config.num_channels)
+        timm_model_kwargs = getattr(config, "timm_model_kwargs", {})


Unpacking None as kwargs causes TypeError at init

Medium Severity

getattr(config, "timm_model_kwargs", {}) returns None (not {}) when the attribute exists on the config but is explicitly None. This happens when a saved config JSON contains "timm_model_kwargs": null or when a backbone_config dict passes it as None. The subsequent **timm_model_kwargs unpacking then raises TypeError. Using getattr(config, "timm_model_kwargs", None) or {} would guard against this.

There's no codepath tmk that produces None here?

harshaljanjani · 2026-03-12T20:13:22Z

Updated with the suggested fix; happy to make further changes if needed :)

zucchini-nlp · 2026-03-13T11:17:13Z

src/transformers/models/omdet_turbo/configuration_omdet_turbo.py

+        timm_kwargs = {}
+        if getattr(backbone_config, "model_type", None) == "timm_backbone":
+            for attr in ("img_size", "always_partition"):
+                if hasattr(backbone_config, attr):
+                    timm_kwargs[attr] = getattr(backbone_config, attr)
+


let's add a comment on why this is needed pls

zucchini-nlp

Great, will merge after adding the comment

zucchini-nlp · 2026-03-13T11:17:36Z

run-slow: omdet_turbo

github-actions · 2026-03-13T11:18:51Z

Workflow Run ⚙️

This comment contains run-slow, running the specified jobs:

models: ["models/omdet_turbo"]
quantizations: []

github-actions · 2026-03-13T11:28:20Z

CI Results

Workflow Run ⚙️

Commit Info

Context	Commit	Description
RUN	04863413	workflow commit (merge commit)
PR	94ce6381	branch commit (from PR)
main	745341d8	base commit (on `main`)

✅ No failing test specific to this PR 🎉 👏 !

github-actions · 2026-03-13T11:31:43Z

[For maintainers] Suggested jobs to run (before merge)

run-slow: omdet_turbo

fix: Add and forward timm_model_kwargs to timm.create_model

357a5cf

harshaljanjani marked this pull request as ready for review March 11, 2026 20:19

github-actions bot requested review from ArthurZucker and Rocketknight1 March 11, 2026 20:20

zucchini-nlp reviewed Mar 12, 2026

View reviewed changes

fix: Revert TimmBackboneConfig, add BC migration comment

414ee30

cursor bot reviewed Mar 12, 2026

View reviewed changes

harshaljanjani requested a review from zucchini-nlp March 12, 2026 12:50

change: Address review

94ce638

zucchini-nlp reviewed Mar 13, 2026

View reviewed changes

zucchini-nlp approved these changes Mar 13, 2026

View reviewed changes

nit: Add comment

45ce043

zucchini-nlp enabled auto-merge March 13, 2026 11:39

zucchini-nlp added this pull request to the merge queue Mar 13, 2026

Merged via the queue into huggingface:main with commit 3dd82fa Mar 13, 2026
23 checks passed

harshaljanjani deleted the fix/omdet-turbo-param-forwarding branch March 13, 2026 11:57

Conversation

harshaljanjani commented Mar 11, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

What does this PR do?

Before submitting

Uh oh!

zucchini-nlp commented Mar 12, 2026

Uh oh!

github-actions bot commented Mar 12, 2026

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

zucchini-nlp Mar 12, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

HuggingFaceDocBuilderDev commented Mar 12, 2026

Uh oh!

github-actions bot commented Mar 12, 2026

CI Results

Commit Info

Uh oh!

cursor bot left a comment

Choose a reason for hiding this comment

Uh oh!

cursor bot Mar 12, 2026

Choose a reason for hiding this comment

Unpacking None as kwargs causes TypeError at init

Uh oh!

harshaljanjani Mar 12, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

harshaljanjani commented Mar 12, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

zucchini-nlp left a comment

Choose a reason for hiding this comment

Uh oh!

zucchini-nlp commented Mar 13, 2026

Uh oh!

github-actions bot commented Mar 13, 2026

Uh oh!

github-actions bot commented Mar 13, 2026

CI Results

Commit Info

Uh oh!

github-actions bot commented Mar 13, 2026

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

harshaljanjani commented Mar 11, 2026 •

edited

Loading

zucchini-nlp Mar 12, 2026 •

edited

Loading

Unpacking `None` as kwargs causes TypeError at init

harshaljanjani Mar 12, 2026 •

edited

Loading

harshaljanjani commented Mar 12, 2026 •

edited

Loading