Repository navigation
Exclude audio modules from conversion process #45683
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
base: main
Are you sure you want to change the base?
Changes from 1 commit
a65c934
994f086
5bc34d1
22104ce
9cda19f
5ba912f
File filter
Filter by extension
Conversations
Jump to
Diff view
Diff view
There are no files selected for viewing
| Original file line number | Diff line number | Diff line change |
|---|---|---|
|
|
@@ -57,6 +57,12 @@ def get_keys_to_not_convert(model) -> list: | |
| } | ||
| modules_to_not_convert = tied_keys | last_module_key | output_emb_keys | ||
|
|
||
| # remove audio modules for multimodal models to prevent uint8 crash | ||
| for name, _ in model.named_modules(): | ||
| if "audio_tower" in name or "embed_audio" in name: | ||
| modules_to_not_convert.add(name) | ||
|
Comment on lines
+63
to
+65
Member
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. is there a better way to check if we have an audio module. also i don't want to add all modules in it, only the top should be added. Make sure you add some tests also for bnb for example
Comment on lines
+60
to
+65
Member
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. hmmm i don't think this is robust enough. Will the audio_tower always be in named_children and not in another children's child ?
Author
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. You're right — audio_tower sits under model in Gemma4ForConditionalGeneration, so named_children() on the top-level model wouldn't find it. I've switched to named_modules() with endswith() check, so it finds audio modules regardless of nesting depth while still only adding the top-level audio module name (e.g. model.audio_tower) not its internal sub-layers. Updated the test to verify nested structure detection as well. |
||
|
|
||
|
|
||
| modules_to_not_convert = list({k.removesuffix(".weight") for k in modules_to_not_convert}) | ||
|
|
||
| return list(modules_to_not_convert) | ||
|
|
||
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
update the comment, this is not only for unint8 crash but just not worth quantizing it as it is quite small already