Skip to content

Fix loading if unexpected keys are present #3720

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Merged
merged 2 commits into from
Jun 8, 2023
Merged

Fix loading if unexpected keys are present #3720

merged 2 commits into from
Jun 8, 2023

Conversation

patrickvonplaten
Copy link
Contributor

@patrickvonplaten patrickvonplaten commented Jun 8, 2023 •
edited
Loading

In transformers >= 4.30, the GPT2 model removes the persistency flag for weights which means that when someone does:

from transformers import GPT2Model

# download gpt2_checkpoint and load as state dict
state_dict = # load gpt2 checkpoint

model = GPT2Model(config)

model.load_state_dict(state_dict) # <- this might now fail

We fix this in diffusers by using the same: _keys_to_ignore_on_load_unexpected logic that is used in transformers:
huggingface/transformers#23256

Need to do a release before end of day so sadly no time.

@patrickvonplaten patrickvonplaten changed the title Fix loading Fix loading if unexpected keys are present Jun 8, 2023
@HuggingFaceDocBuilderDev
Copy link

HuggingFaceDocBuilderDev commented Jun 8, 2023 •
edited
Loading

The documentation is not available anymore as the PR was closed or merged.

@patrickvonplaten patrickvonplaten merged commit f523b11 into main Jun 8, 2023
@patrickvonplaten patrickvonplaten deleted the fix_unidiffusers_loading_with_new_transformer branch June 8, 2023 14:48
patrickvonplaten added a commit that referenced this pull request Jun 8, 2023
@patrickvonplaten
Fix loading if unexpected keys are present (#3720)
5916743 
* Fix loading

* make style
stevhliu pushed a commit that referenced this pull request Jul 18, 2023
@tjdtnsu @seriousran
@apolinario @patrickvonplaten @howsmyanimeprofilepicture @chichead @movie5 @cuchoco @mybloodykeyboard @jjuun0 @szjung-test @idra79haza
[Docs] Korean translation update (#4022)
8b18cd8 
* feat) optimization kr translation

* fix) typo, italic setting

* feat) dreambooth, text2image kr

* feat) lora kr

* fix) LoRA

* fix) fp16 fix

* fix) doc-builder style

* fix) fp16 일부 단어 수정

* fix) fp16 style fix

* fix) opt, training docs update

* merge conflict

* Fix community pipelines (#3266)

* Allow disabling torch 2_0 attention (#3273)

* Allow disabling torch 2_0 attention

* make style

* Update src/diffusers/models/attention.py

* Release: v0.16.1

* feat) toctree update

* feat) toctree update

* Fix custom releases (#3708)

* Fix custom releases

* make style

* Fix loading if unexpected keys are present (#3720)

* Fix loading

* make style

* Release: v0.17.0

* opt_overview

* commit

* Create pipeline_overview.mdx

* unconditional_image_generatoin_1stDraft

* ✨ Add translation for write_own_pipeline.mdx

* conditional-직역, 언컨디셔널

* unconditional_image_generation first draft

* reviese

* Update pipeline_overview.mdx

* revise-2

* ♻️ translation fixed for write_own_pipeline.mdx

* complete translate basic_training.mdx

* other-formats.mdx 번역 완료

* fix tutorials/basic_training.mdx

* other-formats 수정

* inpaint 한국어 번역

* depth2img translation

* translate training/adapt-a-model.mdx

* revised_all

* feedback taken

* using_safetensors.mdx_first_draft

* custom_pipeline_examples.mdx_first_draft

* img2img 한글번역 완료

* tutorial_overview edit

* reusing_seeds

* torch2.0

* translate complete

* fix) 용어 통일 규약 반영

* [fix] 피드백을 반영해서 번역 보정

* 오탈자 정정 + 컨벤션 위배된 부분 정정

* typo, style fix

* toctree update

* copyright fix

* toctree fix

* Update _toctree.yml

---------

Co-authored-by: Chanran Kim <[email protected]>
Co-authored-by: apolinário <[email protected]>
Co-authored-by: Patrick von Platen <[email protected]>
Co-authored-by: Lee, Hongkyu <[email protected]>
Co-authored-by: hyeminan <[email protected]>
Co-authored-by: movie5 <[email protected]>
Co-authored-by: idra79haza <[email protected]>
Co-authored-by: Jihwan Kim <[email protected]>
Co-authored-by: jungwoo <[email protected]>
Co-authored-by: jjuun0 <[email protected]>
Co-authored-by: szjung-test <[email protected]>
Co-authored-by: idra79haza <[email protected]>
Co-authored-by: howsmyanimeprofilepicture <[email protected]>
Co-authored-by: hoswmyanimeprofilepicture <[email protected]>
orpatashnik pushed a commit to orpatashnik/diffusers that referenced this pull request Aug 1, 2023
@tjdtnsu @seriousran
@apolinario @patrickvonplaten @howsmyanimeprofilepicture @chichead @movie5 @cuchoco @mybloodykeyboard @jjuun0 @szjung-test @idra79haza @orpatashnik
[Docs] Korean translation update (huggingface#4022)
219eef2 
* feat) optimization kr translation

* fix) typo, italic setting

* feat) dreambooth, text2image kr

* feat) lora kr

* fix) LoRA

* fix) fp16 fix

* fix) doc-builder style

* fix) fp16 일부 단어 수정

* fix) fp16 style fix

* fix) opt, training docs update

* merge conflict

* Fix community pipelines (huggingface#3266)

* Allow disabling torch 2_0 attention (huggingface#3273)

* Allow disabling torch 2_0 attention

* make style

* Update src/diffusers/models/attention.py

* Release: v0.16.1

* feat) toctree update

* feat) toctree update

* Fix custom releases (huggingface#3708)

* Fix custom releases

* make style

* Fix loading if unexpected keys are present (huggingface#3720)

* Fix loading

* make style

* Release: v0.17.0

* opt_overview

* commit

* Create pipeline_overview.mdx

* unconditional_image_generatoin_1stDraft

* ✨ Add translation for write_own_pipeline.mdx

* conditional-직역, 언컨디셔널

* unconditional_image_generation first draft

* reviese

* Update pipeline_overview.mdx

* revise-2

* ♻️ translation fixed for write_own_pipeline.mdx

* complete translate basic_training.mdx

* other-formats.mdx 번역 완료

* fix tutorials/basic_training.mdx

* other-formats 수정

* inpaint 한국어 번역

* depth2img translation

* translate training/adapt-a-model.mdx

* revised_all

* feedback taken

* using_safetensors.mdx_first_draft

* custom_pipeline_examples.mdx_first_draft

* img2img 한글번역 완료

* tutorial_overview edit

* reusing_seeds

* torch2.0

* translate complete

* fix) 용어 통일 규약 반영

* [fix] 피드백을 반영해서 번역 보정

* 오탈자 정정 + 컨벤션 위배된 부분 정정

* typo, style fix

* toctree update

* copyright fix

* toctree fix

* Update _toctree.yml

---------

Co-authored-by: Chanran Kim <[email protected]>
Co-authored-by: apolinário <[email protected]>
Co-authored-by: Patrick von Platen <[email protected]>
Co-authored-by: Lee, Hongkyu <[email protected]>
Co-authored-by: hyeminan <[email protected]>
Co-authored-by: movie5 <[email protected]>
Co-authored-by: idra79haza <[email protected]>
Co-authored-by: Jihwan Kim <[email protected]>
Co-authored-by: jungwoo <[email protected]>
Co-authored-by: jjuun0 <[email protected]>
Co-authored-by: szjung-test <[email protected]>
Co-authored-by: idra79haza <[email protected]>
Co-authored-by: howsmyanimeprofilepicture <[email protected]>
Co-authored-by: hoswmyanimeprofilepicture <[email protected]>
orpatashnik pushed a commit to orpatashnik/diffusers that referenced this pull request Aug 1, 2023
@tjdtnsu @seriousran
@apolinario @patrickvonplaten @howsmyanimeprofilepicture @chichead @movie5 @cuchoco @mybloodykeyboard @jjuun0 @szjung-test @idra79haza @orpatashnik
[Docs] Korean translation update (huggingface#4022)
80c2271 
* feat) optimization kr translation

* fix) typo, italic setting

* feat) dreambooth, text2image kr

* feat) lora kr

* fix) LoRA

* fix) fp16 fix

* fix) doc-builder style

* fix) fp16 일부 단어 수정

* fix) fp16 style fix

* fix) opt, training docs update

* merge conflict

* Fix community pipelines (huggingface#3266)

* Allow disabling torch 2_0 attention (huggingface#3273)

* Allow disabling torch 2_0 attention

* make style

* Update src/diffusers/models/attention.py

* Release: v0.16.1

* feat) toctree update

* feat) toctree update

* Fix custom releases (huggingface#3708)

* Fix custom releases

* make style

* Fix loading if unexpected keys are present (huggingface#3720)

* Fix loading

* make style

* Release: v0.17.0

* opt_overview

* commit

* Create pipeline_overview.mdx

* unconditional_image_generatoin_1stDraft

* ✨ Add translation for write_own_pipeline.mdx

* conditional-직역, 언컨디셔널

* unconditional_image_generation first draft

* reviese

* Update pipeline_overview.mdx

* revise-2

* ♻️ translation fixed for write_own_pipeline.mdx

* complete translate basic_training.mdx

* other-formats.mdx 번역 완료

* fix tutorials/basic_training.mdx

* other-formats 수정

* inpaint 한국어 번역

* depth2img translation

* translate training/adapt-a-model.mdx

* revised_all

* feedback taken

* using_safetensors.mdx_first_draft

* custom_pipeline_examples.mdx_first_draft

* img2img 한글번역 완료

* tutorial_overview edit

* reusing_seeds

* torch2.0

* translate complete

* fix) 용어 통일 규약 반영

* [fix] 피드백을 반영해서 번역 보정

* 오탈자 정정 + 컨벤션 위배된 부분 정정

* typo, style fix

* toctree update

* copyright fix

* toctree fix

* Update _toctree.yml

---------

Co-authored-by: Chanran Kim <[email protected]>
Co-authored-by: apolinário <[email protected]>
Co-authored-by: Patrick von Platen <[email protected]>
Co-authored-by: Lee, Hongkyu <[email protected]>
Co-authored-by: hyeminan <[email protected]>
Co-authored-by: movie5 <[email protected]>
Co-authored-by: idra79haza <[email protected]>
Co-authored-by: Jihwan Kim <[email protected]>
Co-authored-by: jungwoo <[email protected]>
Co-authored-by: jjuun0 <[email protected]>
Co-authored-by: szjung-test <[email protected]>
Co-authored-by: idra79haza <[email protected]>
Co-authored-by: howsmyanimeprofilepicture <[email protected]>
Co-authored-by: hoswmyanimeprofilepicture <[email protected]>
orpatashnik pushed a commit to orpatashnik/diffusers that referenced this pull request Aug 1, 2023
@tjdtnsu @seriousran
@apolinario @patrickvonplaten @howsmyanimeprofilepicture @chichead @movie5 @cuchoco @mybloodykeyboard @jjuun0 @szjung-test @idra79haza @orpatashnik
[Docs] Korean translation update (huggingface#4022)
6d6b1c6 
* feat) optimization kr translation

* fix) typo, italic setting

* feat) dreambooth, text2image kr

* feat) lora kr

* fix) LoRA

* fix) fp16 fix

* fix) doc-builder style

* fix) fp16 일부 단어 수정

* fix) fp16 style fix

* fix) opt, training docs update

* merge conflict

* Fix community pipelines (huggingface#3266)

* Allow disabling torch 2_0 attention (huggingface#3273)

* Allow disabling torch 2_0 attention

* make style

* Update src/diffusers/models/attention.py

* Release: v0.16.1

* feat) toctree update

* feat) toctree update

* Fix custom releases (huggingface#3708)

* Fix custom releases

* make style

* Fix loading if unexpected keys are present (huggingface#3720)

* Fix loading

* make style

* Release: v0.17.0

* opt_overview

* commit

* Create pipeline_overview.mdx

* unconditional_image_generatoin_1stDraft

* ✨ Add translation for write_own_pipeline.mdx

* conditional-직역, 언컨디셔널

* unconditional_image_generation first draft

* reviese

* Update pipeline_overview.mdx

* revise-2

* ♻️ translation fixed for write_own_pipeline.mdx

* complete translate basic_training.mdx

* other-formats.mdx 번역 완료

* fix tutorials/basic_training.mdx

* other-formats 수정

* inpaint 한국어 번역

* depth2img translation

* translate training/adapt-a-model.mdx

* revised_all

* feedback taken

* using_safetensors.mdx_first_draft

* custom_pipeline_examples.mdx_first_draft

* img2img 한글번역 완료

* tutorial_overview edit

* reusing_seeds

* torch2.0

* translate complete

* fix) 용어 통일 규약 반영

* [fix] 피드백을 반영해서 번역 보정

* 오탈자 정정 + 컨벤션 위배된 부분 정정

* typo, style fix

* toctree update

* copyright fix

* toctree fix

* Update _toctree.yml

---------

Co-authored-by: Chanran Kim <[email protected]>
Co-authored-by: apolinário <[email protected]>
Co-authored-by: Patrick von Platen <[email protected]>
Co-authored-by: Lee, Hongkyu <[email protected]>
Co-authored-by: hyeminan <[email protected]>
Co-authored-by: movie5 <[email protected]>
Co-authored-by: idra79haza <[email protected]>
Co-authored-by: Jihwan Kim <[email protected]>
Co-authored-by: jungwoo <[email protected]>
Co-authored-by: jjuun0 <[email protected]>
Co-authored-by: szjung-test <[email protected]>
Co-authored-by: idra79haza <[email protected]>
Co-authored-by: howsmyanimeprofilepicture <[email protected]>
Co-authored-by: hoswmyanimeprofilepicture <[email protected]>
yoonseokjin pushed a commit to yoonseokjin/diffusers that referenced this pull request Dec 25, 2023
@patrickvonplaten
Fix loading if unexpected keys are present (huggingface#3720)
818d4e1 
* Fix loading

* make style
AmericanPresidentJimmyCarter pushed a commit to AmericanPresidentJimmyCarter/diffusers that referenced this pull request Apr 26, 2024
@patrickvonplaten
Fix loading if unexpected keys are present (huggingface#3720)
fa8deee 
* Fix loading

* make style
AmericanPresidentJimmyCarter pushed a commit to AmericanPresidentJimmyCarter/diffusers that referenced this pull request Apr 26, 2024
@tjdtnsu @seriousran
@apolinario @patrickvonplaten @howsmyanimeprofilepicture @chichead @movie5 @cuchoco @mybloodykeyboard @jjuun0 @szjung-test @idra79haza
[Docs] Korean translation update (huggingface#4022)
48b3f54 
* feat) optimization kr translation

* fix) typo, italic setting

* feat) dreambooth, text2image kr

* feat) lora kr

* fix) LoRA

* fix) fp16 fix

* fix) doc-builder style

* fix) fp16 일부 단어 수정

* fix) fp16 style fix

* fix) opt, training docs update

* merge conflict

* Fix community pipelines (huggingface#3266)

* Allow disabling torch 2_0 attention (huggingface#3273)

* Allow disabling torch 2_0 attention

* make style

* Update src/diffusers/models/attention.py

* Release: v0.16.1

* feat) toctree update

* feat) toctree update

* Fix custom releases (huggingface#3708)

* Fix custom releases

* make style

* Fix loading if unexpected keys are present (huggingface#3720)

* Fix loading

* make style

* Release: v0.17.0

* opt_overview

* commit

* Create pipeline_overview.mdx

* unconditional_image_generatoin_1stDraft

* ✨ Add translation for write_own_pipeline.mdx

* conditional-직역, 언컨디셔널

* unconditional_image_generation first draft

* reviese

* Update pipeline_overview.mdx

* revise-2

* ♻️ translation fixed for write_own_pipeline.mdx

* complete translate basic_training.mdx

* other-formats.mdx 번역 완료

* fix tutorials/basic_training.mdx

* other-formats 수정

* inpaint 한국어 번역

* depth2img translation

* translate training/adapt-a-model.mdx

* revised_all

* feedback taken

* using_safetensors.mdx_first_draft

* custom_pipeline_examples.mdx_first_draft

* img2img 한글번역 완료

* tutorial_overview edit

* reusing_seeds

* torch2.0

* translate complete

* fix) 용어 통일 규약 반영

* [fix] 피드백을 반영해서 번역 보정

* 오탈자 정정 + 컨벤션 위배된 부분 정정

* typo, style fix

* toctree update

* copyright fix

* toctree fix

* Update _toctree.yml

---------

Co-authored-by: Chanran Kim <[email protected]>
Co-authored-by: apolinário <[email protected]>
Co-authored-by: Patrick von Platen <[email protected]>
Co-authored-by: Lee, Hongkyu <[email protected]>
Co-authored-by: hyeminan <[email protected]>
Co-authored-by: movie5 <[email protected]>
Co-authored-by: idra79haza <[email protected]>
Co-authored-by: Jihwan Kim <[email protected]>
Co-authored-by: jungwoo <[email protected]>
Co-authored-by: jjuun0 <[email protected]>
Co-authored-by: szjung-test <[email protected]>
Co-authored-by: idra79haza <[email protected]>
Co-authored-by: howsmyanimeprofilepicture <[email protected]>
Co-authored-by: hoswmyanimeprofilepicture <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers
No reviews
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
2 participants
@patrickvonplaten @HuggingFaceDocBuilderDev