Skip to content

Fix textual inversion loading #2914

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 1 commit into from
Mar 31, 2023
Merged

Conversation

GuiyeC
Copy link
Contributor

@GuiyeC GuiyeC commented Mar 31, 2023

Textual inversion doesn’t produce expected results, I tried commenting on the PR but it got merged without the fix.

Basically it looks like when the embedding’s shape has more than one dimension we take the full embedding but if it has only one we take the first value of it, resulting in a tensor of a single value. This ends up not changing the images generated at all.

For example:

  • Embedding shape is [3, 768]: embedding is considered multi vector and 3 embeddings of shape [768] are added.
  • Embedding shape is [1, 768]: At the moment the full tensor is being loaded, we should load a single embedding of shape [768].
  • Embedding shape is [768]: At the moment the first value of the tensor is being loaded, we should load the full embedding.

@HuggingFaceDocBuilderDev
Copy link

HuggingFaceDocBuilderDev commented Mar 31, 2023

The documentation is not available anymore as the PR was closed or merged.

@patrickvonplaten
Copy link
Contributor

Hey @GuiyeC,

Sorry, you're 100% right here - I went over your comment too quickly the last time. Thanks for the perseverance and in-detail description above.

@patrickvonplaten patrickvonplaten merged commit 1055175 into huggingface:main Mar 31, 2023
@GuiyeC
Copy link
Contributor Author

GuiyeC commented Mar 31, 2023

@patrickvonplaten no problem, thank you for your work on this repo, I'm glad to be a contributor! 😄

w4ffl35 pushed a commit to w4ffl35/diffusers that referenced this pull request Apr 14, 2023
yoonseokjin pushed a commit to yoonseokjin/diffusers that referenced this pull request Dec 25, 2023
AmericanPresidentJimmyCarter pushed a commit to AmericanPresidentJimmyCarter/diffusers that referenced this pull request Apr 26, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants