Skip to content

Commit 1fc2088

Browse files
authored
Add seed resizing to community pipelines (#1011)
* add seed resizing to community examples * actually add the file responsible for seed resizing
1 parent 12fd073 commit 1fc2088

File tree

2 files changed

+452
-0
lines changed

2 files changed

+452
-0
lines changed

examples/community/README.md

Lines changed: 86 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -16,6 +16,8 @@ If a community doesn't work as expected, please open an issue and ping the autho
1616
| Speech to Image | Using automatic-speech-recognition to transcribe text and Stable Diffusion to generate images | [Speech to Image](#speech-to-image) | - | [Mikail Duzenli](https://github.com/MikailINTech)
1717
| Wild Card Stable Diffusion | Stable Diffusion Pipeline that supports prompts that contain wildcard terms (indicated by surrounding double underscores), with values instantiated randomly from a corresponding txt file or a dictionary of possible values | [Wildcard Stable Diffusion](#wildcard-stable-diffusion) | - | [Shyam Sudhakaran](https://github.com/shyamsn97) |
1818
| Composable Stable Diffusion| Stable Diffusion Pipeline that supports prompts that contain "|" in prompts (as an AND condition) and weights (separated by "|" as well) to positively / negatively weight prompts. | [Composable Stable Diffusion](#composable-stable-diffusion) | - | [Mark Rich](https://github.com/MarkRich) |
19+
| Seed Resizing Stable Diffusion| Stable Diffusion Pipeline that supports resizing an image and retaining the concepts of the 512 by 512 generation. | [Seed Resizing](#seed-resizing) | - | [Mark Rich](https://github.com/MarkRich) |
20+
1921

2022

2123
To load a custom pipeline you just need to pass the `custom_pipeline` argument to `DiffusionPipeline`, as one of the files in `diffusers/examples/community`. Feel free to send a PR with your own pipelines, we will merge them quickly.
@@ -371,3 +373,87 @@ for i in range(4):
371373
for i, img in enumerate(images):
372374
img.save(f"./composable_diffusion/image_{i}.png")
373375
```
376+
### Seed Resizing
377+
Test seed resizing. Originally generate an image in 512 by 512, then generate image with same seed at 512 by 592 using seed resizing. Finally, generate 512 by 592 using original stable diffusion pipeline.
378+
379+
```python
380+
import torch as th
381+
import numpy as np
382+
from diffusers import DiffusionPipeline
383+
384+
has_cuda = th.cuda.is_available()
385+
device = th.device('cpu' if not has_cuda else 'cuda')
386+
387+
pipe = DiffusionPipeline.from_pretrained(
388+
"CompVis/stable-diffusion-v1-4",
389+
use_auth_token=True,
390+
custom_pipeline="seed_resize_stable_diffusion"
391+
).to(device)
392+
393+
def dummy(images, **kwargs):
394+
return images, False
395+
396+
pipe.safety_checker = dummy
397+
398+
399+
images = []
400+
th.manual_seed(0)
401+
generator = th.Generator("cuda").manual_seed(0)
402+
403+
seed = 0
404+
prompt = "A painting of a futuristic cop"
405+
406+
width = 512
407+
height = 512
408+
409+
res = pipe(
410+
prompt,
411+
guidance_scale=7.5,
412+
num_inference_steps=50,
413+
height=height,
414+
width=width,
415+
generator=generator)
416+
image = res.images[0]
417+
image.save('./seed_resize/seed_resize_{w}_{h}_image.png'.format(w=width, h=height))
418+
419+
420+
th.manual_seed(0)
421+
generator = th.Generator("cuda").manual_seed(0)
422+
423+
pipe = DiffusionPipeline.from_pretrained(
424+
"CompVis/stable-diffusion-v1-4",
425+
use_auth_token=True,
426+
custom_pipeline="/home/mark/open_source/diffusers/examples/community/"
427+
).to(device)
428+
429+
width = 512
430+
height = 592
431+
432+
res = pipe(
433+
prompt,
434+
guidance_scale=7.5,
435+
num_inference_steps=50,
436+
height=height,
437+
width=width,
438+
generator=generator)
439+
image = res.images[0]
440+
image.save('./seed_resize/seed_resize_{w}_{h}_image.png'.format(w=width, h=height))
441+
442+
pipe_compare = DiffusionPipeline.from_pretrained(
443+
"CompVis/stable-diffusion-v1-4",
444+
use_auth_token=True,
445+
custom_pipeline="/home/mark/open_source/diffusers/examples/community/"
446+
).to(device)
447+
448+
res = pipe_compare(
449+
prompt,
450+
guidance_scale=7.5,
451+
num_inference_steps=50,
452+
height=height,
453+
width=width,
454+
generator=generator
455+
)
456+
457+
image = res.images[0]
458+
image.save('./seed_resize/seed_resize_{w}_{h}_image_compare.png'.format(w=width, h=height))
459+
```

0 commit comments

Comments
 (0)