Skip to content

Commit a793b1f

Browse files
authored
Add imagic to community pipelines (#958)
* initial commit to add imagic to stable diffusion community pipelines * remove some testing changes * comments from PR review for imagic stable diffusion * remove changes from pipeline_stable_diffusion as part of imagic pipeline * clean up example code and add line back in to pipeline_stable_diffusion for imagic pipeline * remove unused functions * small code quality changes for imagic pipeline * clean up readme * remove hardcoded logging values for imagic community example * undo change for DDIMScheduler
1 parent 7fb4b88 commit a793b1f

File tree

2 files changed

+521
-1
lines changed

2 files changed

+521
-1
lines changed

examples/community/README.md

Lines changed: 45 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -18,6 +18,7 @@ If a community doesn't work as expected, please open an issue and ping the autho
1818
| Composable Stable Diffusion| Stable Diffusion Pipeline that supports prompts that contain "|" in prompts (as an AND condition) and weights (separated by "|" as well) to positively / negatively weight prompts. | [Composable Stable Diffusion](#composable-stable-diffusion) | - | [Mark Rich](https://github.com/MarkRich) |
1919
| Seed Resizing Stable Diffusion| Stable Diffusion Pipeline that supports resizing an image and retaining the concepts of the 512 by 512 generation. | [Seed Resizing](#seed-resizing) | - | [Mark Rich](https://github.com/MarkRich) |
2020

21+
| Imagic Stable Diffusion | Stable Diffusion Pipeline that enables writing a text prompt to edit an existing image| [Imagic Stable Diffusion](#imagic-stable-diffusion) | - | [Mark Rich](https://github.com/MarkRich) |
2122

2223

2324
To load a custom pipeline you just need to pass the `custom_pipeline` argument to `DiffusionPipeline`, as one of the files in `diffusers/examples/community`. Feel free to send a PR with your own pipelines, we will merge them quickly.
@@ -373,6 +374,49 @@ for i in range(4):
373374
for i, img in enumerate(images):
374375
img.save(f"./composable_diffusion/image_{i}.png")
375376
```
377+
378+
### Imagic Stable Diffusion
379+
Allows you to edit an image using stable diffusion.
380+
381+
```python
382+
import requests
383+
from PIL import Image
384+
from io import BytesIO
385+
import torch
386+
from diffusers import DiffusionPipeline, DDIMScheduler
387+
has_cuda = torch.cuda.is_available()
388+
device = torch.device('cpu' if not has_cuda else 'cuda')
389+
pipe = DiffusionPipeline.from_pretrained(
390+
"CompVis/stable-diffusion-v1-4",
391+
safety_checker=None,
392+
use_auth_token=True,
393+
custom_pipeline="imagic_stable_diffusion",
394+
scheduler = DDIMScheduler(beta_start=0.00085, beta_end=0.012, beta_schedule="scaled_linear", clip_sample=False, set_alpha_to_one=False)
395+
).to(device)
396+
generator = th.Generator("cuda").manual_seed(0)
397+
seed = 0
398+
prompt = "A photo of Barack Obama smiling with a big grin"
399+
url = 'https://www.dropbox.com/s/6tlwzr73jd1r9yk/obama.png?dl=1'
400+
response = requests.get(url)
401+
init_image = Image.open(BytesIO(response.content)).convert("RGB")
402+
init_image = init_image.resize((512, 512))
403+
res = pipe.train(
404+
prompt,
405+
init_image,
406+
guidance_scale=7.5,
407+
num_inference_steps=50,
408+
generator=generator)
409+
res = pipe(alpha=1)
410+
image = res.images[0]
411+
image.save('./imagic/imagic_image_alpha_1.png')
412+
res = pipe(alpha=1.5)
413+
image = res.images[0]
414+
image.save('./imagic/imagic_image_alpha_1_5.png')
415+
res = pipe(alpha=2)
416+
image = res.images[0]
417+
image.save('./imagic/imagic_image_alpha_2.png')
418+
```
419+
376420
### Seed Resizing
377421
Test seed resizing. Originally generate an image in 512 by 512, then generate image with same seed at 512 by 592 using seed resizing. Finally, generate 512 by 592 using original stable diffusion pipeline.
378422

@@ -456,4 +500,4 @@ res = pipe_compare(
456500

457501
image = res.images[0]
458502
image.save('./seed_resize/seed_resize_{w}_{h}_image_compare.png'.format(w=width, h=height))
459-
```
503+
```

0 commit comments

Comments
 (0)