Running on 8 GB of GPU VRAM

[neurogen-dev]

I would like to report that I was able to run and generate images on the RTX 3060 Ti. To do this I had to close all unnecessary programs, activate flash-attention and generate with the largest side of 512 pixels.

But it took up all my available video memory. I hope in the future to support xformers / sdp or some other effective ways to reduce memory consumption.

[seruva19]

Thank you! I now recall you wrote about this fact on https://t.me/text2image, but I somehow ignored it 🙄
I will add it into wiki.
Yes, I am researching a way to optimize VRAM consumption. The goal is to make possible 768x768 inference with >= 8Gb and training with >= 16 Gb.
There is also an ongoing research on a way to integrate Kandinsky into diffusers pipeline (huggingface/diffusers#2985), eventually it might also help to find an efficient way to lower VRAM usage.