To create images with Stable Diffusion you must follow the steps that we describe shortlys. You can experiment with this incredible open source technology, without having a high performance pc, and with the help of Colab.
Stable Diffusion is a text-to-image latent diffusion model created by the researchers and engineers from CompVis, Stability AI and LAION. It’s trained on 512×512 images from a subset of the LAION-5B database. The model uses a frozen CLIP ViT-L/14 text encoder to condition the model on text prompts. With its 860M UNet and 123M text encoder, the model is relatively lightweight and runs on a GPU with at least 10GB VRAM.
You must follow these steps:
Step 1
You must have a google colab account and must have a google colab account:
The notebook comes with its own instructions.
Step 2
Stable diffusion uses a model to generate each image that is significantly large and of a specific version. You also need to accept the model license before downloading or using the weights. In this post we’ll use model version v1-4, so you’ll need to visit its card, read the license and tick the checkbox if you agree.
You have to be a registered user in Hugging Face Hub, and you’ll also need to use an access token for the code to work. For more information on access tokens, please refer to this section of the documentation.
Step 3
Please make sure you are using a GPU runtime to run the notebook, so inference is much faster. If the following command fails, use the:
!nvidia-smi
Enjoy! Now you can use the promts you want whit code like this:
from torch import autocast
prompt = "a photograph of an astronaut riding a horse"
with autocast("cuda"):
image = pipe(prompt)["sample"][0] # image here is in [PIL format](https://pillow.readthedocs.io/en/stable/)
# Now to display an image you can do either save it such as:
image.save(f"astronaut_rides_horse.png")
# or if you're in a google colab you can directly display it with
image
If you need help to be more creative, you can start with this site: https://lexica.art
Stable Diffusion on GitHub: https://github.com/CompVis/stable-diffusion
More python examples: https://geekole.com/python/