This tutorial will walk through how to run the Stable Diffusion software on Windows computer for free. Stable Diffusion is an artificial intelligence software that creates images from text. This is a totally awesome software because from now on, I can create my own images for the blog articles!
Take a look at what the software can generate just using the default settings:
Art title: “final fantasy, futuristic city, realistic, 8k highly detailed, hyperdetailed, digital art, matte”by Jay’s Comptuer (and Stable Diffusion)
This software is mostly written in Python, so that’s another good reason for me to take look. The project’s official github page is here: https://github.com/CompVis/stable-diffusion
As of Sept 2, 2022, Stable Diffusion:
- Can only run on Nvidia GPU (graphic card), and it doesn’t work on AMD.
- The minimum video RAM required is 4 GB, for this, we’ll need to use an “optimized” version.
Follow these steps to download and run Stable Difussion on a Windows computer.
Download The Model Weights
The model weights are several gigabytes so let’s start with downloading them first. The latest model weights is version 1-4, and can be found here: https://huggingface.co/CompVis/stable-diffusion-v-1-4-original
You’ll need to register an account at Hugging Face and log in to gain access. While this is downloading, let’s set up other things.
Download the Stable Diffusion Source Code
There are two versions of the source code, and we’ll want to get a copy of both.
Official Version (not optimized)
Stable Diffusion is an open source software and written in Python. The official github project page is here: https://github.com/CompVis/stable-diffusion. Feel free to grab a copy of the code.
The official version requires a high-end graphic card with at least 10GB VRAM to generate a 512 x 512 image at default settings. Someone named Basu created an “optimized” version that will allow running the software on a 4GB VRAM graphic card. We can get the optimized version here: https://github.com/basujindal/stable-diffusion. Thanks, Basu!
From the optimized version, we only need files from the folder “optimizedSD”, what I did was copy this folder and put it inside the official version.
Now we have the source code, let’s get Python.
Install Anaconda & Set Up Environment
Maybe you are very used to using pip for installation, I get it (I’m the same). However, using the Anaconda Distribution will make it so much easier, so I highly recommend getting Anaconda first, then setting up the environment. I will explain why very soon.
Go to the Anaconda official website https://www.anaconda.com, and download and install Anaconda. If your Anaconda doesn’t launch properly after the installation, refer to this guide to fix the problem: How to Fix Anaconda Doesn’t Launch Issue.
After installing Anaconda, open the Anaconda Prompt, and navigate to the folder that contains the source code.
This folder contains an “environment.yaml” file, which is like a configuration file for building the Python (conda) environments. This is similar to the “requirements.txt” for pip. feel free to open up the file (in text editor) and see what dependencies are needed for running Stable Diffusion.
In the Anaconda Promot, type the following, which will first create a conda environment (download and install all required dependencies), and then activate it.
conda env create -f environment.yaml conda activate ldm
The above command will also install CUDA, which is a software developed by Nvidia that allows other software to directly control the graphic card for parallel computation. CUDA is not a Python library and not on the Pypi index. This is the reason we don’t want to use pip to set the environment – because we’ll have to jump through a few hoops.
One Last Step – Model Weights
By now, the model weight we downloaded at the beginning is probably finished. We will need to move it to a specific location.
Go to this folder first: \stable-diffusion-main\models\ldm
Then create a new folder, name it stable-diffusion-v1. This folder did not exist when we first downloaded the code.
Move the downloaded weight file sd-v1-4.ckpt into this new folder we just created, then rename the weight file to model.ckpt. It should look like this after you complete the above steps:
Start Creating Art!
Now the setup is finished, and we are ready to run Stable Diffusion on our Windows computer! Inside the same Anaconda Prompt, type the following command:
python scripts/txt2img.py --prompt "beautiful portrait, hyperdetailed, art by Yoshitaka Amano, 8k highly detailed" --plms
Note the string following –prompt is the text input describing what we want for the image. Feel free to try whatever is creative here. Each time we run the software, Stable Diffusion will give random results. Here are a few examples of what I got from the above prompt (in the code):
Wait, The Program Crashed??
Yes, without tweaking, you’ll need a very high-end (beyond consumer grade) graphic card to run this program. Below are two ways to run this program without crashing
If You Have A Good Graphic Card (RTX 3080+)
The bad news is the official version (un-optimized) still does not run on a 3080 card. I’m not sure about 3090 and above though. You might get an “out of memory” error. Yes, it can go out of memory on a 10GB VRAM card…
With this kind of GPU, we have two options.
Option 1: Minor adjustment in the txt2img.py file
Add model.half() inside the function load_model_from_config(). Something like the one below. This will significantly reduce the VRAM usage.
With this change, I was able to generate images using an RTX 3080, almost maxing out the VRAM.
Option 2: Use the optimized Stable Diffusion version
This version will allow us to run Stable Diffusion with a 4GB graphic card. We simply need to run another Python script inside the optimizedSD folder, which will look like the following. Note this method will take longer time to run because it takes less VRAM.
python optimizedSD\optimized_txt2img.py --prompt "beautiful portrait, hyperdetailed, art by Yoshitaka Amano, 8k highly detailed"
If You Have A Lower End Graphic Card (RTX 2000s and below)
For a 4GB VRAM graphic card, the best option is to use the “optimized” version. See Option 2 above.
Enjoy creating your art!