Stable Diffusion

Stable Diffusion is a text-to-image model that generates images from text prompts using a diffusion process. It runs on consumer GPUs with at least 6 GB VRAM, making it accessible for local deployment. Operators encounter it through tools like AUTOMATIC1111's WebUI or ComfyUI, where they can adjust parameters like steps, CFG scale, and sampler to control output quality and speed.

Stable Diffusion works by starting with random noise and iteratively denoising it over a series of steps, guided by a text prompt encoded via a CLIP text encoder. The core model is a U-Net that predicts the noise to remove at each step. Variants include SD 1.5, SD 2.1, SDXL (larger, higher resolution), and SD 3 (with a different architecture). Operators can use different samplers (e.g., Euler, DPM++ 2M Karras) that affect generation speed and quality. The model is typically run in FP16, requiring about 4-8 GB VRAM for SD 1.5 and 8-12 GB for SDXL. Quantization (e.g., to INT8) can reduce VRAM usage but may degrade quality.

An operator with an RTX 3060 12 GB can run SDXL at 1024x1024 resolution with around 20 steps using the Euler sampler, generating an image in about 10-15 seconds. Using a more advanced sampler like DPM++ 2M Karras may take 20-25 seconds but yields better quality. Lowering resolution to 768x768 speeds up generation to 5-8 seconds.

In AUTOMATIC1111's WebUI, an operator inputs a prompt like 'a cat wearing a hat', selects the SDXL model, sets steps to 30, CFG scale to 7, and sampler to 'Euler a'. After clicking Generate, the UI shows a progress bar and outputs a 1024x1024 image in ~15 seconds on an RTX 3060. The operator can then tweak settings or use img2img to modify the result.