Landmark 2 stops to get here · leads to 2

Diffusion Model

A generative model trained to remove noise a step at a time, so it can turn pure random noise into a new image, audio clip or other sample.

Your route here

2 stops · basics first
  1. Machine Learning ✓ understood

    Building systems that learn patterns from data instead of following hand-written rules, getting better at a task as they see more examples.

  2. Neural Network ✓ understood

    A model built from layers of simple units, each taking a weighted sum of its inputs and applying a nonlinearity; training adjusts the weights so the whole stack maps inputs to useful outputs.

  3. Diffusion Model · you are here ✓ understood

Picture it

Forward (noising)

  • Fixed, no learning
  • Add a little Gaussian noise
  • Repeat until pure noise
  • Makes training examples

Reverse (denoising)

  • Learned neural network
  • Predict and remove noise
  • Repeat step by step
  • Pure noise becomes an image
Notice that the model only learns the reverse direction; generation starts from random noise and denoises step by step.

A diffusion model generates data by learning to undo noise. During training, real examples are gradually corrupted with random noise until nothing of the original is left, and a neural network learns to reverse each small step. To create something new, you start from pure noise and run the learned reversal many times until a clean sample appears.

The idea comes from a 2015 paper by Sohl-Dickstein and colleagues, borrowed from non-equilibrium physics: slowly destroy the structure in data, then learn a process that restores it. It became a serious image generator in 2020, when Ho, Jain and Abbeel’s denoising diffusion probabilistic models reached a state-of-the-art image-quality score for unconditional CIFAR-10 generation, ahead of the GANs that had dominated the field.

The two directions

The diagram’s two columns are not mirror images. The forward process has nothing to learn: it’s a fixed recipe that adds a little Gaussian noise at each of many steps (the DDPM experiments used 1,000). Because the recipe is known, you can jump straight to any noise level for any training image in one calculation, which makes each training example cheap.

The reverse process is the learned part. In DDPM a U-Net looks at a noisy image and the step number and predicts the noise that was added; removing part of that prediction moves the sample one step closer to clean data. The training objective is almost disarmingly simple: add known noise, ask the network to predict it, penalize the difference.

Why it caught on

  • Stable training. Each step is an ordinary regression problem, with no fragile generator-versus-discriminator balance to manage.
  • Easy to steer. Conditioning the denoiser on text, a layout or a masked image turns the same machinery into prompt-driven image generation, inpainting or super-resolution.
  • Variety. Different starting noise gives genuinely different samples.

The catch

Generation is slow: every sample needs many sequential passes through the network, where a GAN needs one. Working on raw pixels at high resolution is also expensive; the latent diffusion paper notes that pixel-space models often consume hundreds of GPU days to train. Latent diffusion moved the denoising into the compressed latent space of a pretrained autoencoder, cutting that cost while keeping quality, and it became the basis of Stable Diffusion. Reducing the number of sampling steps remains an active line of research.

Where it sits

Before this

Neural Network
Diffusion Model

Leads to

Latent Diffusion Stable Diffusion

Explore nearby

Vision & Multimodal Latent Diffusion A diffusion model that runs its denoising in the compressed latent space of a pretrained autoencoder instead of on raw pixels, cutting compute while keeping detail. Vision & Multimodal Stable Diffusion A latent diffusion model for text-to-image generation that operates in compressed latent space for efficiency. Vision & Multimodal Image Generation Creating new images, often from a text prompt, with generative models such as GANs, VAEs and diffusion models that learn to turn random noise into realistic pictures. Vision & Multimodal Generative Adversarial Network A framework where two networks (generator and discriminator) compete, with the generator learning to create realistic data. Neural Networks Variational Autoencoder A generative model that learns a probabilistic latent space, allowing sampling of new data points similar to training data. Vision & Multimodal U-Net A CNN architecture with encoder-decoder structure and skip connections, widely used for image segmentation tasks.

In the research

All papers →

20 papers that build on Diffusion Model ; showing 5, canon first.

Canon · 2020 Denoising Diffusion Probabilistic Models DDPM is the recipe behind Stable Diffusion, DALL·E, Midjourney and today’s video models. Canon · 2021 High-Resolution Image Synthesis with Latent Diffusion Models It’s the architecture behind Stable Diffusion, which put high-quality text-to-image generation on ordinary GPUs. Frontier · Aug 2025 · 875 citations Qwen-Image Technical Report Legible, editable text in generated images was a long-standing weakness; this open model largely fixes it. Frontier · Sep 2025 · 226 citations Seedream 4.0: Toward Next-generation Multimodal Image Generation It set the bar for commercial image generation and editing in one fast system. Frontier · Oct 2025 · 221 citations Diffusion Transformers with Representation Autoencoders It argues the generation and understanding sides of vision should share one latent space.

Sources

  1. Sohl-Dickstein et al., "Deep Unsupervised Learning using Nonequilibrium Thermodynamics" . 2015
  2. Ho, Jain and Abbeel, "Denoising Diffusion Probabilistic Models" . 2020
  3. Rombach et al., "High-Resolution Image Synthesis with Latent Diffusion Models" . 2021