Learn · Intermediate

Classifier-Free Guidance

Classifier-free guidance is the technique that makes text-to-image and text-to-video models follow prompts. It works by running the model twice at every denoising step - once conditioned on your prompt, once on nothing - and then extrapolating along the direction between the two predictions. Introduced by Jonathan Ho and Tim Salimans at Google in 2022, it is now the default sampling method in essentially every production diffusion system, and the "CFG scale" slider in every image-generation interface is its one exposed parameter.

To see why it is needed, start with what a diffusion model is doing. Training teaches it to remove noise: take a real image, corrupt it with a known amount of noise, and train the network to predict what was added. At generation time you start from pure noise and repeatedly subtract the model's prediction, and a coherent image emerges over many steps.

Now add a text prompt. The model has learned the joint distribution of images and captions, so conditioning on "a red bicycle in the rain" should nudge every denoising step toward that region. In practice, it barely does. The model has seen vastly more about what images look like in general than about what any specific caption means, so the pull of ordinary image plausibility overwhelms the pull of your particular words. You ask for a red bicycle in the rain and get a competent photograph of a bicycle on a dry street.

The first fix, from Prafulla Dhariwal and Alex Nichol's Diffusion Models Beat GANs, was classifier guidance. Train a separate classifier that can recognize the target class even in noisy images, then at each step nudge the sample in the direction that increases the classifier's confidence. It worked, and it was awkward: you had to train and maintain an extra network, and it had to work on noisy inputs, which ordinary classifiers do not.

Ho and Salimans's insight was that the extra network is unnecessary, because the model already contains the information. During training, drop the caption on some fraction of examples - typically around 10% - replacing it with a null token. One network thereby learns two behaviors: predicting noise given a prompt, and predicting noise given nothing.

At sampling time you query both. The conditioned prediction says "here is where an image matching this prompt lives." The unconditioned prediction says "here is where any plausible image lives." The vector between them isolates the part of the model's belief that is specifically because of your prompt - and guidance simply pushes further along it:

guided = unconditioned + scale x (conditioned - unconditioned)

At scale 1 the formula collapses to the ordinary conditioned prediction. Above 1, you are extrapolating beyond it - deliberately overshooting into territory the model considers more prompt-specific than any real training example was.

An analogy. Imagine two chefs describing dinner. One knows the request was "something spicy"; the other only knows it is dinner. Both suggest a dish. The difference between their suggestions is the pure signal of "spicy," stripped of everything that is just generic dinner-ness. Guidance takes that difference and amplifies it - which is why turning the scale up gives you more obviously spicy food, and why turning it far too high gives you something inedible.

That failure mode is real and worth knowing. Typical scales run from 5 to 8. Push higher and images become oversaturated, high-contrast, and strangely rigid, because you are extrapolating into a region the model was never trained to represent. Guidance also visibly reduces diversity: strongly guided samples converge toward one prototypical rendering of the prompt. It buys fidelity with variety, and there is no setting that gets both.

The cost is the other honest caveat: every step runs the model twice, so guided sampling is roughly twice as expensive as unguided. Since a diffusion sample takes many steps, that doubling is significant, and a great deal of subsequent research - including diffusion distillation and Salimans and Ho's progressive distillation - exists to fold guidance into a model that only needs one pass, or to cut the step count entirely. Note that heavily distilled models often ignore the guidance scale altogether, because the guided behavior was baked in during distillation; setting a negative prompt on such a model can do literally nothing.

The negative prompt, incidentally, is a small extension of the same idea. Instead of comparing your prompt against nothing, compare it against a description of what you do not want. The unconditioned branch is replaced by an anti-conditioned one, and the sampler pushes away from it.

That second branch turns out to be subtler than it looks. A July 2026 paper on distilling guided diffusion models identifies a failure the authors call Negative Branch Asymmetry: if a student model is trained only to match the teacher's final combined prediction, errors in the positive and negative branches can cancel out, so the student learns a shortcut that reproduces the sum without understanding either part. It looks fine at the guidance scale it was trained at and breaks when the user changes the slider. Their fix is to supervise the positive prediction and the positive-minus-negative direction separately, which removes the cancellation solution.

The general lesson generalizes past diffusion: guidance is not a property of the model, it is a property of the difference between two of its predictions. Anywhere you can ask a model both "with X" and "without X," you can isolate and amplify X. That framing has since shown up in language-model decoding, constrained decoding work, and safety-oriented steering.

To go deeper, read the original paper - it is unusually short - then latent diffusion for how this fits into a real production system. Related lessons: diffusion models, flow matching, and how AI picks its next word.

Ground Truth. (2026, July 28). Classifier-Free Guidance. Ground Truth. https://groundtruth.day/learn/classifier-free-guidance.html

@misc{groundtruth:classifier-free-guidance,
  title  = {Classifier-Free Guidance},
  author = {{Ground Truth}},
  year   = {2026},
  month  = {jul},
  url    = {https://groundtruth.day/learn/classifier-free-guidance.html}
}

Topics: diffusion · image-generation · video-generation · sampling · fundamentals