Elamur Research
☰
home / papers / 2022 / Classifier-Free Diffusion Guidance

Classifier-Free Diffusion Guidance

Written for the Research library (claude-opus-5-5) from the paper text, in our own words: the license "arXiv — non-exclusive license to distribute" does not allow redistributing the text itself. Original — arxiv.org/pdf/2207.12598

Summary

Diffusion models had no working counterpart to the truncation or "low temperature" sampling used with GANs and flow models. Naively scaling scores or reducing sampling noise only produces blurry images. Classifier guidance (Dhariwal & Nichol, 2021) solved this by pushing samples along the gradients of an extra classifier, but that classifier must be trained on noisy inputs. This makes the pipeline more complicated and raises the suspicion that metric gains come from adversarially exploiting classifiers. Ho and Salimans propose classifier-free guidance. A single network learns both conditional and unconditional denoising: during training, the class label is randomly replaced by a null token with probability p_uncond. During sampling, the two noise predictions are blended with a weight w, and larger w gives higher fidelity at the cost of diversity. On class-conditional ImageNet at 64x64 and 128x128, sweeping w yields an FID/IS trade-off similar to classifier guidance. This shows that a purely generative model can score highly on classifier-based quality metrics without using any classifier gradients.

Key points

  • Method. One continuous-time diffusion model with ε-prediction is trained jointly for conditional and unconditional generation (Algorithm 1). Conditioning is dropped with probability p_uncond, so no separate classifier is needed.
  • ImageNet 64x64 results.
    • Unguided: FID 1.80, IS 53.71.
    • w = 1.0: FID 12.6, IS 170.1.
    • w = 3.0: FID 24.83, IS 250.4.
  • ImageNet 128x128 results.
    • Unguided: FID 7.27, IS 82.45.
    • w = 1.0: FID 7.86, IS 297.98.
    • w = 4.0: FID 21.53, IS 421.03.
  • Trade-off. Stronger guidance raises IS and individual sample quality but reduces diversity. The authors note this may hurt underrepresented parts of the data.
  • Context. The work responds to the classifier guidance of Dhariwal & Nichol (2021). An abbreviated version was presented at a NeurIPS 2021 workshop focused on deep generative models and their downstream applications.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

Fields: Other

Cited by

276 papers of our library cite it (of the 140673 whose bibliographies we read); 8 of them have a page here, 268 not yet.

Cite

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.48550/arxiv.2207.12598 · arXiv 2207.12598

@misc{ho2022classifier,
  title = {{Classifier-Free Diffusion Guidance}},
  author = {Ho, Jonathan and Salimans, Tim},
  year = {2022},
  doi = {10.48550/arxiv.2207.12598},
  eprint = {2207.12598},
  archivePrefix = {arXiv},
  url = {https://doi.org/10.48550/arxiv.2207.12598}
}

References

23 references, 5 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Andrew Brock, Jeff Donahue, and Karen Simonyan. Large scale GAN training for high fidelity natural image synthesis. In International Conference on Learning Representations, 2019. find
  2. Nanxin Chen, Yu Zhang, Heiga Zen, Ron J Weiss, Mohammad Norouzi, and William Chan. Wave-Grad: Estimating gradients for waveform generation. International Conference on Learning Representations, 2021. find
  3. Prafulla Dhariwal and Alex Nichol. Diffusion models beat GANs on image synthesis. arXiv preprint arXiv:2105.05233, 2021. — [arXiv:2105.05233](https://arxiv.org/abs/2105.05233) find
  4. Yves Grandvalet and Yoshua Bengio. Semi-supervised learning by entropy minimization. In Proceedings of the 17th International Conference on Neural Information Processing Systems, pp. 529–536, 2004. find
  5. Peter Grünwald and John Langford. Suboptimal behavior of bayes and mdl in classification under misspecification. Machine Learning, 66(2-3):119–149, 2007. find
  6. Martin Heusel, Hubert Ramsauer, Thomas Unterthiner, Bernhard Nessler, and Sepp Hochreiter. GANs trained by a two time-scale update rule converge to a local Nash equilibrium. In Advances in Neural Information Processing Systems, pp. 6626–6637, 2017. find
  7. Jonathan Ho, Ajay Jain, and Pieter Abbeel. Denoising diffusion probabilistic models. In Advances in Neural Information Processing Systems, pp. 6840–6851, 2020. find
  8. Jonathan Ho, Chitwan Saharia, William Chan, David J Fleet, Mohammad Norouzi, and Tim Salimans. Cascaded diffusion models for high fidelity image generation. arXiv preprint arXiv:2106.15282, 2021. — [arXiv:2106.15282](https://arxiv.org/abs/2106.15282) find
  9. Aapo Hyvärinen and Peter Dayan. Estimation of non-normalized statistical models by score matching. Journal of Machine Learning Research, 6(4), 2005. find
  10. Diederik P Kingma and Prafulla Dhariwal. Glow: Generative flow with invertible 1x1 convolutions. In Advances in Neural Information Processing Systems, pp. 10215–10224, 2018. find
  11. Diederik P Kingma, Tim Salimans, Ben Poole, and Jonathan Ho. Variational diffusion models. arXiv preprint arXiv:2107.00630, 2021. — [arXiv:2107.00630](https://arxiv.org/abs/2107.00630) find
  12. Zhifeng Kong, Wei Ping, Jiaji Huang, Kexin Zhao, and Bryan Catanzaro. DiffWave: A Versatile Diffusion Model for Audio Synthesis. International Conference on Learning Representations, 2021. find
  13. Alex Nichol and Prafulla Dhariwal. Improved denoising diffusion probabilistic models. International Conference on Machine Learning, 2021. find
  14. Ali Razavi, Aaron van den Oord, and Oriol Vinyals. Generating diverse high-fidelity images with VQ-VAE-2. In Advances in Neural Information Processing Systems, pp. 14837–14847, 2019. find
  15. Olga Russakovsky, Jia Deng, Hao Su, Jonathan Krause, Sanjeev Satheesh, Sean Ma, Zhiheng Huang, Andrej Karpathy, Aditya Khosla, Michael Bernstein, et al. ImageNet large scale visual recognition challenge. International Journal of Computer Vision, 115(3):211–252, 2015. find our card
  16. Tim Salimans and Jonathan Ho. Should EBMs model the energy or the score? In Energy Based Models Workshop-ICLR 2021, 2021. find
  17. Tim Salimans, Ian Goodfellow, Wojciech Zaremba, Vicki Cheung, Alec Radford, and Xi Chen. Improved techniques for training GANs. In Advances in Neural Information Processing Systems, pp. 2234–2242, 2016. find our card
  18. Jascha Sohl-Dickstein, Eric Weiss, Niru Maheswaranathan, and Surya Ganguli. Deep unsupervised learning using nonequilibrium thermodynamics. In International Conference on Machine Learning, pp. 2256–2265, 2015. find our card
  19. Yang Song and Stefano Ermon. Generative modeling by estimating gradients of the data distribution. In Advances in Neural Information Processing Systems, pp. 11895–11907, 2019. find our card
  20. Yang Song, Conor Durkan, Iain Murray, and Stefano Ermon. Maximum likelihood training of score-based diffusion models. arXiv e-prints, pp. arXiv–2101, 2021a. find
  21. Yang Song, Jascha Sohl-Dickstein, Diederik P Kingma, Abhishek Kumar, Stefano Ermon, and Ben Poole. Score-based generative modeling through stochastic differential equations. International Conference on Learning Representations, 2021b. find our card
  22. Pascal Vincent. A connection between score matching and denoising autoencoders. Neural Computation, 23(7):1661–1674, 2011. find
  23. Yan Wu, Jeff Donahue, David Balduzzi, Karen Simonyan, and Timothy Lillicrap. LOGAN: Latent optimisation for generative adversarial networks. arXiv preprint arXiv:1912.00953, 2019. — [arXiv:1912.00953](https://arxiv.org/abs/1912.00953) find