Skip to content

Auto-Encoding Variational Bayes (Variational Autoencoders)

Authors: Diederik P. Kingma, Max Welling (2013, published 2014) Year: 2013 (arXiv), Published 2014 Venue: ICLR 2014 Citations: 30,000+

Summary

Introduces Variational Autoencoders (VAEs), combining variational inference with deep generative models. Learns to encode data to latent distribution and decode from it. Foundation for probabilistic generative models and variational inference in deep learning.

Key Concepts

  • Variational Inference: Approximate posterior with tractable distribution
  • ELBO (Evidence Lower Bound): Variational objective combines reconstruction and KL divergence
  • Encoder-Decoder: Neural networks parameterize inference and generative models
  • Latent Variables: Learns meaningful latent representations
  • Reparameterization Trick: Enables backpropagation through sampling
  • Generative Modeling: Learns to generate new samples from latent space

Impact

  • 30,000+ citations
  • Foundation of modern deep generative models
  • Inspired Diffusion Models and modern generative AI
  • Enabled practical variational inference with deep learning
  • Widely used for unsupervised learning and generation
  • Connected Bayesian methods with deep learning
  • Influenced VAE variants (beta-VAE, conditional VAE, etc.)

Key Results

  • Learned meaningful latent representations
  • Generated new images from latent space
  • Better reconstruction than autoencoders
  • Principled probabilistic framework
  • Variational Inference (Jordan et al., 1999)
  • Deep Generative Models (Bengio et al., 2013)
  • Generative Adversarial Networks (Goodfellow et al., 2014)
  • Diffusion Models (Ho et al., 2020)