Skip to content

Learning Representations by Back-propagating Errors

Authors: David E. Rumelhart, Geoffrey E. Hinton, Ronald J. Williams Year: 1986 Venue: Nature, 1986 Citations: 60,000+

Summary

Seminal work introducing backpropagation algorithm for training multilayer neural networks. Error gradients propagated backward through layers via chain rule, enabling deep networks to learn complex patterns. Foundation of modern deep learning.

Key Concepts

  • Backpropagation Algorithm: Efficient gradient computation via chain rule
  • Chain Rule in Layers: Propagates error signals backward
  • Gradient Descent: Iterative weight updates using computed gradients
  • Multilayer Perceptrons: Networks with hidden layers
  • Hidden Representations: Automatic non-linear transformations

Impact

  • 60,000+ citations - revolutionized neural networks
  • Made practical deep learning feasible
  • Solved credit assignment problem
  • Foundation of CNNs, RNNs, Transformers
  • Transformed AI to practical engineering
  • Perceptrons (Rosenblatt, 1958)
  • Stochastic Gradient Descent (Robbins & Monro, 1951)