Long Short-Term Memory Recurrent Neural Networks

Authors: Hochreiter, S., & Schmidhuber, J. Year: 1997 ArXiv: https://www.bioinf.jku.at/publications/older/2604.pdf

Summary

LSTM introduces memory cells with gates (input, forget, output) that enable learning long-range dependencies. It solves the vanishing gradient problem in standard RNNs and became the dominant architecture for sequential modeling.

Key Concepts

• Memory cell with input/forget/output gates
• Gating mechanism for information flow
• Long-range dependency learning
• Handles variable-length sequences
• Predecessor to transformer attention

Impact

LSTM revolutionized sequence modeling in NLP and time series. For 20+ years, it remained the standard for any task requiring sequential processing before being largely replaced by transformers.

Related Papers

• GRU: Gated Recurrent Unit
• Sequence to Sequence with LSTMs
• Attention Is All You Need

---

Citation:

@article{hochreiter1997long,
  title={Long short-term memory},
  author={Hochreiter, Sepp and Schmidhuber, J{\"u}rgen},
  journal={Neural Computation},
  volume={9},
  number={8},
  pages={1735--1780},
  year={1997}
}