Classic Foundations (Pre-2012)

Foundational papers that established core concepts in machine learning, information retrieval, and neural networks.

Word Embeddings & NLP

• Efficient Estimation of Word Representations in Vector Space (Word2Vec) - Skip-gram and CBOW embeddings

Machine Learning Algorithms

• Backpropagation: Learning in Multilayer Perceptrons - Foundational learning algorithm
• Support Vector Machines - Maximum margin classifiers
• Classification and Regression Trees (CART) - Decision tree methodology
• Random Forests - Ensemble of decision trees
• Gradient Boosting Machines - Sequential ensemble learning

Information Retrieval

• TF-IDF: Term Frequency-Inverse Document Frequency - Statistical term weighting
• Okapi BM25: Probabilistic Retrieval Model - State-of-the-art ranking
• The PageRank Citation Ranking: Bringing Order to the Web - Link-based ranking

Sequence Modeling

• Recurrent Neural Networks (RNNs) & LSTM Fundamentals - Long short-term memory for sequences

Computer Vision

• Gradient-Based Learning Applied to Document Recognition (LeNet) - Foundational CNNs

Bayesian & Probabilistic Methods

• The EM Algorithm: Maximum Likelihood from Incomplete Data - EM for latent variables
• Latent Dirichlet Allocation (Topic Modeling) - Generative topic model

---

Key Insight: These papers established the mathematical and algorithmic foundations that power modern deep learning and NLP.