Notes
01 langfuse
Initializing search
Notes
00 papers
01 agent design
01 modeling
01 models
02 open source
05 py3
06 pytorch
09 jax
Notes
Notes
00 papers
00 papers
Papers
00 classic foundations
00 classic foundations
Classic Foundations (Pre-2012)
A Statistical Interpretation of Term Specificity and Its Application in Retrieval
The EM Algorithm: Maximum Likelihood from Incomplete Data
Learning Representations by Back-propagating Errors
Okapi at TREC-3: Probabilistic Retrieval Model
Classification and Regression Trees
Support Vector Machines
Gradient-Based Learning Applied to Document Recognition
Gradient-Based Learning Applied to Document Recognition (LeNet)
The PageRank Citation Ranking: Bringing Order to the Web
Gradient Boosting Machines
Random Forests
Latent Dirichlet Allocation (Topic Modeling)
01 deep learning era
01 deep learning era
Deep Learning Era (2012-2016)
Auto-Encoding Variational Bayes (Variational Autoencoders)
Rectified Linear Units Improve Restricted Boltzmann Machines
A Primer on Neural Network Architectures for Natural Language Processing
ImageNet Classification with Deep Convolutional Neural Networks (AlexNet)
ImageNet Classification with Deep Convolutional Neural Networks
Dropout: A Simple Way to Prevent Neural Networks from Overfitting
ImageNet Large Scale Visual Recognition Challenge
RMSProp: Neural Networks for Machine Learning
Playing Atari with Deep Reinforcement Learning (Deep Q-Networks)
Efficient Estimation of Word Representations in Vector Space (Word2Vec)
Adam: A Method for Stochastic Optimization
Adam: A Method for Stochastic Optimization
Generative Adversarial Networks (GANs)
GloVe: Global Vectors for Word Representation
Rich feature hierarchies for accurate object detection and semantic segmentation
Rich feature hierarchies for accurate object detection and semantic segmentation (R-CNN)
Very Deep Convolutional Networks for Large-Scale Image Recognition
Very Deep Convolutional Networks for Large-Scale Image Recognition (VGGNet)
Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift
Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks
Long Short-Term Memory Recurrent Neural Networks
Deep Residual Learning for Image Recognition
You Only Look Once: Unified, Real-Time Object Detection
Asynchronous Methods for Deep Reinforcement Learning (A3C)
Densely Connected Convolutional Networks (DenseNet)
Communication-Efficient Learning of Deep Networks from Decentralized Data (Federated Averaging)
Inception-v4, Inception-ResNet and the Impact of Residual Connections
Mastering the Game of Go with Deep Neural Networks and Tree Search (AlphaGo)
SqueezeNet: AlexNet-level Accuracy with 50x Fewer Parameters
SSD: Single Shot MultiBox Detector
XGBoost: A Scalable Tree Boosting System
You Only Look Once: Unified, Real-Time Object Detection
Inductive Representation Learning on Large Graphs (GraphSAGE)
Model-Agnostic Meta-Learning for Fast Adaptation of Deep Networks (MAML)
Prototypical Networks for Few-shot Learning
Proximal Policy Optimization Algorithms (PPO)
Semi-Supervised Classification with Graph Convolutional Networks (GCN)
Squeeze-and-Excitation Networks
02 transformer architectures
02 transformer architectures
Transformer Architectures (2017-2023)
Attention Is All You Need
BERT: Pre-training of Deep Bidirectional Transformers
BERT: Pre-training of Deep Bidirectional Transformers
Deep contextualized word representations (ELMo)
Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer
Introducing LLMs with Sparse Attention
Language Models are Unsupervised Multitask Learners
Multi-Head Attention Visualization
RoBERTa: A Robustly Optimized BERT Pretraining Approach
RoFormer: Enhanced Transformer with Rotary Position Embedding
ALiBi: Train Short, Test Long
FlashAttention: Fast and Memory-Efficient Exact Attention with IO-Awareness
FlashAttention-2: Faster Attention with Better Parallelism and Work Partitioning
03 large language models
03 large language models
Large Language Models (2020-2024)
GPT-3: Language Models are Few-Shot Learners
Language Models are Few-Shot Learners
Constitutional AI: Harmlessness from AI Feedback
GPT-4 Technical Report
Llama 2: Open Foundation and Fine-Tuned Chat Models
LLaMA: Open and Efficient Foundation Language Models
Mistral 7B
Mixtral of Experts
04 training techniques
04 training techniques
Training Techniques & Optimization (2019-2023)
Fixing Weight Decay Regularization in Adam
On the Importance of Initialization and Momentum in Deep Learning
Gradient Checkpointing for Efficient Backpropagation
Mixed Precision Training
Adapter: Parameter-Efficient Transfer Learning for NLP
Adapter: Parameter-Efficient Transfer Learning for NLP
Learning to Summarize from Human Feedback
Learning to Summarize from Human Feedback
LoRA: Low-Rank Adaptation of Large Language Models
LoRA: Low-Rank Adaptation of Large Language Models
Prefix Tuning: Optimizing Continuous Prompts for Generation
Prefix Tuning: Optimizing Continuous Prompts for Generation
Supervised Fine-Tuning of Large Language Models
Training Language Models to Follow Instructions with Human Feedback
Scaling Instruction-Finetuned Language Models
Training Language Models to Follow Instructions with Human Feedback
Direct Preference Optimization: Your Language Model is Secretly a Reward Model
The Flan Collection: Designing Data and Methods for Effective Instruction Tuning
QLoRA: Efficient Finetuning of Quantized LLMs
QLoRA: Efficient Finetuning of Quantized LLMs
05 inference deployment
05 inference deployment
Inference & Deployment (2017-2024)
Quantization and Training of Neural Networks
Beam Search, Greedy Decoding & Sampling: Trade-offs
GPTQ: Accurate Post-Training Quantization
AWQ: Activation-Aware Weight Quantization
Efficient Memory Management for Large Language Model Serving
Speculative Decoding for Faster Large Language Model Inference
vLLM: Easy, Fast, and Cheap LLM Serving
Medusa: Parallel Decoding with Multiple Heads
06 retrieval knowledge
06 retrieval knowledge
Retrieval & Knowledge Integration (2019-2023)
Dense Retrieval for Open-Domain Question Answering
Enriching Word Vectors with Subword Information (FastText)
Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks
Dense Passage Retrieval for Open-Domain Question Answering
Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks
When Not to Trust Language Models: Investigating Effectiveness of Parametric & Non-Parametric Memories
07 multimodal vision
07 multimodal vision
Multimodal & Vision (2017-2023)
Mask R-CNN
MobileNets: Efficient Convolutional Neural Networks for Mobile Vision Applications
Focal Loss for Dense Object Detection
ShuffleNet: An Extremely Efficient Convolutional Neural Network for Mobile Devices
EfficientNet: Rethinking Model Scaling for Convolutional Neural Networks
An Image is Worth 16x16 Words: Transformers for Image Recognition
Momentum Contrast for Unsupervised Visual Representation Learning (MoCo)
A Simple Framework for Contrastive Learning of Visual Representations (SimCLR)
CLIP: Learning Transferable Models for Multimodal Learning
Flamingo: a Visual Language Model for Few-Shot Learning
GPT-4V: GPT-4 with Vision Capabilities
LLaVA: Large Language and Vision Assistant
08 reasoning interpretability
08 reasoning interpretability
Reasoning & Interpretability (2013-2023)
The Saliency MAP Shows What Regular Visualization Cannot
Attention Is Not Explanation
Chain-of-Thought Prompting Elicits Reasoning in Large Language Models
Self-Consistency Improves Chain of Thought Reasoning
Tree of Thoughts: Deliberate Problem Solving with LLMs
09 safety alignment
09 safety alignment
Safety & Alignment (2013-2023)
Adversarial Examples Are Not Bugs, They Are Features
Universal Adversarial Triggers for Attacking and Analyzing NLP
TruthfulQA: Measuring How Models Mimic Human Falsehoods
Holistic Evaluation of Language Models
Prompt Injection Attacks on Language Models
10 benchmarking evaluation
10 benchmarking evaluation
Benchmarking & Evaluation (2004-2023)
ROUGE: A Package for Automatic Evaluation of Summarization
GLUE: A Multi-Task Benchmark for NLU
BERTScore: Evaluating Text Generation with BERT
SuperGLUE: A Stickier Benchmark for General-Purpose Language Understanding Systems
MMLU: Measuring Massive Multitask Language Understanding
MT-Bench: A Benchmark for Evaluating Language Model Instruction Following
11 prompting context
11 prompting context
Prompting & In-Context Learning (2020-2021)
Prompt Learning for Few-Shot Finetuning of Language Models
12 emerging topics
12 emerging topics
Emerging Topics (2015-2023)
Distilling the Knowledge in a Neural Network
ONNX: Open Neural Network Exchange
ReAct: Synergizing Reasoning and Acting in Language Models
Generative Agents: Interactive Simulacra of Human Behavior
LongNet: Scaling Transformers to 1M tokens
01 agent design
01 agent design
Agentic System Design
01 foundations
01 foundations
Part 1
Agent Anatomy
Agent Loop
Design Philosophy
Evolution
02 core design patterns
02 core design patterns
Part 2
Andrew Ng's 4 Core Patterns
Anthropic's 5 Workflow Patterns
12 Foundational Patterns
Pattern Selection Framework
03 architecture
03 architecture
Part 3
Single Agent Architecture
Multi-Agent Systems
Coordination Strategies
Graph-Based Orchestration
Hierarchical Agent Systems
04 memory systems
04 memory systems
Part 4
Memory Fundamentals
Short-Term Memory
Long-Term Memory
Vector Stores & Retrieval
Memory Compression
Episodic & Procedural Memory
05 planning reasoning
05 planning reasoning
Part 5
Planning Fundamentals
Chain-of-Thought
Tree-of-Thought
Goal-Oriented Reasoning
Adaptive Planning
06 tool use
06 tool use
Part 6
Function Calling
Tool Interfaces
Tool Composition
Error Handling
Tool Discovery
Grounding in Reality
07 safety reliability
07 safety reliability
Part 7
Safety Fundamentals
Input Validation & Injection Prevention
Output Safety & Content Filtering
Bounded Execution
Policy Enforcement
Monitoring & Detection
Reliability Patterns
08 evaluation
08 evaluation
Part 8
Agent Benchmarks
Multi-Dimensional Assessment
Lab vs Production
Custom Evaluation Frameworks
Behavioral Testing
09 production patterns
09 production patterns
Part 9
State Management & Recovery
Routing & Escalation
Human-in-the-Loop
Context Management
Observability & Monitoring
Error Recovery
10 advanced topics
10 advanced topics
Part 10
Reflection & Self-Critique
Self-Evolution & Learning
Emergent Behaviors
Swarm Agents
Reasoning Optimization
11 frameworks
11 frameworks
Part 11
LangGraph
11 frameworks technologies
11 frameworks technologies
Part 11
MCP Protocol
MCP 2.0
Skills System
Claude Agents API
Subagents
LangGraph
CrewAI
AutoGen
Additional Frameworks & Quick Reference
Framework Comparison & Decision Guide
Structured Outputs
Prompt Caching
Batch Processing API
Vision & Multimodal
Extended Thinking & Reasoning
Agent Protocol
Function Calling Protocol Variations
Streaming Protocol
SDK Specifics
Message Protocol
12 applications
12 applications
Part 12
Enterprise Use Cases
Code Generation & Software Development
Research & Analysis
Creative Applications
Integration Patterns
Scaling Strategies
01 modeling
01 modeling
LLM Optimization & Modeling Knowledge Base
00 fundamentals
00 fundamentals
Fundamentals
00 tokenization
00 tokenization
Index
00 tokenization fundamentals
BPE
SentencePiece
Token Efficiency & Compression
Tokenization Best Practices
Multilingual Tokenization
WordPiece
01 attention
01 attention
Attention Mechanisms
RNN, LSTM & GRU
Query, Key & Value
Scaled Dot-Product Attention
Types of Attention
Multi-Head Attention
Position Information
Attention Complexity & Cost
PyTorch Implementation of Attention
Interpretability
01 architecture
01 architecture
01-Architecture
01 core designs
01 core designs
Mixture of Experts (MoE)
Multi-Query Attention (MQA) & Grouped Query Attention (GQA)
Rotary Position Embeddings (RoPE)
Sliding Window Attention
02 attention optimization
02 attention optimization
Flash Attention v2
Flash Attention
Kernel Fusion
03 memory management
03 memory management
KV Cache
PagedAttention
04 batching
04 batching
Continuous Batching
Medusa
Speculative Decoding
02 training
02 training
01 efficiency
01 efficiency
Distributed Training
Gradient Checkpointing
Mixed Precision Training (AMP)
02 scaling
02 scaling
Scaling Laws & Optimal Allocation
03 compression
03 compression
Model Distillation
Pruning & Sparsity
Token Merging
04 adaptation
04 adaptation
LoRA
Adapter Methods Beyond LoRA
QLoRA
05 alignment
05 alignment
Rlhf
Finetuning
Finetuning
Domain-Specific Fine-tuning
DPO
Fine-tuning Best Practices
Fine-tuning Fundamentals
Instruction Tuning & SFT
03 inference
03 inference
01 optimization
01 optimization
Decoding Strategies
Length Extrapolation & Long Context
LLM Inference Optimization
02 quantization
02 quantization
AWQ Quantization
GPTQ Quantization
03 knowledge integration
03 knowledge integration
Rag
Rag
Chunking & Document Preparation
RAG Evaluation & Metrics
Retrieval-Augmented Generation (RAG)
RAG Integration & Prompt Engineering
Reranking & Ranking
Retrieval Strategies
Vector Databases & Embeddings
Frameworks
Frameworks
Framework Comparison & Selection Guide
Inference Frameworks Fundamentals
vLLM
04 production
04 production
01 deployment
01 deployment
Agent Framework Integration
Cost Optimization Strategies
Load Balancing & Request Routing
02 operations
02 operations
Model Merging & Ensemble Methods
Monitoring & Observability
Safety and alignment
Safety and alignment
Prompt Injection & Adversarial Examples
Safety & Alignment Fundamentals
05 benchmarks
05 benchmarks
LLM Benchmarks & Evaluation Framework
Research Benchmarks
Production Metrics
Capability Classification
Building Custom Benchmarks & Evaluation Frameworks
01 models
01 models
Popular LLM Models
GLM
Anthropic
Anthropic
Anthropic
Claude 1
Claude 2 / 2.1
Claude 3 Family
Deepseek
Deepseek
DeepSeek
DeepSeek LLM
DeepSeek Coder
DeepSeek MoE
Google
Google
Google
Google BERT
Google T5
Google PaLM
Google Gemini
Kimi
Kimi
Kimi
Kimi
Meta
Meta
Meta
Meta Llama 1
Meta Llama 2
Meta Llama 3
Mistral
Mistral
Mistral AI
Mistral 7B
Mixtral 8x7B
Mixtral 8x22B
Openai
Openai
OpenAI
GPT-3
GPT-3.5
GPT-4
02 open source
02 open source
Open-Source Tools for LLM Development
01 training
01 training
Part 1
TRL
Axolotl
Unsloth
Ludwig
02 inference
02 inference
Part 2
vLLM
Ollama
TensorRT-LLM
04 litellm
03 agent building
03 agent building
Part 3
LangChain
02 llamaindex
03 crewai
04 autogen
Haystack
04 knowledge management
04 knowledge management
Part 4
01 chroma
02 elasticsearch
Milvus
LanceDB
05 tool integration
05 tool integration
Part 5
Composio
06 monitoring
06 monitoring
Part 6
01 langfuse
Weights & Biases
MLflow
07 infrastructure
07 infrastructure
Part 7
FastAPI
Docker
Kubernetes
04 langgraph
05 py3
05 py3
Python for ML & LLM Inference
How Python Features Enable ML Frameworks
Complete Outline
Python for ML & LLM Inference - Quick Start
01 fundamentals
01 fundamentals
Type System & Annotations
Data Structures Essentials for ML
Iterator & Generator Protocol
Context Managers & Resource Management
02 object oriented patterns
02 object oriented patterns
Object-Oriented Patterns
Classes & Inheritance
Decorators
Magic Methods
Descriptors & Properties
Metaclasses & Advanced OOP
03 functional programming
03 functional programming
Functional Programming
Functional Programming Fundamentals
Higher-Order Functions & Closures
Function Composition & Piping
Immutability & Persistent Data Structures
JAX & Functional ML
04 c extensions and ffi
04 c extensions and ffi
C Extensions & FFI
ctypes & CFFI
Cython
PyBind11
05 memory and performance
05 memory and performance
Memory & Performance
Reference Counting & Garbage Collection
Memory Layout & Cache Efficiency
Global Interpreter Lock (GIL)
Multithreading vs Multiprocessing
Async/Await
Memory Profiling & Optimization
06 dynamic features
06 dynamic features
Dynamic Features
getattr/setattr & Proxy Objects
Introspection & Reflection
07 advanced typing
07 advanced typing
Advanced Typing
Type Hints & Validation
Generics & Protocols
08 module system
08 module system
Module System
Packages & Organization
Imports & Dependencies
09 bytecode and execution
09 bytecode and execution
Bytecode & Execution
Python Bytecode Fundamentals
Execution Model & Compilation
JIT Compilation & Optimization
Profiling & Performance Analysis
Custom Bytecode & Metaprogramming
10 ml specific patterns
10 ml specific patterns
ML-Specific Patterns
Tensor Abstractions
Autograd Implementation
Distributed Training
Model Serialization
Custom Operators
Inference Optimization Patterns
06 pytorch
06 pytorch
Advanced PyTorch Techniques
01 foundations and tensor mastery
01 foundations and tensor mastery
Foundations & Tensor Mastery
Tensor Internals— Storage, Views & Strides
Autograd— The Gradient Engine
Device & Memory Management
Precision & Numerics— dtypes, fp16, bf16, fp8
02 module and layer engineering
02 module and layer engineering
Module & Layer Engineering
Custom Layers & Advanced Containers
Weight Initialization
Parameter Sharing & Weight Tying
Hooks— Debug, Inject, Extract
03 custom autograd and model transformation
03 custom autograd and model transformation
Custom Autograd & Model Transformation
Custom Autograd.Function
Higher-Order Gradients & Jacobians
fx— Symbolic Model Transformation
04 performance and compilation
04 performance and compilation
Performance & Compilation
torch.compile Deep Dive
CUDA Graphs
Memory Formats & Layouts
Profiling & Benchmarking
Kernel Fusion at the PyTorch Level
05 distributed training
05 distributed training
Distributed Training
DDP in Depth
FSDP Deep Dive
Tensor & Pipeline Parallelism
Checkpointing & Fault Tolerance
06 mixed precision and quantization
06 mixed precision and quantization
Mixed Precision & Quantization
Automatic Mixed Precision
BF16, FP8 & Low-Precision Numerics
Post-Training Quantization
Quantization-Aware Training
07 export deployment and production
07 export deployment and production
Export, Deployment & Production
Export— TorchScript, ONNX, torch.export
Inference Optimization & Serving
C++ & libtorch Integration
08 advanced training techniques
08 advanced training techniques
Advanced Training Techniques
Gradient Accumulation & Clipping
Model Averaging— EMA, SWA
Activation Checkpointing & Memory Restructuring
Schedules, Warmup & Optimizer Tricks
09 custom c++ and cuda extensions
09 custom c++ and cuda extensions
Custom C++ / CUDA Extensions
Building C++ Extensions
Writing Custom CUDA Kernels
Extensions in Practice & Pitfalls
09 jax
09 jax
JAX
01 fundamentals
01 fundamentals
Fundamentals
Chapter 1
Chapter 2
Chapter 3
02 core transformations
02 core transformations
Chapter 4
Chapter 5
Chapter 6
Chapter 7
03 advanced patterns
03 advanced patterns
Chapter 8
Chapter 9
Chapter 10
04 machine learning
04 machine learning
Chapter 11
Chapter 12
Chapter 13
05 production and performance
05 production and performance
Chapter 14
Chapter 15
Chapter 16
01 langfuse
Back to top