Skip to content

Notes

Fundamentals

Initializing search

Notes
00 papers
01 agent design
01 agents system design
01 modeling
01 models
02 llm modeling
02 llm models popular
02 open source
02 open source tools
02 python3
02 pytorch
05 py3
06 pytorch
09 jax

Notes

Notes
00 papers
00 papers
01 agent design
01 agent design
- Agentic System Design
- 01 foundations
  01 foundations
- 02 core design patterns
  02 core design patterns
- 03 architecture
  03 architecture
- 04 memory systems
  04 memory systems
- 05 planning reasoning
  05 planning reasoning
- 06 tool use
  06 tool use
- 07 safety reliability
  07 safety reliability
- 08 evaluation
  08 evaluation
- 09 production patterns
  09 production patterns
- 10 advanced topics
  10 advanced topics
- 11 frameworks
  11 frameworks
  - Part 11
  - LangGraph
- 11 frameworks technologies
  11 frameworks technologies
- 12 applications
  12 applications
01 agents system design
01 agents system design
01 modeling
01 modeling
- LLM Optimization & Modeling Knowledge Base
- 00 fundamentals
  00 fundamentals
  - Fundamentals Fundamentals
    Table of contents
    
    Topics Covered
  - 00 tokenization
    00 tokenization
    
    Index
    
    00 tokenization fundamentals
    
    BPE
    
    SentencePiece
    
    Token Efficiency & Compression
    
    Tokenization Best Practices
    
    Multilingual Tokenization
    
    WordPiece
  - 01 attention
    01 attention
    
    Attention Mechanisms
    
    RNN, LSTM & GRU
    
    Query, Key & Value
    
    Scaled Dot-Product Attention
    
    Types of Attention
    
    Multi-Head Attention
    
    Position Information
    
    Attention Complexity & Cost
    
    PyTorch Implementation of Attention
    
    Interpretability
- 01 architecture
  01 architecture
  - 01-Architecture
  - 01 core designs
    01 core designs
    
    Mixture of Experts (MoE)
    
    Multi-Query Attention (MQA) & Grouped Query Attention (GQA)
    
    Rotary Position Embeddings (RoPE)
    
    Sliding Window Attention
  - 02 attention optimization
    02 attention optimization
    
    Flash Attention v2
    
    Flash Attention
    
    Kernel Fusion
  - 03 memory management
    03 memory management
    
    KV Cache
    
    PagedAttention
  - 04 batching
    04 batching
    
    Continuous Batching
    
    Medusa
    
    Speculative Decoding
- 02 training
  02 training
  - 01 efficiency
    01 efficiency
    
    Distributed Training
    
    Gradient Checkpointing
    
    Mixed Precision Training (AMP)
  - 02 scaling
    02 scaling
    
    Scaling Laws & Optimal Allocation
  - 03 compression
    03 compression
    
    Model Distillation
    
    Pruning & Sparsity
    
    Token Merging
  - 04 adaptation
    04 adaptation
    
    LoRA
    
    Adapter Methods Beyond LoRA
    
    QLoRA
  - 05 alignment
    05 alignment
    
    Rlhf
  - Finetuning
    Finetuning
    
    Domain-Specific Fine-tuning
    
    DPO
    
    Fine-tuning Best Practices
    
    Fine-tuning Fundamentals
    
    Instruction Tuning & SFT
- 03 inference
  03 inference
  - 01 optimization
    01 optimization
    
    Decoding Strategies
    
    Length Extrapolation & Long Context
    
    LLM Inference Optimization
  - 02 quantization
    02 quantization
    
    AWQ Quantization
    
    GPTQ Quantization
  - 03 knowledge integration
    03 knowledge integration
    
    Rag
    Rag
    
    Chunking & Document Preparation
    
    RAG Evaluation & Metrics
    
    Retrieval-Augmented Generation (RAG)
    
    RAG Integration & Prompt Engineering
    
    Reranking & Ranking
    
    Retrieval Strategies
    
    Vector Databases & Embeddings
  - Frameworks
    Frameworks
    
    Framework Comparison & Selection Guide
    
    Inference Frameworks Fundamentals
    
    vLLM
- 04 production
  04 production
  - 01 deployment
    01 deployment
    
    Agent Framework Integration
    
    Cost Optimization Strategies
    
    Load Balancing & Request Routing
  - 02 operations
    02 operations
    
    Model Merging & Ensemble Methods
    
    Monitoring & Observability
    
    Safety and alignment
    Safety and alignment
    
    Prompt Injection & Adversarial Examples
    
    Safety & Alignment Fundamentals
- 05 benchmarks
  05 benchmarks
01 models
01 models
- Popular LLM Models
- GLM
- Anthropic
  Anthropic
- Deepseek
  Deepseek
- Google
  Google
- Kimi
  Kimi
  - Kimi
  - Kimi
- Meta
  Meta
- Mistral
  Mistral
- Openai
  Openai
  - OpenAI
  - GPT-3
  - GPT-3.5
  - GPT-4
02 llm modeling
02 llm modeling
- LLM Optimization & Modeling Knowledge Base
- 00 fundamentals
  00 fundamentals
- 01 architecture
  01 architecture
  - 01-Architecture: LLM Design & Optimization
  - 01 core designs
    01 core designs
    
    Mixture of Experts (MoE): Scaling to Trillions of Parameters
    
    Multi-Query Attention (MQA) & Grouped Query Attention (GQA)
    
    Rotary Position Embeddings (RoPE)
    
    Sliding Window Attention: Efficient Long-Context Attention
  - 02 attention optimization
    02 attention optimization
    
    Flash Attention v2: Improvements and Differences from v1
    
    Flash Attention: Complete Technical Guide
    
    Kernel Fusion: Complete Technical Guide
  - 03 memory management
    03 memory management
    
    KV Cache: Comprehensive Technical Guide
    
    PagedAttention: Detailed Technical Guide
  - 04 batching
    04 batching
    
    Continuous Batching: Complete Technical Guide
    
    Medusa: Multi-Head Decoding for Faster Inference
    
    Speculative Decoding: Complete Technical Guide
- 02 training
  02 training
  - 01 efficiency
    01 efficiency
    
    Distributed Training: Multi-GPU and Multi-Node LLM Training
    
    Gradient Checkpointing: Trading Compute for Memory
    
    Mixed Precision Training (AMP): Speed and Memory with Reduced Precision
  - 02 scaling
    02 scaling
    
    Scaling Laws & Optimal Allocation: Computing the Perfect Model Size
  - 03 compression
    03 compression
    
    Model Distillation: Compressing Knowledge into Smaller Models
    
    Pruning & Sparsity: Removing Redundant Parameters and Activations
    
    Token Merging: Reducing Attention Compute by Merging Tokens
  - 04 adaptation
    04 adaptation
    
    Adapter Methods Beyond LoRA: Efficient Fine-tuning Alternatives
    
    LoRA: Low-Rank Adaptation - Complete Technical Guide
    
    QLoRA: Quantized LoRA - Complete Comparison with LoRA
  - 05 alignment
    05 alignment
    
    Rlhf
  - Finetuning
    Finetuning
    
    Domain-Specific Fine-tuning: Adapting to Your Industry
    
    DPO: Direct Preference Optimization - Learning from Preferences Without RL
    
    Fine-tuning Best Practices: Tips, Tricks & Common Pitfalls
    
    Fine-tuning Fundamentals: Adapting Models to Your Task
    
    Instruction Tuning & SFT: Teaching Models to Follow Instructions
- 03 inference
  03 inference
  - 01 optimization
    01 optimization
    
    Decoding Strategies: Beam Search vs Greedy vs Sampling
    
    Length Extrapolation & Long Context: Training for Generalization to Longer Sequences
    
    LLM Inference Optimization: Complete Guide to Production Serving
  - 02 quantization
    02 quantization
    
    AWQ Quantization: Complete Technical Guide
    
    GPTQ Quantization: Complete Technical Guide
  - 03 knowledge integration
    03 knowledge integration
    
    Rag
    Rag
    
    Chunking & Document Preparation: Preparing Documents for RAG
    
    RAG Evaluation & Metrics: Measuring Performance
    
    Retrieval-Augmented Generation (RAG): Fundamentals
    
    RAG Integration & Prompt Engineering: Using Retrieved Context Effectively
    
    Reranking & Ranking: Improving Retrieval Quality
    
    Retrieval Strategies: Finding the Right Documents
    
    Vector Databases & Embeddings: The Foundation of RAG
  - Frameworks
    Frameworks
    
    Framework Comparison & Selection Guide
    
    Inference Frameworks Fundamentals: Choosing the Right Tool
    
    vLLM: Fast and Easy LLM Inference
- 04 production
  04 production
  - 01 deployment
    01 deployment
    
    Agent Framework Integration: LLM Deployment in Agentic Systems
    
    Cost Optimization Strategies: Reducing Training and Inference Expenses
    
    Load Balancing & Request Routing: Efficient Multi-Model Serving
  - 02 operations
    02 operations
    
    Model Merging & Ensemble Methods: Combining Models for Better Performance
    
    Monitoring & Observability: Production LLM Systems
    
    Safety & alignment
    Safety & alignment
    
    Prompt Injection & Adversarial Examples: Attack Patterns & Defenses
    
    Safety & Alignment Fundamentals: Keeping LLMs Safe and Aligned
- 05 benchmarks
  05 benchmarks
02 llm models popular
02 llm models popular
02 open source
02 open source
- Open-Source Tools for LLM Development
- 01 training
  01 training
  - Part 1
  - TRL
  - Axolotl
  - Unsloth
  - Ludwig
- 02 inference
  02 inference
  - Part 2
  - vLLM
  - Ollama
  - TensorRT-LLM
  - 04 litellm
- 03 agent building
  03 agent building
  - Part 3
  - LangChain
  - 02 llamaindex
  - 03 crewai
  - 04 autogen
  - Haystack
- 04 knowledge management
  04 knowledge management
  - Part 4
  - 01 chroma
  - 02 elasticsearch
  - Milvus
  - LanceDB
- 05 tool integration
  05 tool integration
  - Part 5
  - Composio
- 06 monitoring
  06 monitoring
- 07 infrastructure
  07 infrastructure
  - Part 7
  - FastAPI
  - Docker
  - Kubernetes
  - 04 langgraph
02 open source tools
02 open source tools
- Open-Source Tools for LLM Development
- 01 training
  01 training
- 02 inference
  02 inference
- 03 agent building
  03 agent building
- 04 knowledge management
  04 knowledge management
- 05 tool integration
  05 tool integration
  - Part 5: Tool Integration & Orchestration
  - Composio: Universal Tool Integration
- 06 monitoring
  06 monitoring
- 07 infrastructure
  07 infrastructure
02 python3
02 python3
- Python for ML & LLM Inference
- How Python Features Enable ML Frameworks
- Complete Outline: Python for ML & LLM Inference
- Python for ML & LLM Inference - Quick Start
- 01 fundamentals
  01 fundamentals
- 02 object oriented patterns
  02 object oriented patterns
- 03 functional programming
  03 functional programming
- 04 c extensions & ffi
  04 c extensions & ffi
- 05 memory & performance
  05 memory & performance
- 05 memory\ \&\ performance
  05 memory\ \&\ performance
- 06 dynamic features
  06 dynamic features
- 07 advanced typing
  07 advanced typing
- 08 module system
  08 module system
- 09 bytecode & execution
  09 bytecode & execution
- 10 ml specific patterns
  10 ml specific patterns
02 pytorch
02 pytorch
- 🧠 Advanced PyTorch Techniques
- 01 foundations & tensor mastery
  01 foundations & tensor mastery
- 02 module & layer engineering
  02 module & layer engineering
- 03 custom autograd & model transformation
  03 custom autograd & model transformation
- 04 performance & compilation
  04 performance & compilation
- 05 distributed training
  05 distributed training
- 06 mixed precision & quantization
  06 mixed precision & quantization
- 07 export, deployment & production
  07 export, deployment & production
- 08 advanced training techniques
  08 advanced training techniques
- 09 custom c++ and cuda extensions
  09 custom c++ and cuda extensions
05 py3
05 py3
- Python for ML & LLM Inference
- How Python Features Enable ML Frameworks
- Complete Outline
- Python for ML & LLM Inference - Quick Start
- 01 fundamentals
  01 fundamentals
- 02 object oriented patterns
  02 object oriented patterns
- 03 functional programming
  03 functional programming
- 04 c extensions and ffi
  04 c extensions and ffi
- 05 memory and performance
  05 memory and performance
- 06 dynamic features
  06 dynamic features
- 07 advanced typing
  07 advanced typing
- 08 module system
  08 module system
- 09 bytecode and execution
  09 bytecode and execution
- 10 ml specific patterns
  10 ml specific patterns
06 pytorch
06 pytorch
- Advanced PyTorch Techniques
- 01 foundations and tensor mastery
  01 foundations and tensor mastery
- 02 module and layer engineering
  02 module and layer engineering
- 03 custom autograd and model transformation
  03 custom autograd and model transformation
- 04 performance and compilation
  04 performance and compilation
- 05 distributed training
  05 distributed training
- 06 mixed precision and quantization
  06 mixed precision and quantization
- 07 export deployment and production
  07 export deployment and production
- 08 advanced training techniques
  08 advanced training techniques
- 09 custom c++ and cuda extensions
  09 custom c++ and cuda extensions
09 jax
09 jax
- JAX
- 01 fundamentals
  01 fundamentals
- 02 core transformations
  02 core transformations
- 03 advanced patterns
  03 advanced patterns
- 04 machine learning
  04 machine learning
- 05 production and performance
  05 production and performance

Table of contents

Topics Covered

Fundamentals¶

Core foundations for LLM understanding and development.

Topics Covered¶

Tokenization
Attention Mechanisms

Copyright © 2024-2026 Jayaprakash

Made with Material for MkDocs