Training Techniques & Optimization (2019-2023)

Methods for efficiently training and adapting large language models.

Parameter-Efficient Fine-Tuning

• Adapter: Parameter-Efficient Transfer Learning for NLP - Lightweight adapter modules (0.5-2%)
• Prefix Tuning: Optimizing Continuous Prompts for Generation - Learnable prompt prefixes
• LoRA: Low-Rank Adaptation of Large Language Models - 10,000x parameter reduction
• QLoRA: Efficient Finetuning of Quantized LLMs - 4-bit + LoRA on consumer GPUs

Supervised Fine-Tuning (SFT)

• Supervised Fine-Tuning of Large Language Models - Foundation for instruction-following
• Scaling Instruction-Finetuned Language Models - PaLM instruction tuning at scale
• The Flan Collection: Designing Data and Methods for Effective Instruction Tuning - 473 tasks, 1.8M examples

Alignment & Human Feedback

• Learning to Summarize from Human Feedback - RLHF foundations
• Training Language Models to Follow Instructions with Human Feedback - RLHF at scale (InstructGPT)
• Direct Preference Optimization: Your Language Model is Secretly a Reward Model - Simpler RLHF alternative

Optimization Fundamentals

• Gradient Checkpointing for Efficient Backpropagation - Memory-efficient training
• Mixed Precision Training - FP16 training benefits

---

Key Insight: Parameter-efficient methods enable LLM customization without full retraining, democratizing model adaptation.