Gradient Checkpointing for Efficient Backpropagation

Authors: Chen et al. Year: 2016 ArXiv/Link: https://arxiv.org/abs/1604.06174

Summary

Memory-efficient training technique that trades computation for memory by recomputing activations during backpropagation.

Key Concepts

• Memory efficiency
• Recomputation
• Trade-off strategies
• Backpropagation
• Training optimization

Impact

Enabled training of large models with limited memory

Category

Optimization & Efficiency

---