QLoRA: Efficient Finetuning of Quantized LLMs

Authors: Dettmers, T., Pagnoni, A., Holtzman, A., & Schwettmann, S. Year: 2023 Venue: NeurIPS ArXiv: https://arxiv.org/abs/2305.14314

Summary

QLoRA combines 4-bit quantization with LoRA for extreme parameter efficiency. It enables fine-tuning of 65B parameter models on a single 48GB GPU, making LLM customization accessible without enterprise-grade hardware.

Key Concepts

• 4-bit NormalFloat (NF4) quantization
• Double quantization for weights
• Paged optimizers for memory efficiency
• Combines quantization + LoRA
• 33B model on single GPU feasible
• Minimal performance degradation from full precision
• Unified memory approach

Impact

QLoRA democratized LLM fine-tuning by making 70B+ models accessible on consumer GPUs. It proved quantization doesn't compromise adapter fine-tuning quality, enabling widespread model customization.

Related Papers

• LoRA: Low-Rank Adaptation
• Quantization and Training of Neural Networks
• GPTQ: Accurate Post-Training Quantization

---

Citation:

@inproceedings{dettmers2023qlora,
  title={QLoRA: Efficient Finetuning of Quantized LLMs},
  author={Dettmers, Tim and Pagnoni, Artidoro and Holtzman, Ari and Schwettmann, Samir},
  booktitle={Advances in Neural Information Processing Systems},
  year={2023}
}