GPTQ: Accurate Post-Training Quantization¶
Authors: Frantar et al. Year: 2022 ArXiv/Link: https://arxiv.org/abs/2210.17323
Summary¶
Weight-only quantization method achieving 3-4 bit compression with minimal accuracy loss through optimal quantization order.
Key Concepts¶
- Quantization
- Weight compression
- Post-training
- Minimal overhead
- Inference efficiency
Impact¶
Enabled practical deployment of large models with reduced memory
Category¶
Quantization