AWQ Quantization: Complete Technical Guide

Overview

AWQ (Activation-aware Quantization) is a post-training quantization technique that quantizes model weights based on activation distributions. It outperforms GPTQ by exploiting the observation that quantization error matters differently depending on activation magnitudes.

• Paper: "AWQ: Activation-aware Weight Quantization for LLM Compression and Acceleration"
• Authors: Lin et al., MIT-IBM Watson AI Lab (2023)
• Key Innovation: Weight quantization guided by activation statistics
• Impact: Higher accuracy than GPTQ for same compression ratio
• Advantages: Simpler algorithm, faster quantization, better accuracy

---

The Key Insight: Activation-Aware Quantization

GPTQ's Limitation

GPTQ Strategy:
  - Uses Hessian (based on data statistics)
  - Quantizes based on parameter importance
  - Assumes equal importance across all activations
  - Result: Good, but leaves room for improvement

GPTQ weakness:
When activation values vary greatly across samples:
  - Some activations are consistently large
  - Some activations are consistently small
  - GPTQ treats all equally
  - Misses opportunity to optimize!

Example:
Layer computation: output = W × x (weight × activation)

If x is always large:
  - Quantization error in W gets amplified
  - Need more precision in W

If x is always small:
  - Quantization error in W gets dampened
  - Can use less precision in W

GPTQ doesn't exploit this insight!

AWQ's Insight

AWQ Strategy:
  - Observe actual activation distributions
  - Identify which weights have large activations
  - Quantize "small activation" weights more aggressively
  - Quantize "large activation" weights conservatively
  - Result: Better accuracy at same compression!

Why this works:
Output = W × x (multiplication)
Error = quantization_error_in_W × x

If x is large:
  - Small quantization error → big output error (bad!)
  - Need to preserve W precision

If x is small:
  - Even big quantization error → small output error (ok!)
  - Can aggressively quantize W

AWQ exploits this!

Visual Comparison

Weight value distribution across a layer:

GPTQ (treats all weights equally):
- ┌──────────────────────────────────────────────┐
    - Hessian-based importance                     │
  - ┤
    - ███████ ███ ██████ █ ███ ██ ███ ████ ██     │
    - 0.8  0.3 0.9  0.2  0.7 0.4 0.8  0.6  0.5    │
    - Quantize based on Hessian only               │
  - ┘

AWQ (considers activation magnitude):
- ┌──────────────────────────────────────────────┐
    - Activation magnitude                         │
  - ┤
    - XXXXX  X  XXXXX    X   X       XXXX          │
    - Large  Sm Large  Sm Small    Medium Large    │
  - ┘

Combined decision:
  - Large weight + Large activation = Protect (high precision)
  - Small weight + Large activation = Protect (moderate precision)
  - Large weight + Small activation = Can quantize (ok if error small)
  - Small weight + Small activation = Aggressive (error dampened)

Result: Better allocation of quantization budget!

---

The AWQ Algorithm

Three Phases

Phase 1: Profiling (Collect Statistics)
  - Run inference on calibration data
  - Track activation values per layer
  - Compute activation statistics (mean, max, std)
  - Identify "important" weights (those with large activations)

Phase 2: Quantization (Exploit Statistics)
  - For each layer:
    - Identify weights that receive large activations
    - Protect these weights (higher precision)
    - Quantize others more aggressively
  - Perform per-channel quantization

Phase 3: Finalization (Save Model)
  - Save quantized weights
  - Save activation scaling factors
  - Verify accuracy

Detailed Algorithm: Weight-Activation Correlation

AWQ Algorithm Pseudocode:

# Phase 1: Collect Activation Statistics
activation_stats = {}
for layer in model.layers:
    activation_stats[layer] = {
        'max_per_channel': collect_activation_max(layer),
        'mean_per_channel': collect_activation_mean(layer),
        'std_per_channel': collect_activation_std(layer),
    }

# Phase 2: Quantization-aware scaling
for layer in model.layers:
    W = layer.weight  # Shape: (out_features, in_features)
    x = layer.activation  # Shape: (batch, seq_len, in_features)

    # Step 1: Compute importance (activation-aware)
    # Which weights have larger activations?
    activation_max = activation_stats[layer]['max_per_channel']

    # For each input feature dimension:
    importance = []
    for i in range(in_features):
        # If activations for feature i are large,
        # then weights in column W[:, i] are important
        importance.append(activation_max[i])

    # Step 2: Scale weights based on importance
    # High importance → keep high precision
    # Low importance → quantize aggressively
    scale_factor = 1.0 / torch.sqrt(importance + eps)

    # W_scaled: easier to quantize (but tracks original values)
    W_scaled = W * scale_factor

    # Step 3: Quantize scaled weights
    W_quantized = quantize_int4(W_scaled)

    # Step 4: Dequantize with scale factors
    # During inference: W_effective = W_quantized / scale_factor
    # This recovers approximate original values, but optimally

    # Step 5: Save both
    layer.weight_quantized = W_quantized
    layer.scale_factor = scale_factor

# Phase 3: Inference
# During forward pass:
for layer in model.layers:
    x = layer.input
    W_quantized = layer.weight_quantized
    scale_factor = layer.scale_factor

    # Recover (approximately) original weights
    W = W_quantized / scale_factor

    # Standard linear layer computation
    output = x @ W.T

Key Differences from GPTQ

GPTQ:
  - Uses Hessian (second-order statistics)
  - Quantizes one weight at a time
  - Compensates other weights
  - Complex algorithm, ~O(n²) complexity
  - Quantization time: ~30-90 min for 70B

AWQ:
  - Uses activation statistics (first-order)
  - Applies scaling to all weights at once
  - No compensation needed
  - Simpler algorithm, ~O(n) complexity
  - Quantization time: ~5-10 min for 70B

---

Mathematical Foundation

Activation Distribution Analysis

Neural Network Layer:
output = activation(W × input + bias)

In a transformer:
input shape: (batch_size, seq_len, hidden_dim)
weight shape: (hidden_dim, hidden_dim)

Key observation:
Different input channels have different magnitude distributions!

Example: First layer of attention
  - Embedding dimensions 0-31: Consistently large values
  - Embedding dimensions 32-63: Consistently small values
  - Embedding dimensions 64-95: Medium values
  - ... and so on

This is NOT random! It's part of model structure.

Impact on quantization:
For dimension i with large activations:
  - W[:, i] receives large values during inference
  - Quantization error gets magnified
  - Need to preserve precision

For dimension j with small activations:
  - W[:, j] receives small values during inference
  - Quantization error gets dampened
  - Can quantize aggressively

Scaling Formulation

Linear layer: y = W × x

If we scale x and inverse-scale W:
y = (W / α) × (α × x)
y' = W' × x'

Mathematical equivalence: y = y' (same output!)

But numerically different:
  - W': Easier to quantize (all values scaled similarly)
  - x': Larger values (but only used in calibration, not stored)

AWQ uses this trick!

For each input channel i:
  - If x[i] is always large
  - Then W[:, i] can have smaller quantized values
  - Because they multiply with large x[i]

Formally:
W'[:, i] = W[:, i] / scale[i]
x'[i] = x[i] × scale[i]
y = W' × x' = W × x (mathematically the same)

scale[i] chosen based on activation statistics:
scale[i] = sqrt(max_activation[i])

Result: W' is more quantization-friendly!

---

AWQ in Practice: Complete Workflow

Step 1: Install and Setup

# Install AutoAWQ
pip install autoawq

# Or from source for latest
git clone https://github.com/casper-hansen/AutoAWQ
cd AutoAWQ
pip install -e .

Step 2: Quantize Model

from awq import AutoAWQForCausalLM
from transformers import AutoTokenizer

# Model configuration
model_path = "meta-llama/Llama-2-7b-hf"
quant_config = {
    "zero_point": True,
    "q_group_size": 128,
    "w_bit": 4,
    "version": "GEMM"  # or "GEMV"
}

# Load model
model = AutoAWQForCausalLM.from_pretrained(
    model_path,
    **quant_config
)

tokenizer = AutoTokenizer.from_pretrained(model_path)

# Prepare calibration data
calibration_data = [
    "The quick brown fox jumps over the lazy dog.",
    "Machine learning is a subset of artificial intelligence.",
    # ... more samples (typically 128-512 samples)
]

# Tokenize calibration data
calibration_tokens = [
    tokenizer(text, return_tensors="pt")
    for text in calibration_data
]

# Quantize (FAST! ~5 min for 7B on A100)
model.quantize(calibration_tokens)

# Save
model.save_quantized("./llama-2-7b-awq")
print("Quantization complete!")

Step 3: Use Quantized Model

from awq import AutoAWQForCausalLM
from transformers import AutoTokenizer

# Load quantized model
model = AutoAWQForCausalLM.from_quantized(
    "./llama-2-7b-awq",
    fuse_layers=True,  # Fuse operations for faster inference
)

tokenizer = AutoTokenizer.from_pretrained("meta-llama/Llama-2-7b-hf")

# Generate
prompt = "Explain quantum computing"
inputs = tokenizer(prompt, return_tensors="pt").to(model.device)

outputs = model.generate(
    **inputs,
    max_new_tokens=256,
    do_sample=True,
    top_p=0.95,
    temperature=0.7
)

print(tokenizer.decode(outputs[0]))

Step 4: Integration with vLLM

from vllm import LLM, SamplingParams

# vLLM auto-detects AWQ
llm = LLM(
    model="./llama-2-7b-awq",
    quantization="awq",  # Explicitly specify
    gpu_memory_utilization=0.95
)

prompts = [
    "What is machine learning?",
    "Explain deep learning",
    "How do transformers work?"
]

sampling_params = SamplingParams(
    temperature=0.7,
    top_p=0.95,
    max_tokens=256
)

outputs = llm.generate(prompts, sampling_params)

for output in outputs:
    print(output.outputs[0].text)

---

Configuration Details

Quantization Bits (w_bit)

w_bit=4 (INT4):
  - Range: -8 to 7
  - Compression: 8x (float32) or 4x (float16)
  - Size for 7B: 3.5GB
  - Accuracy: 97-99%
  - Most common choice

w_bit=3 (INT3, experimental):
  - Range: -4 to 3
  - Compression: 10.6x
  - Size for 7B: 2.6GB
  - Accuracy: 90-95%
  - Requires careful calibration
  - Not widely adopted yet

w_bit=8 (INT8):
  - Range: -128 to 127
  - Compression: 4x
  - Size for 7B: 7GB
  - Accuracy: >99%
  - Usually not needed (float16 is 7GB too)

Group Size (q_group_size)

Group size: How many weights share a quantization scale

q_group_size=128:
  - Divide 4096 input dims into 32 groups
  - Each group: 128 weights
  - 32 scale factors per output channel
  - Memory overhead: minimal
  - Accuracy: Best (more fine-grained)
  - Inference: Standard overhead

q_group_size=1024:
  - Divide 4096 input dims into 4 groups
  - Each group: 1024 weights
  - 4 scale factors per output channel
  - Memory overhead: Minimal
  - Accuracy: Slightly worse
  - Inference: Faster (fewer group calculations)

q_group_size=-1 (channel-wise):
  - One scale per output channel
  - Coarser quantization
  - Accuracy: Worse
  - Inference: Very fast
  - Rarely used in practice

Recommendation: 128 (best accuracy)

Zero Point

zero_point=True:
  - Use both scale and zero point
  - INT4 range: -8 to 7 with offset
  - More precise quantization
  - Slightly more memory (store zero points)
  - Better accuracy

zero_point=False:
  - Only use scale
  - INT4 range: -8 to 7
  - Simpler, slightly faster
  - Slightly worse accuracy

Recommendation: True (better accuracy)

Version (GEMM vs GEMV)

GEMM (General Matrix-Matrix Multiply):
  - For batch inference
  - Multiple requests at once
  - Optimized for large batch size
  - Used in inference servers (vLLM)
  - Better throughput

GEMV (General Matrix-Vector Multiply):
  - For single-request inference
  - One request at a time
  - Optimized for single sample
  - Used in real-time apps
  - Lower latency

Recommendation: GEMM for server, GEMV for latency-critical

---

Performance Comparison: AWQ vs GPTQ

Benchmark Results

Model: Llama 2 7B
Hardware: A100 GPU
Calibration: 128 samples

Configuration:
  - GPTQ: INT4, group_size=128, true_sequential=True
  - AWQ: INT4, group_size=128, w_bit=4
  - Both: 3.5GB model size

Quantization Time:
  - GPTQ: 25-30 minutes
  - AWQ: 5-8 minutes ✓ (3-5x faster!)

Accuracy (ARC-Challenge):
  - Original: 53.8%
  - GPTQ INT4: 52.2% (-1.6%)
  - AWQ INT4: 52.8% (-1.0%) ✓ Better!

Accuracy (MMLU):
  - Original: 45.9%
  - GPTQ INT4: 44.7% (-2.6%)
  - AWQ INT4: 45.3% (-1.2%) ✓ Better!

Perplexity (WikiText2):
  - Original: 6.02
  - GPTQ INT4: 6.28 (+4.3%)
  - AWQ INT4: 6.15 (+2.2%) ✓ Better!

Inference Speed (tokens/sec):
  - FP16: 200
  - GPTQ INT4: 480 (2.4x)
  - AWQ INT4: 500 (2.5x)

Accuracy Loss Comparison

Compression Ratio vs Accuracy Loss:

Loss (%)
  - 5 │
  - × GPTQ INT4
  - 4 │  ╱
  - ╱   ○ AWQ INT4
  - 3 │×
  - ╱
  - 2 │ ○
  - ╲
  - 1 │  ○─────
    │
  - 0 └─────────────────
    3x    4x    5x    8x
    Compression Ratio

Conclusion: AWQ consistently better than GPTQ!
(Lower accuracy loss for same compression)

---

Real-World Examples

Example 1: Quick Quantization of Llama 2 7B

# Fastest way to quantize

from awq import AutoAWQForCausalLM
from transformers import AutoTokenizer, TextIteratorStreamer
from datasets import load_dataset
import torch

model_path = "meta-llama/Llama-2-7b-hf"

# Quick calibration data (just 32 samples, very fast)
dataset = load_dataset('wikitext', 'wikitext-2-raw-v1', split='train')
calibration_data = [dataset['text'][i] for i in range(32)]

# Quantize config
quant_config = {
    "zero_point": True,
    "q_group_size": 128,
    "w_bit": 4,
    "version": "GEMM",
}

# Load and quantize
print("Loading model...")
model = AutoAWQForCausalLM.from_pretrained(model_path, **quant_config)

print("Preparing calibration data...")
tokenizer = AutoTokenizer.from_pretrained(model_path)
calibration_tokens = [
    tokenizer(text, return_tensors="pt", max_length=512, truncation=True)
    for text in calibration_data
]

print("Quantizing... (5-10 minutes)")
import time
start = time.time()
model.quantize(calibration_tokens)
print(f"Quantization took {(time.time() - start) / 60:.1f} minutes")

# Save
model.save_quantized("./llama-2-7b-awq")

# Test
tokenizer.pad_token_id = tokenizer.eos_token_id
inputs = tokenizer("Hello world", return_tensors="pt")
outputs = model.generate(**inputs, max_new_tokens=50)
print(tokenizer.decode(outputs[0]))

Example 2: Quantizing 70B Model

# Quantizing large 70B model

from awq import AutoAWQForCausalLM
from transformers import AutoTokenizer
import torch

model_path = "meta-llama/Llama-2-70b-hf"

# For 70B, use more efficient settings
quant_config = {
    "zero_point": True,
    "q_group_size": 128,
    "w_bit": 4,
    "version": "GEMM",
}

# Load with device_map for multi-GPU support
print("Loading 70B model (using device_map)...")
model = AutoAWQForCausalLM.from_pretrained(
    model_path,
    device_map="auto",  # Automatically distribute across GPUs
    **quant_config
)

# Use same calibration approach
from datasets import load_dataset
dataset = load_dataset('wikitext', 'wikitext-2-raw-v1', split='train')
calibration_data = dataset['text'][:64]  # 64 samples for 70B

print("Preparing calibration...")
tokenizer = AutoTokenizer.from_pretrained(model_path)
calibration_tokens = [
    tokenizer(text, return_tensors="pt", max_length=512, truncation=True)
    for text in calibration_data
]

print("Quantizing 70B (8-15 minutes with device_map)...")
import time
start = time.time()
model.quantize(calibration_tokens)
quantization_time = (time.time() - start) / 60
print(f"✓ Quantization took {quantization_time:.1f} minutes")

# Save (saves across devices if needed)
model.save_quantized("./llama-2-70b-awq")

# Result: 35GB model from 280GB!
print("✓ Model saved: 35GB (from 280GB original)")

Example 3: Deployment with vLLM

# Production deployment with vLLM

from vllm import LLM, SamplingParams
import time

# Load quantized model
print("Loading AWQ model with vLLM...")
llm = LLM(
    model="./llama-2-7b-awq",
    quantization="awq",
    tensor_parallel_size=1,
    gpu_memory_utilization=0.95,
    dtype="half",
)

# Benchmark
prompts = [
    "Explain quantum computing in simple terms",
    "Write Python code to calculate Fibonacci numbers",
    "What are the benefits of machine learning?",
] * 10  # 30 requests total

sampling_params = SamplingParams(
    temperature=0.7,
    top_p=0.95,
    max_tokens=256
)

print("Generating 30 requests...")
start = time.time()
outputs = llm.generate(prompts, sampling_params)
total_time = time.time() - start

print(f"✓ Generated 30 requests in {total_time:.1f}s")
print(f"✓ Throughput: {30 / total_time:.1f} requests/sec")
print(f"✓ Average latency: {total_time / 30 * 1000:.1f}ms")

for i, output in enumerate(outputs[:3]):  # Show first 3
    print(f"\nRequest {i+1}:")
    print(output.outputs[0].text[:100] + "...")

---

Challenges and Solutions

Challenge 1: Activation Statistics Collection

Problem:
Need to collect statistics for all layers
But activations vary with input

Solution:
Use representative calibration data:
  - 128-512 samples from task domain
  - Covers diverse activation patterns
  - Takes ~2-5 minutes to profile
  - Only done once during quantization

Trade-off:
  - More calibration data → more accurate
  - But overhead only during quantization
  - Final model unaffected

Challenge 2: Channel Misalignment

Problem:
Different channels have very different activation ranges:
  - Channel 0: activations in range [100, 200]
  - Channel 1: activations in range [0.1, 0.2]
  - Channel 2: activations in range [10, 20]

Naive quantization: All channels same precision → waste

Solution: Per-channel scaling
  - Compute scale per channel
  - Adjust weights accordingly
  - Each channel gets optimal precision allocation
  - Built into AWQ!

Challenge 3: Mixed Precision Issues

Problem:
If you mix AWQ INT4 with other layers:
  - Some layers quantized to INT4
  - Some layers in float16
  - Potential numerical instability

Solution 1: Quantize consistently
  - Quantize all weight matrices
  - Keep activations in float16
  - This is what AWQ does!

Solution 2: Mixed precision (advanced)
  - Sensitive layers: float16
  - Robust layers: INT4
  - Requires careful tuning
  - Not typically needed

---

AWQ vs GPTQ: Detailed Comparison

Aspect                  AWQ             GPTQ
─────────────────────────────────────────────────
Algorithm               Activation-aware Hessian-based
Complexity              O(n)            O(n²)
Quantization time       5-10 min (7B)   25-30 min (7B)
Accuracy (INT4)         97-99%          96-98%
Accuracy advantage      ✓ Better        
Speed to quantize       ✓ Faster        
Hardware friendly       ✓ Simpler GPU   
Post-training QAT       ✓ Post-train    Post-train
Model size (INT4)       3.5GB (7B)      3.5GB (7B)
Inference speed         2.5x            2.4x
Tool maturity           Good            Excellent
Adoption                Growing         Widespread

Verdict:
AWQ is catching up and often preferred for:
- Faster quantization workflow
- Slightly better accuracy
- Simpler algorithm

GPTQ still preferred for:
- Massive model zoo (pre-quantized)
- Proven stability in production
- Wider tool support

---

Quantized Models: Hugging Face Hub

Available AWQ Models

Pre-quantized AWQ models on HF Hub:

Llama Models:
  - TheBloke/Llama-2-7B-AWQ
  - TheBloke/Llama-2-13B-AWQ
  - TheBloke/Llama-2-70B-AWQ
  - TheBloke/Llama-2-Chat-7B-AWQ
  - ... and many more

Mistral Models:
  - TheBloke/Mistral-7B-v0.1-AWQ
  - TheBloke/Mistral-7B-Instruct-v0.1-AWQ
  - ... variants

Qwen Models:
  - TheBloke/Qwen-7B-AWQ
  - TheBloke/Qwen-14B-AWQ
  - ...

Falcon Models:
  - TheBloke/Falcon-7b-AWQ
  - TheBloke/Falcon-40b-AWQ
  - ...

Search: https://huggingface.co/search/full-text?q=AWQ

Usage (instant):
```python
from awq import AutoAWQForCausalLM

model = AutoAWQForCausalLM.from_quantized(
    "TheBloke/Llama-2-7B-Chat-AWQ"
)

---

## Integration with Tools

### AutoAWQ Library

```python
# Main features of AutoAWQ

from awq import AutoAWQForCausalLM

# 1. Quantization
model = AutoAWQForCausalLM.from_pretrained(model_name)
model.quantize(calibration_data)
model.save_quantized(output_dir)

# 2. Inference
model = AutoAWQForCausalLM.from_quantized(quantized_path)
outputs = model.generate(prompt)

# 3. Features
  - # ├─ Fused layers (faster)
  - # ├─ Multi-GPU support
  - # ├─ Integration with vLLM
  - # ├─ Model zoo support
  - # └─ Both INT4 and INT3

vLLM Integration

# vLLM auto-detects AWQ quantization

pip install vllm

# Load AWQ model
python -m vllm.entrypoints.openai.api_server \
    --model TheBloke/Llama-2-7B-Chat-AWQ \
    --quantization awq \
    --dtype half

Ollama (Coming Soon)

# Ollama is adding AWQ support

# Expected usage:
ollama pull llama2-awq:7b
ollama run llama2-awq:7b "Explain quantum computing"

---

Best Practices

✅ Do's

1. Use AWQ for new projects (simpler, faster, better accuracy)
2. Collect representative calibration data (different domains need different data)
3. Verify accuracy on your specific task (different tasks may have different trade-offs)
4. Use group_size=128 for best accuracy
5. Enable zero_point=True for better precision
6. Profile before deployment (benchmark on target hardware)
7. Use fused layers in vLLM for faster inference
8. Compare with GPTQ models if available on HF Hub

❌ Don'ts

1. ❌ Use generic calibration data (use domain-specific)
2. ❌ Assume all models quantize equally well
3. ❌ Skip accuracy verification before deployment
4. ❌ Use very small calibration sets (< 32 samples) without reason
5. ❌ Mix AWQ and GPTQ in same model
6. ❌ Forget to test on target hardware
7. ❌ Use without proper error handling in production
8. ❌ Assume INT4 AWQ is as good as original (there's always loss)

---

Performance Estimation

Will AWQ Help Your Use Case?

Decision tree:

Is model > 7B?
  - Yes: AWQ definitely helps
  - No: Maybe (depends on constraints)

Do you need accuracy preservation?
  - Yes: AWQ is better than GPTQ (higher accuracy)
  - No: GPTQ fine too

Do you want fast quantization?
  - Yes: AWQ is 3-5x faster
  - No: Either is fine

Expected benefits:
  - 8x compression (float32 → int4)
  - 4x compression (float16 → int4)
  - 2-3x inference speedup
  - 1-2% accuracy loss (INT4)
  - 0.5-1% accuracy loss (INT8)
  - Fits on 1 smaller GPU

Not good for:
  - Models < 3B (not worth it)
  - Already quantized models (re-quant expensive)
  - Models requiring int2 or int1 (not practical)
  - Training scenarios (use LoRA instead)

---

Recent Developments

AWQ vs Newer Methods

Timeline of quantization methods:

2023 (Early):
  - GPTQ established (widely adopted)
  - AWQ released (improvement)

2023 (Mid):
  - TeQ, SqueezeLLM (alternatives)
  - QuaRot (rotation-based)

2024:
  - Advanced AWQ (better calibration)
  - Speculative Decoding with Quant
  - On-device dynamic quantization

Current state (2024):
  - GPTQ: Most models available, proven
  - AWQ: Better accuracy, growing adoption
  - Others: Emerging, less adopted
  - Recommendation: GPTQ or AWQ (both great!)

---

Key Takeaways

🔑 AWQ uses activation statistics to optimize quantization
⚡ 3-5x faster quantization than GPTQ
📊 1-2% better accuracy at same compression (typically)
💾 Same model size as GPTQ (3.5GB for 7B INT4)
⏱️ 5-10 minutes to quantize 7B (vs 25-30 for GPTQ)
🎯 Perfect for production LLM serving

---

Comparison Summary

Aspect	GPTQ	AWQ	Winner
Accuracy	96-98%	97-99%	AWQ
Speed	25-30 min	5-10 min	AWQ
Complexity	Complex	Simple	AWQ
Adoption	Very high	Growing	GPTQ
Model zoo	Massive	Growing	GPTQ
New projects	OK	Better	AWQ

Recommendation: - New project? → AWQ (faster, better accuracy) - Production with many models? → GPTQ (ecosystem) - Can't decide? → Use GPTQ (safe), then try AWQ for next iteration

---

Further Reading

• AWQ Paper: "AWQ: Activation-aware Weight Quantization for LLM Compression and Acceleration"
• AutoAWQ GitHub: github.com/casper-hansen/AutoAWQ
• Comparison with GPTQ: Recent benchmark papers
• Quantization Survey: Comprehensive LLM quantization overview

---

Related Notes

• Quantization Gptq - Previous post-training method
• Kv Cache - Complement with KV quantization
• 01 Vllm - Uses AWQ for efficient inference
• Model Optimization - Other optimization techniques