Reranking & Ranking: Improving Retrieval Quality¶
Overview¶
Reranking reorders retrieved documents using more sophisticated models. Ranking assigns scores to determine order. Reranking can improve retrieval accuracy by 5-20% but adds latency.
- Retriever: Fast but approximate (e.g., embedding search)
- Reranker: Slow but accurate (e.g., cross-encoder)
- Two-stage: Retrieve many candidates, rerank top-K
- Trade-off: Accuracy vs latency
Two-Stage Retrieval Architecture¶
Why Reranking¶
Problem: Initial retrieval imperfect
Retriever (embedding-based):
- Fast: 10ms per query
- Approximate: ~80-90% recall
- Returns: top-100 candidates
- Some relevant docs might be ranked low!
Example:
Query: "How to fix a broken door hinge?"
Top retrieved by embeddings:
1. "Door hinge replacement" (score: 0.92)
2. "Hinge lubrication tips" (score: 0.88)
3. "Door frame repair" (score: 0.85)
4. "Door lock installation" (score: 0.82)
5. "Hinge joint anatomy" (score: 0.80)
Problem:
- Documents 1-2: Good matches
- Documents 3-5: Related but not perfect
- Relevant doc about "fixing broken hinges" ranked 8th!
Solution: Rerank top-100
Reranker (cross-encoder):
- Slow: 100ms for 100 documents
- Accurate: Near-perfect relevance scoring
- Reranks top-100 from retriever
- Result: Right documents at top!
After reranking:
1. "Fixing broken hinges" (score: 0.98)
2. "Door hinge replacement" (score: 0.96)
3. "Hinge lubrication tips" (score: 0.90)
4. "Hinge joint anatomy" (score: 0.88)
5. "Door frame repair" (score: 0.82)
Benefit:
- Most relevant document now #1 (was #8)!
Ranking Models¶
1. Embedding-based Ranking (Bi-Encoder)¶
Model type: Bi-encoder (SBERT, text-embedding-3)
How it works:
query_embedding = embed(query)
for each doc:
doc_embedding = embed(doc)
score = cosine(query_embedding, doc_embedding)
Pros:
✅ Fast (embeddings pre-computed)
✅ Scalable (efficient search)
✅ Works for retrieval stage
Cons:
❌ Less accurate than cross-encoders
❌ Symmetric similarity (misses nuance)
❌ Can't use query-document interaction
Speed: ~50ms for 1M documents
2. Cross-Encoder Ranking¶
Model type: Cross-encoder (DistilBERT, RoBERTa)
How it works:
for each query-doc pair:
score = model([query, doc])
- Model reads both together!
- Can capture interactions
Example model input:
"[CLS] How to fix a broken door hinge? [SEP]
Door hinges can be repaired by..."
→ Model outputs: 0.95 (very relevant)
Pros:
✅ More accurate (reads query + doc together)
✅ Captures query-document interaction
✅ Best for ranking quality
Cons:
❌ Slower (N queries = N forward passes)
❌ Can't pre-compute (need query at ranking time)
❌ Limited to re-ranking (not first retrieval)
Speed: ~2-5ms per pair, ~100ms for 20 pairs
3. Learning-to-Rank (LTR)¶
Model type: Learned ranker (LambdaMART, neural)
How it works:
- Combine multiple ranking signals
- Learn optimal combination
- Rank documents
Signals:
- BM25 score
- Embedding similarity
- Document length
- Document age
- Click history
- etc.
Example:
score = 0.3 * bm25 + 0.5 * embedding + 0.1 * popularity + 0.1 * freshness
Pros:
✅ Combine multiple signals
✅ Learn optimal weights
✅ Flexible
Cons:
❌ Needs training data
❌ Complex to implement
❌ Slower than simple scoring
Use case: Large-scale search (Google, Bing)
Reranking Strategies¶
Strategy 1: Simple Reranking¶
Algorithm:
1. Initial retrieval: Get top-100 from embeddings
2. Rerank: Score top-100 with cross-encoder
3. Return: Top-K reranked results
Code:
```python
from sentence_transformers import CrossEncoder
from langchain.vectorstores import FAISS
# Step 1: Retrieve
retriever = FAISS.load_local("documents")
docs = retriever.similarity_search(query, k=100)
# Step 2: Rerank
reranker = CrossEncoder("cross-encoder/mmarco-MiniLMv2-L12-H384-v1")
pairs = [[query, doc.page_content] for doc in docs]
scores = reranker.predict(pairs)
# Step 3: Sort
ranked_docs = [doc for _, doc in sorted(zip(scores, docs), reverse=True)]
return ranked_docs[:10]
Latency: - Retrieval: 50ms - Reranking: 100ms - Total: 150ms (vs 50ms retrieval only)
Quality improvement: - Retrieval alone: 82% accuracy - With reranking: 91% accuracy - Gain: +9% (worth the extra latency!)
### Strategy 2: Multi-Stage Reranking
Stage 1: Coarse retrieval - Get top-1000 with embeddings (50ms) - Fast, approximate - Filter for next stage
Stage 2: First reranking - Score top-1000 with lightweight reranker (100ms) - Keeps top-100 - Still cheap
Stage 3: Fine reranking - Score top-100 with heavy-duty reranker (50ms) - Keeps top-20 - Expensive but limited scale
Stage 4: Return - Top-20 results - Total latency: 200ms
Benefit: - Combine speed (stages 1-2) + accuracy (stages 3-4)
### Strategy 3: Query-Specific Reranking
Query type 1: Factual ("What year was X invented?") - Needs exact match - Rerank for factuality - Short reranker OK
Query type 2: Complex ("Compare X and Y") - Needs comprehensive match - Rerank for comprehensiveness - Heavy reranker needed
Implementation:
def adaptive_rerank(query, docs):
"""Choose reranker based on query"""
query_type = classify_query(query) # factual? complex? etc
if query_type == "factual":
reranker = CrossEncoder("lightweight-reranker")
top_k = 5
elif query_type == "complex":
reranker = CrossEncoder("heavy-reranker")
top_k = 20
else:
reranker = CrossEncoder("default-reranker")
top_k = 10
pairs = [[query, doc.page_content] for doc in docs]
scores = reranker.predict(pairs)
ranked = [d for _, d in sorted(zip(scores, docs), reverse=True)]
return ranked[:top_k]
---
## Advanced Ranking Techniques
### Fusion Ranking (Reciprocal Rank Fusion)
Signals: 1. BM25 ranking 2. Embedding ranking 3. Citation ranking 4. Recency ranking
Algorithm: For each document: score = 1/(bm25_rank + 1) + 1/(embedding_rank + 1) + ...
Result: Combined ranking leverages all signals!
Benefit: - Robust (no single signal dominates) - Handles diverse query types - Better overall quality
### Learning-to-Rank with Feature Engineering
Text-level features: - Query length - Document length - Character overlap - Term overlap - BM25 score
Semantic features: - Embedding similarity - Query-doc semantic relatedness - Topic match - Entity overlap
Document features: - Document age - Document popularity - Inlinks count - Quality score
Feed all features to ranker: - Model learns optimal combination - Often beats hand-tuned weights
---
## Performance Comparison
When to use: - Interactive (user waiting): Cross-encoder (150ms OK) - Batch (no rush): Full heavy ranking - High volume: Multi-stage (balance) - Research: LTR (study interactions)
---
## Implementation Considerations
### Latency Constraints
RAG latency breakdown: - Query embedding: 1ms - Retrieval: 50ms - Reranking: 100-200ms - LLM generation: 1-5 seconds - Total: 1-6 seconds (dominated by LLM)
Reranking is <20% of total latency! Worth 5-10% quality improvement. ```
Key Takeaways¶
🎯 Two-stage: Retrieve many (fast), rerank few (accurate)
⚡ Cross-encoder: More accurate but slower than embeddings
📊 Multi-stage: Balance speed and accuracy at scale
🔀 Fusion: Combine multiple signals for robustness
✅ 5-20% quality improvement typical with reranking
Related Notes in RAG Subdirectory¶
- Retrieval Strategies - Initial retrieval
- Vector Databases & Embeddings - Where reranking fits