Reranking & Ranking¶

Overview¶

Reranking reorders retrieved documents using more sophisticated models. Ranking assigns scores to determine order. Reranking can improve retrieval accuracy by 5-20% but adds latency.

Retriever: Fast but approximate (e.g., embedding search)
Reranker: Slow but accurate (e.g., cross-encoder)
Two-stage: Retrieve many candidates, rerank top-K
Trade-off: Accuracy vs latency

Two-Stage Retrieval Architecture¶

Why Reranking¶

Problem: Initial retrieval imperfect

Retriever (embedding-based):
 - Fast: 10ms per query
 - Approximate: ~80-90% recall
 - Returns: top-100 candidates
 - Some relevant docs might be ranked low!

Example:
Query: "How to fix a broken door hinge?"

Top retrieved by embeddings:
1. "Door hinge replacement" (score: 0.92)
2. "Hinge lubrication tips" (score: 0.88)
3. "Door frame repair" (score: 0.85)
4. "Door lock installation" (score: 0.82)
5. "Hinge joint anatomy" (score: 0.80)

Problem:
 - Documents 1-2: Good matches
 - Documents 3-5: Related but not perfect
 - Relevant doc about "fixing broken hinges" ranked 8th!

Solution: Rerank top-100

Reranker (cross-encoder):
 - Slow: 100ms for 100 documents
 - Accurate: Near-perfect relevance scoring
 - Reranks top-100 from retriever
 - Result: Right documents at top!

After reranking:
1. "Fixing broken hinges" (score: 0.98)
2. "Door hinge replacement" (score: 0.96)
3. "Hinge lubrication tips" (score: 0.90)
4. "Hinge joint anatomy" (score: 0.88)
5. "Door frame repair" (score: 0.82)

Benefit:
 - Most relevant document now #1 (was #8)!

Ranking Models¶

1. Embedding-based Ranking (Bi-Encoder)¶

Model type: Bi-encoder (SBERT, text-embedding-3)

How it works:
query_embedding = embed(query)
for each doc:
 doc_embedding = embed(doc)
 score = cosine(query_embedding, doc_embedding)

Pros:
Fast (embeddings pre-computed)
Scalable (efficient search)
Works for retrieval stage

Cons:
Less accurate than cross-encoders
Symmetric similarity (misses nuance)
Can't use query-document interaction

Speed: ~50ms for 1M documents

2. Cross-Encoder Ranking¶

Model type: Cross-encoder (DistilBERT, RoBERTa)

How it works:
for each query-doc pair:
 score = model([query, doc])
 - Model reads both together!
 - Can capture interactions

Example model input:
"[CLS] How to fix a broken door hinge? [SEP] 
Door hinges can be repaired by..."
→ Model outputs: 0.95 (very relevant)

Pros:
More accurate (reads query + doc together)
Captures query-document interaction
Best for ranking quality

Cons:
Slower (N queries = N forward passes)
Can't pre-compute (need query at ranking time)
Limited to re-ranking (not first retrieval)

Speed: ~2-5ms per pair, ~100ms for 20 pairs

3. Learning-to-Rank (LTR)¶

Model type: Learned ranker (LambdaMART, neural)

How it works:
 - Combine multiple ranking signals
 - Learn optimal combination
 - Rank documents

Signals:
 - BM25 score
 - Embedding similarity
 - Document length
 - Document age
 - Click history
 - etc.

Example:
score = 0.3 * bm25 + 0.5 * embedding + 0.1 * popularity + 0.1 * freshness

Pros:
Combine multiple signals
Learn optimal weights
Flexible

Cons:
Needs training data
Complex to implement
Slower than simple scoring

Use case: Large-scale search (Google, Bing)

-

Reranking Strategies¶

Strategy 1: Simple Reranking¶

Algorithm:
1. Initial retrieval: Get top-100 from embeddings
2. Rerank: Score top-100 with cross-encoder
3. Return: Top-K reranked results

Code:
```python
from sentence_transformers import CrossEncoder
from langchain.vectorstores import FAISS

# Step 1
retriever = FAISS.load_local("documents")
docs = retriever.similarity_search(query, k=100)

# Step 2
reranker = CrossEncoder("cross-encoder/mmarco-MiniLMv2-L12-H384-v1")
pairs = [[query, doc.page_content] for doc in docs]
scores = reranker.predict(pairs)

# Step 3
ranked_docs = [doc for _, doc in sorted(zip(scores, docs), reverse=True)]

return ranked_docs[:10]

Latency:

Retrieval: 50ms
Reranking: 100ms
Total: 150ms (vs 50ms retrieval only)

Quality improvement:

Retrieval alone: 82% accuracy
With reranking: 91% accuracy
Gain: +9% (worth the extra latency!)

### Strategy 2: Multi-Stage Reranking

For very large retrieval sets (>1000 documents):

Stage 1: Coarse retrieval

Get top-1000 with embeddings (50ms)
Fast, approximate
Filter for next stage

Stage 2: First reranking

Score top-1000 with lightweight reranker (100ms)
Keeps top-100
Still cheap

Stage 3: Fine reranking

Score top-100 with heavy-duty reranker (50ms)
Keeps top-20
Expensive but limited scale

Stage 4: Return

Top-20 results
Total latency: 200ms

Benefit:

Combine speed (stages 1-2) + accuracy (stages 3-4)

### Strategy 3: Query-Specific Reranking

Different queries need different reranking

Query type 1: Factual ("What year was X invented?")

Needs exact match
Rerank for factuality
Short reranker OK

Query type 2: Complex ("Compare X and Y")

Needs comprehensive match
Rerank for comprehensiveness
Heavy reranker needed

Implementation:

def adaptive_rerank(query, docs):
 """Choose reranker based on query"""

 query_type = classify_query(query) # factual? complex? etc

 if query_type == "factual":
 reranker = CrossEncoder("lightweight-reranker")
 top_k = 5
 elif query_type == "complex":
 reranker = CrossEncoder("heavy-reranker")
 top_k = 20
 else:
 reranker = CrossEncoder("default-reranker")
 top_k = 10

 pairs = [[query, doc.page_content] for doc in docs]
 scores = reranker.predict(pairs)

 ranked = [d for _, d in sorted(zip(scores, docs), reverse=True)]
 return ranked[:top_k]

-

## Advanced Ranking Techniques

### Fusion Ranking (Reciprocal Rank Fusion)

Combine multiple ranking signals (like retrieval fusion):

Signals:

BM25 ranking
Embedding ranking
Citation ranking
Recency ranking

Algorithm: For each document: score = 1/(bm25_rank + 1) + 1/(embedding_rank + 1) +...

Result: Combined ranking leverages all signals!

Benefit:

Robust (no single signal dominates)
Handles diverse query types
Better overall quality

### Learning-to-Rank with Feature Engineering

Compute rich features per query-doc pair:

Text-level features:

Query length
Document length
Character overlap
Term overlap
BM25 score

Semantic features:

Embedding similarity
Query-doc semantic relatedness
Topic match
Entity overlap

Document features:

Document age
Document popularity
Inlinks count
Quality score

Feed all features to ranker:

Model learns optimal combination
Often beats hand-tuned weights

---

## Performance Comparison

Reranking method: Accuracy Latency Use Case ───────────────────────────────────────────────── No reranking 0.80 50ms Speed critical Cross-encoder 0.92 150ms Accuracy critical Multi-stage 0.90 200ms Large scale LTR 0.94 100ms High volume Fusion + LTR 0.96 200ms Maximum quality

When to use:

Interactive (user waiting): Cross-encoder (150ms OK)
Batch (no rush): Full heavy ranking
High volume: Multi-stage (balance)
Research: LTR (study interactions)

---

## Implementation Considerations

### Latency Constraints

User tolerance:

Interactive search: <300ms total
Batch processing: <5 seconds OK
Background: <1 minute OK

RAG latency breakdown:

Query embedding: 1ms
Retrieval: 50ms
Reranking: 100-200ms
LLM generation: 1-5 seconds
Total: 1-6 seconds (dominated by LLM)

Reranking is <20% of total latency! Worth 5-10% quality improvement.

```

Key Takeaways¶

Two-stage: Retrieve many (fast), rerank few (accurate) Cross-encoder: More accurate but slower than embeddings Multi-stage: Balance speed and accuracy at scale Fusion: Combine multiple signals for robustness 5-20% quality improvement typical with reranking

-

Retrieval Strategies - Initial retrieval
[Vector Databases & Embeddings](/01-modeling/03-inference/03-knowledge-integration/rag/(vector-databases-embeddings/) - Where reranking fits