Retrieval Strategies¶

Overview¶

Retrieval Strategies determine how to find relevant documents from a knowledge base. Different approaches have different trade-offs in accuracy, speed, and complexity.

Keyword-based (BM25): Fast, simple, exact matches
Dense retrieval (embeddings): Semantic understanding, slower
Hybrid: Combines both for best results
Advanced: Reranking, fusion, query expansion

Retrieval Approaches¶

1. Keyword-Based (BM25)¶

Algorithm: Best Match 25 (traditional information retrieval)

How it works:
 - Extract keywords from query
 - Find documents containing keywords
 - Rank by keyword frequency and rarity
 - Return top-K documents

Example:
Query: "How does photosynthesis work?"
Keywords: [photosynthesis, work, how]

Search:
 - Document 1: "Photosynthesis is the process..."
 - Contains "photosynthesis" → Good match
 - Document 2: "Plants and photosynthesis..."
 - Contains "photosynthesis" → Good match
 - Document 3: "How to fix solar panels..."
 - Contains "how" but not "photosynthesis" → Lower rank
 - Rank: Document 1 > Document 2 > Document 3

Pros:
Fast (keyword index)
Interpretable (see which keywords matched)
Good for exact topic match
Works without embeddings

Cons:
Misses semantic similarity
Fails on synonyms ("plant growth" vs "photosynthesis")
Limited to exact terms

2. Dense Retrieval (Embedding-based)¶

Algorithm: Semantic similarity via embeddings

How it works:
 - Convert query to embedding
 - Convert all documents to embeddings
 - Find documents most similar to query
 - Return top-K documents

Example:
Query: "How do plants make food?"
Query embedding: [0.12, -0.45, 0.78,...]

Document embeddings:
 - Document 1: "Photosynthesis: plant food production"
 - Embedding: [0.11, -0.46, 0.79,...] → Similarity: 0.98 (great!)
 - Document 2: "Plant growth requires energy"
 - Embedding: [0.09, -0.40, 0.82,...] → Similarity: 0.95 (good)
 - Document 3: "Solar panels convert light"
 - Embedding: [0.20, 0.30, -0.50,...] → Similarity: 0.45 (poor)
 - Rank: Document 1 > Document 2 > Document 3

Pros:
Understands meaning (synonyms work!)
Finds conceptually related documents
Better for complex queries

Cons:
Slower than BM25
Requires embedding computation
Black box (hard to debug why document matched)

3. Hybrid Retrieval¶

Combine keyword + semantic search:

Algorithm:
 - Get BM25 results (keyword match)
 - Get dense results (semantic match)
 - Merge and rank
 - Return top-K from merged results

Example implementation:
```python
def hybrid_search(query, documents, alpha=0.5):
 """
 alpha controls weight:
 - alpha=0: Pure keyword search
 - alpha=0.5: Balanced (recommended)
 - alpha=1: Pure semantic search
 """

 # Get keyword scores
 bm25_scores = bm25_search(query, documents) # 0-1

 # Get semantic scores
 query_embedding = embed(query)
 semantic_scores = []
 for doc in documents:
 doc_embedding = embed(doc)
 similarity = cosine(query_embedding, doc_embedding)
 semantic_scores.append(similarity)

 # Combine scores
 combined_scores = []
 for i in range(len(documents)):
 combined = alpha * semantic_scores[i] + (1-alpha) * bm25_scores[i]
 combined_scores.append(combined)

 # Rank and return top-K
 ranked = sorted(zip(documents, combined_scores), key=lambda x: x[1], reverse=True)
 return [doc for doc, score in ranked[:k]]

# Usage:
results = hybrid_search(
 query="photosynthesis food production",
 documents=knowledge_base,
 alpha=0.5 # 50% keyword, 50% semantic
)

Pros: Best of both worlds Keyword catches exact matches Semantic catches concepts Higher recall and precision

Cons: Slower than pure keyword More complex to tune

---

## Advanced Strategies

### Query Expansion

Problem: Short queries lack context

Query: "photosynthesis"

Documents about photosynthesis: Found
Documents about plant growth: Missed
Documents about energy production: Missed

Solution: Expand query with related terms

Expansion techniques:

Synonym expansion:
photosynthesis → [photosynthesis, plant photosynthesis, food production]
Search for all expanded terms
LLM-based expansion:
Use LLM to generate related questions
"What are related questions to 'photosynthesis'?"
LLM generates: [
"How do plants convert light to energy?",
"What is the role of chlorophyll?",
"How does plant growth relate to photosynthesis?" -]
Search for all variations
Query decomposition:
Break complex query into sub-queries
"Photosynthesis vs cellular respiration" →
["What is photosynthesis?", "What is cellular respiration?", "Difference?"]
Retrieve for each sub-query

Result: More comprehensive retrieval!

Cost-benefit:

Pro: Better recall (find more relevant docs)
Con: More searches (slower, more cost)
Trade-off: 5-10% better quality for 2-3x cost

### Reranking

Problem: Initial retrieval isn't perfect

Retrieval returns top-10, but relevance ranking might be off:

Position 1: Somewhat relevant
Position 2: Very relevant
Position 3: Moderately relevant -...
Position 10: Not relevant

Solution: Rerank retrieved documents

Reranking process:

┌─────────────────────────────────────────┐
Initial retrieval: top-100 documents │
┤
Reranker (cross-encoder) │
"Is this document relevant to query?" │
Score each document 0-1 │
┤
New ranking │
┤
Return top-10 reranked │
┘

Example: Query: "How do plants absorb water?" Initial retrieval:

Osmosis in cells (score: 0.8)
Root structure (score: 0.75)
Photosynthesis (score: 0.65)
Plant metabolism (score: 0.60)

Reranking (cross-encoder):

Root structure (score: 0.92) ← Better match!
Osmosis in cells (score: 0.88)
Water transport (score: 0.85)
Plant metabolism (score: 0.55)

Result: Better ranking after reranking!

Cost vs benefit:

Cost: ~100ms per query (compute intensive)
Benefit: 5-15% accuracy improvement
Typical: Use if queries are expensive (complex topics)

-

## Retrieval Fusion

Combine multiple retrieval methods:

Method 1: Keyword search → results_1 Method 2: Semantic search → results_2 Method 3: Sparse BM25 → results_3

Fusion algorithm:

Get results from each method
results_1: [doc_a, doc_c, doc_e]
results_2: [doc_b, doc_a, doc_d]
results_3: [doc_a, doc_b, doc_c]
Score based on ranking
Reciprocal Rank Fusion (RRF)
For each doc: score = 1/(rank_1 + 1) + 1/(rank_2 + 1) + 1/(rank_3 + 1) │
doc_a: 1/1 + 1/2 + 1/1 = 2.5
doc_b: 0 + 1/1 + 1/2 = 1.5
doc_c: 1/2 + 0 + 1/3 = 0.83
doc_d: 0 + 1/3 + 0 = 0.33
doc_e: 1/3 + 0 + 0 = 0.33
Final ranking: doc_a > doc_b > doc_c > doc_d > doc_e

Benefit:

Combine strengths of multiple methods!
Each method catches different relevant docs

---

## Performance Comparison

Metric Keyword Dense Hybrid Reranked ──────────────────────────────────────────────── Speed 5ms 50ms 55ms 155ms Recall 0.75 0.90 0.95 0.95 Precision 0.70 0.85 0.92 0.95 F1 Score 0.72 0.87 0.93 0.95 Cost $0.01 $0.05 $0.06 $0.16

Recommendation by use case:

Simple queries (facts):

Keyword search (fast, cheap)

Complex queries (reasoning):

Hybrid (good balance)

Production (accuracy critical):

Hybrid + reranking (best quality)

High volume (cost critical):

Keyword (cheap) or cached results

Summary:

Default: Hybrid retrieval
If too slow: Reduce to keyword or dense only
If not accurate: Add reranking

```

Key Takeaways¶

Keyword: Fast, but misses semantic meaning Dense: Slower, but understands concepts Hybrid: Best balance of speed and quality Query expansion: Find more relevant docs Reranking: Improve ranking with cross-encoders

-

[Vector Databases & Embeddings](/01-modeling/03-inference/03-knowledge-integration/rag/(vector-databases-embeddings/) - Dense retrieval foundation
[Chunking & Document Preparation](/01-modeling/03-inference/03-knowledge-integration/rag/(chunking-document-preparation/) - Affects retrieval quality
[Reranking & Ranking](/01-modeling/03-inference/03-knowledge-integration/rag/(reranking-ranking/) - Advanced ranking techniques