Skip to content

Retrieval Strategies: Finding the Right Documents

Overview

Retrieval Strategies determine how to find relevant documents from a knowledge base. Different approaches have different trade-offs in accuracy, speed, and complexity.

  • Keyword-based (BM25): Fast, simple, exact matches
  • Dense retrieval (embeddings): Semantic understanding, slower
  • Hybrid: Combines both for best results
  • Advanced: Reranking, fusion, query expansion

Retrieval Approaches

1. Keyword-Based (BM25)

Algorithm: Best Match 25 (traditional information retrieval)

How it works:
  - Extract keywords from query
  - Find documents containing keywords
  - Rank by keyword frequency and rarity
  - Return top-K documents

Example:
Query: "How does photosynthesis work?"
Keywords: [photosynthesis, work, how]

Search:
  - Document 1: "Photosynthesis is the process..."
    - Contains "photosynthesis" → Good match
  - Document 2: "Plants and photosynthesis..."
    - Contains "photosynthesis" → Good match
  - Document 3: "How to fix solar panels..."
    - Contains "how" but not "photosynthesis" → Lower rank
  - Rank: Document 1 > Document 2 > Document 3

Pros:
✅ Fast (keyword index)
✅ Interpretable (see which keywords matched)
✅ Good for exact topic match
✅ Works without embeddings

Cons:
❌ Misses semantic similarity
❌ Fails on synonyms ("plant growth" vs "photosynthesis")
❌ Limited to exact terms

2. Dense Retrieval (Embedding-based)

Algorithm: Semantic similarity via embeddings

How it works:
  - Convert query to embedding
  - Convert all documents to embeddings
  - Find documents most similar to query
  - Return top-K documents

Example:
Query: "How do plants make food?"
Query embedding: [0.12, -0.45, 0.78, ...]

Document embeddings:
  - Document 1: "Photosynthesis: plant food production"
    - Embedding: [0.11, -0.46, 0.79, ...]  → Similarity: 0.98 (great!)
  - Document 2: "Plant growth requires energy"
    - Embedding: [0.09, -0.40, 0.82, ...]  → Similarity: 0.95 (good)
  - Document 3: "Solar panels convert light"
    - Embedding: [0.20, 0.30, -0.50, ...]  → Similarity: 0.45 (poor)
  - Rank: Document 1 > Document 2 > Document 3

Pros:
✅ Understands meaning (synonyms work!)
✅ Finds conceptually related documents
✅ Better for complex queries

Cons:
❌ Slower than BM25
❌ Requires embedding computation
❌ Black box (hard to debug why document matched)

3. Hybrid Retrieval

Combine keyword + semantic search:

Algorithm:
  - Get BM25 results (keyword match)
  - Get dense results (semantic match)
  - Merge and rank
  - Return top-K from merged results

Example implementation:
```python
def hybrid_search(query, documents, alpha=0.5):
    """
    alpha controls weight:
    - alpha=0: Pure keyword search
    - alpha=0.5: Balanced (recommended)
    - alpha=1: Pure semantic search
    """

    # Get keyword scores
    bm25_scores = bm25_search(query, documents)  # 0-1

    # Get semantic scores
    query_embedding = embed(query)
    semantic_scores = []
    for doc in documents:
        doc_embedding = embed(doc)
        similarity = cosine(query_embedding, doc_embedding)
        semantic_scores.append(similarity)

    # Combine scores
    combined_scores = []
    for i in range(len(documents)):
        combined = alpha * semantic_scores[i] + (1-alpha) * bm25_scores[i]
        combined_scores.append(combined)

    # Rank and return top-K
    ranked = sorted(zip(documents, combined_scores), key=lambda x: x[1], reverse=True)
    return [doc for doc, score in ranked[:k]]

# Usage:
results = hybrid_search(
    query="photosynthesis food production",
    documents=knowledge_base,
    alpha=0.5  # 50% keyword, 50% semantic
)

Pros: ✅ Best of both worlds ✅ Keyword catches exact matches ✅ Semantic catches concepts ✅ Higher recall and precision

Cons: ❌ Slower than pure keyword ❌ More complex to tune 

---

## Advanced Strategies

### Query Expansion
Problem: Short queries lack context

Query: "photosynthesis" - Documents about photosynthesis: Found ✓ - Documents about plant growth: Missed ✗ - Documents about energy production: Missed ✗

Solution: Expand query with related terms

Expansion techniques:

  1. Synonym expansion:
  2. photosynthesis → [photosynthesis, plant photosynthesis, food production]

  3. Search for all expanded terms

  4. LLM-based expansion:

  5. Use LLM to generate related questions
  6. "What are related questions to 'photosynthesis'?"
  7. LLM generates: [
  8. "How do plants convert light to energy?",
  9. "What is the role of chlorophyll?",
  10. "How does plant growth relate to photosynthesis?"
  11. ]

  12. Search for all variations

  13. Query decomposition:

  14. Break complex query into sub-queries
  15. "Photosynthesis vs cellular respiration" →
  16. ["What is photosynthesis?", "What is cellular respiration?", "Difference?"]
  17. Retrieve for each sub-query

Result: More comprehensive retrieval!

Cost-benefit: - Pro: Better recall (find more relevant docs) - Con: More searches (slower, more cost) - Trade-off: 5-10% better quality for 2-3x cost 

### Reranking
Problem: Initial retrieval isn't perfect

Retrieval returns top-10, but relevance ranking might be off: - Position 1: Somewhat relevant - Position 2: Very relevant - Position 3: Moderately relevant - ... - Position 10: Not relevant

Solution: Rerank retrieved documents

Reranking process: - ┌─────────────────────────────────────────┐ - Initial retrieval: top-100 documents │ - ┤ - Reranker (cross-encoder) │ - "Is this document relevant to query?" │ - Score each document 0-1 │ - ┤ - New ranking │ - ┤ - Return top-10 reranked │ - ┘

Example: Query: "How do plants absorb water?" Initial retrieval: 1. Osmosis in cells (score: 0.8) 2. Root structure (score: 0.75) 3. Photosynthesis (score: 0.65) 4. Plant metabolism (score: 0.60)

Reranking (cross-encoder): 1. Root structure (score: 0.92) ← Better match! 2. Osmosis in cells (score: 0.88) 3. Water transport (score: 0.85) 4. Plant metabolism (score: 0.55)

Result: Better ranking after reranking!

Cost vs benefit: - Cost: ~100ms per query (compute intensive) - Benefit: 5-15% accuracy improvement - Typical: Use if queries are expensive (complex topics) 

---

## Retrieval Fusion
Combine multiple retrieval methods:

Method 1: Keyword search → results_1 Method 2: Semantic search → results_2 Method 3: Sparse BM25 → results_3

Fusion algorithm:

  1. Get results from each method
  2. results_1: [doc_a, doc_c, doc_e]
  3. results_2: [doc_b, doc_a, doc_d]

  4. results_3: [doc_a, doc_b, doc_c]

  5. Score based on ranking

  6. Reciprocal Rank Fusion (RRF)
  7. For each doc: score = 1/(rank_1 + 1) + 1/(rank_2 + 1) + 1/(rank_3 + 1) │
  8. doc_a: 1/1 + 1/2 + 1/1 = 2.5
  9. doc_b: 0 + 1/1 + 1/2 = 1.5
  10. doc_c: 1/2 + 0 + 1/3 = 0.83
  11. doc_d: 0 + 1/3 + 0 = 0.33

  12. doc_e: 1/3 + 0 + 0 = 0.33

  13. Final ranking: doc_a > doc_b > doc_c > doc_d > doc_e

Benefit: - Combine strengths of multiple methods! - Each method catches different relevant docs 

---

## Performance Comparison
Metric Keyword Dense Hybrid Reranked ──────────────────────────────────────────────── Speed 5ms 50ms 55ms 155ms Recall 0.75 0.90 0.95 0.95 Precision 0.70 0.85 0.92 0.95 F1 Score 0.72 0.87 0.93 0.95 Cost $0.01 $0.05 $0.06 $0.16

Recommendation by use case:

Simple queries (facts): - Keyword search (fast, cheap)

Complex queries (reasoning): - Hybrid (good balance)

Production (accuracy critical): - Hybrid + reranking (best quality)

High volume (cost critical): - Keyword (cheap) or cached results

Summary: - Default: Hybrid retrieval - If too slow: Reduce to keyword or dense only - If not accurate: Add reranking ```

Key Takeaways

🔍 Keyword: Fast, but misses semantic meaning
📚 Dense: Slower, but understands concepts
🔀 Hybrid: Best balance of speed and quality
🔄 Query expansion: Find more relevant docs
📊 Reranking: Improve ranking with cross-encoders