Memory Fundamentals: Types, Purpose, and Architecture¶
Overview¶
Memory is the system that stores and retrieves information over time, enabling agents to: - Maintain context across interactions - Learn from past experiences - Make better decisions based on history - Improve performance over time
The Memory Problem¶
Without Memory¶
User: "What's my favorite color?"
Agent: "I don't know. What is it?"
User: "Blue"
Agent: [Remembers for this conversation]
[Later, new conversation]
User: "What's my favorite color?"
Agent: "I don't know. What is it?"
[Forgot everything from before]
With Memory¶
User: "What's my favorite color?"
Agent: "I don't know. What is it?"
User: "Blue"
Agent: [Stores: "User's favorite color = blue"]
[Later, new conversation]
User: "What's my favorite color?"
Agent: "Blue" [Retrieved from memory]
4 Memory Types (Cognitive Science Inspiration)¶
1. Short-Term/Working Memory¶
What: Information currently in focus (current task)
Duration: Seconds to minutes (single conversation)
Capacity: Small (like context window of LLM)
Storage: LLM context window, current execution state
Purpose: - Track current goal - Store intermediate results - Hold reasoning steps - Manage current state
Example:
class ShortTermMemory:
def __init__(self):
self.current_context = ""
self.working_state = {}
self.recent_events = []
def add_to_context(self, text):
"""Add to current working context"""
self.current_context += text
def get_context(self):
"""Get current context for LLM"""
return self.current_context
def clear(self):
"""Clear at end of task"""
self.current_context = ""
2. Episodic Memory¶
What: Memories of specific events that happened
Duration: Hours to days to lifetime
Capacity: Large (can grow indefinitely)
Storage: Vector database, time-indexed
Purpose: - Remember what happened before - Recall similar past events - Learn from past experiences - Provide grounding and context
Example:
class EpisodicMemory:
def __init__(self):
self.events = [] # List of past events
self.vector_db = VectorStore()
def add_event(self, event: dict):
"""Store an event"""
event["timestamp"] = now()
self.events.append(event)
# Add to vector DB for semantic search
embedding = embed(event["description"])
self.vector_db.add(embedding, event)
def recall(self, query: str, k=5):
"""Retrieve similar past events"""
query_embedding = embed(query)
return self.vector_db.search(query_embedding, k)
def get_recent(self, hours=24):
"""Get recent events"""
cutoff = now() - timedelta(hours=hours)
return [e for e in self.events if e["timestamp"] > cutoff]
3. Semantic Memory¶
What: General knowledge and facts (not tied to specific events)
Duration: Persistent (lifetime)
Capacity: Very large
Storage: Knowledge graphs, knowledge bases, embeddings
Purpose: - Store facts and knowledge - Don't need to remember when/where learned - Support reasoning and decision-making - Enable transfer learning
Example:
class SemanticMemory:
def __init__(self):
self.knowledge_base = KnowledgeGraph()
self.facts = {}
def add_fact(self, subject, predicate, object):
"""Store a fact"""
# Example: add_fact("GPT-4", "is_a", "language_model")
self.knowledge_base.add_triple(subject, predicate, object)
self.facts[f"{subject}:{predicate}"] = object
def query(self, subject, predicate):
"""Query for a fact"""
return self.knowledge_base.query(subject, predicate)
def get_related(self, entity):
"""Get all facts about an entity"""
return self.knowledge_base.get_all(entity)
4. Procedural Memory¶
What: Knowledge of how to do things (skills, procedures)
Duration: Persistent
Capacity: Large
Storage: Learned patterns, policies, neural weights
Purpose: - Remember how to accomplish tasks - Improve performance through practice - Store successful strategies - Enable skill learning
Example:
class ProceduralMemory:
def __init__(self):
self.procedures = {}
self.policies = {}
self.success_rates = {}
def learn_procedure(self, task_type: str, procedure: Callable):
"""Learn how to do a task"""
self.procedures[task_type] = procedure
self.success_rates[task_type] = 0.0
def record_success(self, task_type: str):
"""Update success rate for procedure"""
current = self.success_rates.get(task_type, 0.0)
self.success_rates[task_type] = current + 0.1 # Improve
def get_best_procedure(self, task_type: str):
"""Get most successful procedure for task"""
if task_type in self.procedures:
return self.procedures[task_type]
return None
Memory Architecture¶
Hierarchical Memory System¶
graph TD
A["LLM Context Window<br/>(Short-Term Memory)<br/>• Current goal<br/>• Recent observations<br/>• Working state<br/>Capacity: 100K-1M tokens"]
B["Retrieval-Augmented System"]
C["Vector DB<br/>(Episodic)<br/>Similar past events<br/>to retrieve"]
D["Knowledge Base<br/>(Semantic)<br/>Facts and<br/>relationships"]
E["Policy Store<br/>(Procedural)<br/>Learned procedures<br/>& success patterns"]
F["Persistent Storage Layer<br/>• Database<br/>• Long-term archives"]
A --> B
B --> C
B --> D
B --> E
B --> F
C -.->|Retrieval| A
D -.->|Retrieval| A
E -.->|Retrieval| A---
## Memory Operations
### Core Operations
#### 1. **Store/Add**
```python
memory.add(event, embedding, metadata)
2. Retrieve/Recall¶
results = memory.retrieve(query, k=5)
3. Update¶
memory.update(id, new_content)
4. Forget/Delete¶
memory.forget(id)
5. Consolidate¶
memory.consolidate()
Memory Levels¶
Level 1: Immediate (LLM Context)¶
Capacity: 100K-1M tokens
Duration: Single turn
Access: Fast (already in context)
Use: Current reasoning
Level 2: Conversation (Session Memory)¶
Capacity: 10M-100M tokens
Duration: Single conversation (hours)
Access: Fast (can fit in context)
Use: Conversation history
Level 3: Long-Term (Persistent)¶
Capacity: Unlimited
Duration: Weeks/months/years
Access: Medium (retrieval needed)
Use: Cross-session learning
Memory Retrieval Strategies¶
1. Semantic Similarity (Vector Search)¶
# Find memories similar to query
query_embedding = embed("Find research on climate change")
similar = vector_db.search(query_embedding, k=5)
# Returns: [paper1, paper2, paper3, ...]
Best for: Conceptual matching
2. Keyword/BM25 Search¶
# Find memories with matching keywords
results = memory.search("climate change global warming", k=5)
Best for: Exact matching
3. Time-Based Retrieval¶
# Find recent memories
recent = memory.get_since(hours=24)
Best for: Temporal relevance
4. Metadata Filtering¶
# Find memories by properties
results = memory.filter(
source="research_paper",
year=2024,
confidence=0.9
)
Best for: Structured queries
5. Hybrid Retrieval¶
# Combine multiple strategies
results = memory.hybrid_search(
query="climate research",
filters={"year": 2024},
recent_weight=0.3
)
Best for: Complex retrieval
Memory Retention Policies¶
Not all memories should be kept forever. Design retention policies:
Policy 1: Time-Based¶
class TimeBasedRetention:
def apply(self, memory_item):
age = now() - memory_item.created_at
if age < 1_day:
return True # Keep
elif age < 7_days:
return random() < 0.8 # 80% chance
elif age < 30_days:
return random() < 0.2 # 20% chance
else:
return False # Delete
Policy 2: Importance-Based¶
class ImportanceBasedRetention:
def apply(self, memory_item):
if memory_item.importance > 0.8:
return True # Keep high-importance
elif memory_item.access_count > 5:
return True # Keep frequently accessed
elif memory_item.relevance_score > 0.6:
return True # Keep relevant
else:
return False # Delete low-value
Policy 3: Space-Based (LRU)¶
class SpaceBasedRetention:
def __init__(self, max_size_gb=10):
self.max_size = max_size_gb
def apply(self, memory):
if memory.current_size() > self.max_size:
# Delete least recently used
lru = memory.get_least_recently_used()
memory.delete(lru)
Memory Challenges¶
Challenge 1: Context Window Limit¶
Problem:
• Agents generate lots of text (actions, reasoning, results)
• LLM context windows finite (100K-1M tokens)
• Can't fit all history in context
Solution:
• Use short-term memory for current task only
• Use vector DB for retrieval-augmented context
• Compress/summarize old conversations
Challenge 2: Semantic Drift¶
Problem:
• Memories get less relevant over time
• What was important may become stale
• Semantic meaning can change
Solution:
• Update embeddings periodically
• Re-rank by relevance
• Use time-based decay
Challenge 3: Hallucination/Contamination¶
Problem:
• LLM might modify memories
• Incorrect information stored
• Corrupts future decisions
Solution:
• Validate before storing
• Version control memories
• Separate facts from reasoning
Challenge 4: Scalability¶
Problem:
• Memory systems grow over time
• Search becomes slow (O(n) or worse)
• Storage costs increase
Solution:
• Index vectors for fast search
• Partition by time or domain
• Implement compression
Memory System Design Pattern¶
class AgentMemory:
def __init__(self):
# Short-term: current context
self.context_window = []
# Medium-term: current session
self.conversation_history = []
# Long-term: persistent storage
self.episodic_store = VectorDB() # Events
self.semantic_store = KnowledgeBase() # Facts
self.procedural_store = PolicyStore() # Procedures
def remember(self, event: dict, memory_type: str):
"""Store event in appropriate memory level"""
if memory_type == "immediate":
self.context_window.append(event)
elif memory_type == "session":
self.conversation_history.append(event)
elif memory_type == "episodic":
self.episodic_store.add(event)
elif memory_type == "semantic":
self.semantic_store.add_fact(event)
elif memory_type == "procedural":
self.procedural_store.add_procedure(event)
def recall(self, query: str, memory_type: str = "all"):
"""Retrieve relevant memories"""
results = []
if memory_type in ["all", "immediate"]:
results.extend(self._search_context(query))
if memory_type in ["all", "session"]:
results.extend(self.conversation_history)
if memory_type in ["all", "episodic"]:
results.extend(self.episodic_store.search(query))
if memory_type in ["all", "semantic"]:
results.extend(self.semantic_store.query(query))
if memory_type in ["all", "procedural"]:
results.extend(self.procedural_store.find(query))
return self._rank_results(results, query)
def consolidate(self):
"""Clean up and compress memories"""
# Apply retention policies
self._apply_retention_policies()
# Compress old conversations
self._compress_conversations()
# Update indices
self._update_indices()
Memory Best Practices¶
1. Be Explicit About What to Remember¶
# ✅ Good: Clear memory intent
memory.remember(
{
"type": "fact",
"subject": "user",
"content": "User prefers concise responses",
"importance": "high"
},
memory_type="semantic"
)
# ❌ Bad: Vague
memory.remember({"data": "something"})
2. Index for Fast Retrieval¶
# ✅ Good: Index on important fields
memory.create_index("user_id")
memory.create_index("timestamp")
memory.create_index("importance")
# ❌ Bad: No indexing
memory.search(query) # O(n) scan each time
3. Have a Retention Policy¶
# ✅ Good: Delete old low-value memories
memory.apply_retention_policy(
"delete_if_old_and_low_importance",
days=30,
importance_threshold=0.3
)
# ❌ Bad: Keep everything forever
memory.remember(everything)
4. Monitor Memory Health¶
# ✅ Good: Track metrics
print(f"Memory size: {memory.size_gb()}")
print(f"Most retrieved: {memory.most_accessed(k=10)}")
print(f"Search latency: {memory.avg_search_time_ms()}")
# ❌ Bad: No monitoring
# Don't know if memory is healthy
Memory vs Knowledge¶
Memory (Episodic)¶
- Specific events that happened
- Time-stamped and contextual
- "I helped a user analyze financial data on Jan 15"
- Used for: Learning from experience
Knowledge (Semantic)¶
- General facts and concepts
- Not tied to specific time/place
- "Financial analysis involves trend analysis"
- Used for: Understanding and reasoning
Both are needed: - Memory → Learn from specific experiences - Knowledge → Transfer learning across situations
Key Takeaways¶
- 4 memory types - Short-term, Episodic, Semantic, Procedural
- Hierarchical architecture - From LLM context to persistent storage
- Retrieval critical - Memory only useful if you can find what you need
- Retention policies - Can't keep everything forever
- Challenges exist - Context limits, semantic drift, hallucination
- Design for scale - Start simple, add indexing as needed
Next Steps¶
- Read Short Term Memory - Current context management
- Read Long Term Memory - Persistent storage
- Read Vector Stores - Semantic search
Last Updated: August 9, 2026