Guides · Implementation
How to Build Long-Term Memory for an LLM Agent
Build long-term memory with an extraction pipeline, embedding model, vector or graph store, retrieval injection and consolidation job — durable memory that survives session restarts.
LTM build pipeline
Overview
Long-term memory pipeline
Extract → embed → store → retrieve → consolidate → forget.
Step 1
Define what to remember
Design a memory schema: user preferences, durable facts, episodic summaries — tagged by memory type.
Example fields: preference:diet, fact:location, episodic:last_support_ticket. Tag each extraction so retrieval can filter by type.
Step 2
Extraction
Run an LLM extraction prompt after each turn to pull durable facts from conversation — structured output, PII filter optional.
Pattern: “Extract only facts worth remembering across sessions. Output JSON array of {fact, type}.” Deduplicate against existing store before write.
Step 3
Storage backend
Choose vector vs graph based on whether facts change over time and whether relationships matter.
| If you need… | Use |
|---|---|
| Weaviate-native unified stack | Engram |
| Framework-agnostic managed API | Mem0 |
| Temporal facts + relationships | Zep (Graphiti) |
| Full control | pgvector, Pinecone, Redis |
Step 4
Retrieval and injection
Search top-k memories (typically 5–10), score by recency × importance × relevance (Park et al., 2023), inject into system prompt.
Mem0’s LOCOMO eval uses median search latency of 0.148 s with ~1,800 tokens per query vs 26,000 for full-context (Chhikara et al., 2025) — selective retrieval is the cost win.
Step 5
Consolidation and maintenance
Run background jobs to merge duplicates, resolve conflicts and apply TTLs on stale facts.
When a user corrects a fact (“I moved to Berlin” after “I live in Munich”), invalidate or supersede the old memory — Zep’s temporal graph does this natively; vector stores need explicit conflict handling.
Example
Worked example: personal assistant LTM
User says “I’m vegetarian and allergic to nuts” → extracted as two facts → stored with user_id → recalled next week when user asks about restaurants.
Week 1: extraction writes {fact: "vegetarian", type: "preference:diet"} and {fact: "nut allergy", type: "fact:health"}. Week 2: retrieval surfaces both before the LLM answers “suggest dinner spots near me” — without resending the full Week 1 transcript.
FAQ
Frequently asked questions
What is the fastest long-term memory stack?
Engram on Weaviate Cloud if already on Weaviate; Mem0 API for framework-agnostic managed memory; LangMem for LangGraph. See Step 3 above and best AI memory tools.
Can I use Mem0 for long-term memory?
Yes — Mem0 is built for per-user long-term semantic and episodic memory across sessions. LOCOMO J score 66.9 in published eval (Chhikara et al., 2025). See Mem0 alternatives.
How do I build long-term memory in LangGraph?
Use LangMem's checkpointer + store for native LTM, or Engram/Mem0/Zep as external APIs. See LangMem and add memory guide.
How does Engram handle long-term memory?
Engram runs async extract → transform → commit pipelines on Weaviate, scoped per user. Memories persist across sessions via hybrid search. See Engram explained.
How does Zep handle long-term memory?
Zep stores facts in a temporal knowledge graph (Graphiti) with validity windows — ideal when facts change over time. LongMemEval accuracy 71.2% on gpt-4o (Rasmussen et al., 2025). See Zep alternatives.
What does the MemGPT paper say about long-term memory?
MemGPT (Packer et al., 2023) pages memories between context (RAM) and deep store — 93.4% on the DMR benchmark. Letta implements this pattern. See virtual context and MemGPT.
Is a vector database enough for long-term memory?
A vector DB is the storage layer — you still need extraction, retrieval ranking, consolidation and eviction. Or use a framework (Engram, Mem0, Zep) that implements the full pipeline. See vector databases for memory.
How do I benchmark long-term memory?
Run LOCOMO for long-conversation recall and LongMemEval for cross-session recall on your domain queries. See evaluation hub.
How do I add long-term memory to Claude-based agents?
Claude's API is stateless — implement external LTM via Engram, Mem0, Zep or DIY vector store. See persist conversation memory.