Guides · RAG + memory
How to Combine RAG and Memory in an Agent
Combine RAG and memory by retrieving static documents from your knowledge base and dynamic user memories from a memory store — injecting both into the prompt with separate sections and ranking.
RAG
Static org docs · shared
Memory
Dynamic user facts · per-user
Why
Why combine RAG and memory
RAG holds company truth; memory holds user and session truth — production agents need both.
Example support bot: RAG retrieves the refund policy (static); memory recalls that this customer filed ticket #8821 last Tuesday (dynamic). RAG alone cannot personalize; memory alone cannot answer policy questions.
Pattern
Architecture pattern
Query → parallel RAG retriever + memory retriever → merge context under token budget → LLM.
User query
Incoming message
RAG search
Doc index top-k
Memory search
User facts top-k
Merge
Separate prompt sections
Generate
LLM response
Steps
Implementation steps
Separate indexes, dual retrieval, distinct prompt sections — never conflate stores.
- Separate indexes —
docs_collectionfor RAG,memories_collectionfor user facts (or separate namespaces on Engram/Weaviate) - Dual retrieval — run both searches in parallel on each turn
- Prompt template —
## Company knowledge(RAG) +## User memories(memory) - Write path — only memory store updates per turn; RAG re-indexes on doc changes
docs = rag.search(query, top_k=5)
memories = memory.search(query, user_id=user.id, top_k=5)
prompt = template.format(docs=docs, memories=memories, query=query)
response = llm(prompt)Frameworks
Framework notes
Engram unifies RAG + memory on Weaviate; Mem0 pairs with any RAG index; LangChain/LlamaIndex compose both retrievers.
- Engram (Weaviate) — doc chunks + user memories on one platform, separate collections
- Mem0 + vector RAG — Mem0 for user memory, Pinecone/pgvector for docs
- LangMem + RAG — LangGraph retriever chain + memory store
- Zep + RAG — temporal memory for facts; RAG for static policies
FAQ
Frequently asked questions
Can RAG and memory use the same vector database?
Yes — separate collections or namespaces: one for document chunks, one for user memories. Engram (Weaviate) unifies both on one platform. See vector databases.
What if RAG and memory conflict?
RAG wins for org policy (authoritative docs). Memory wins for user-specific context. Label sections clearly in the prompt so the model prioritizes policy for compliance questions.
How do I combine Mem0 with RAG?
Run Mem0 memory.search() for user facts and your RAG retriever for docs in parallel — inject both into the prompt. Engram can do both on Weaviate. See Mem0 alternatives.
How do I combine Engram with RAG?
Use separate Weaviate collections — docs for RAG, memories for user facts. Single platform, dual retrieval. See Engram explained.
What is the token cost of RAG + memory?
Budget both retrievers — e.g. 5 doc chunks + 5 memories. Mem0 selective retrieval uses ~1,800 tokens vs ~26,000 full-context (Chhikara et al., 2025). Tune top_k per section.
Can Zep handle both RAG and memory?
Zep is temporal agent memory — pair it with a separate RAG index for static documents. Zep for evolving facts; RAG for policy manuals.
How do you evaluate RAG + memory together?
RAG: doc-QA hit rate. Memory: LOCOMO/LongMemEval recall. Run both on your domain before production. See evaluation hub.
LangChain RAG + memory pattern?
Compose a RAG retriever chain and Engram/Mem0/LangMem memory store — inject both into the prompt template. See add memory guide.
Should memory write to the RAG index?
No — keep stores separate. User memories update per turn; RAG re-indexes on document changes. Conflating them breaks per-user scoping.