Infrastructure · RAG
RAG Explained: Retrieval-Augmented Generation and Memory
RAG retrieves relevant documents from a knowledge base and injects them into the prompt — grounding answers in facts; agent memory adds dynamic, per-user information RAG alone cannot provide.
RAG
Static docs · shared corpus · batch index
Memory
Per-user facts · write each turn · dynamic
Definition
What is RAG?
Index documents → embed chunks → retrieve on query → augment generation with retrieved context.
Retrieval-augmented generation (Lewis et al., 2020) grounds LLM answers in a static or slowly changing document corpus — product manuals, policy PDFs, knowledge bases. The corpus is shared across users; updates happen via batch re-indexing, not per conversation.
Pipeline
RAG pipeline components
- Chunking — split documents into retrievable segments
- Embedding — vectorize chunks for similarity search
- Vector store — index embeddings (Weaviate, Pinecone, pgvector)
- Retriever — fetch top-k chunks for the query
- Reranker — optional second-stage relevance scoring
- Prompt template — inject chunks into the LLM context
Same retrieval technology powers agent memory — different data lifecycle. RAG indexes docs once; memory writes per interaction.
Comparison
RAG vs AI agent memory
RAG retrieves from a fixed knowledge base; memory stores dynamic per-user facts updated every turn.
| Dimension | RAG | Agent memory |
|---|---|---|
| Data | Document corpus | User/session interactions |
| Updates | Batch re-index | Per interaction write |
| Personalization | None (shared docs) | Per-user facts |
| Query | “What does the manual say?” | “What did this user tell me?” |
Combined
Combining RAG and memory
Production agents use RAG for company knowledge and memory for user context — separate or shared vector stores.
Architecture: RAG index for org docs + Engram/Mem0/Zep collection for per-user memories. Retrieve from both before each LLM call; context engineering allocates token budget across sources.
FAQ
Frequently asked questions
Is RAG the same as AI agent memory?
No — RAG retrieves from a static document corpus shared across users. Agent memory is dynamic, per-user and updated each interaction. See memory vs RAG.
Do production agents need both RAG and memory?
Usually yes — RAG for org knowledge (manuals, policies), memory for user preferences and conversation facts. See RAG with memory.
Can RAG and memory share the same vector database?
Yes — separate collections in Weaviate/Engram, Pinecone namespaces or pgvector tables. Different write pipelines and metadata schemas.
Mem0 vs RAG?
Agent memory APIs (Engram, Mem0) extract, store and retrieve per-user facts. RAG indexes static documents. They complement each other — RAG for org knowledge, memory for user state.
LangChain RAG + memory together?
LangChain supports both VectorStoreRetriever (RAG) and memory classes (ConversationBuffer, LangMem store). Combine in a single chain with context budget allocation.
Zep vs RAG?
Zep is temporal graph memory for per-user facts — not document RAG. Use Zep for evolving user relationships; RAG for static org docs. Zep LongMemEval +18.5% vs baseline (Rasmussen et al., 2025).
Engram vs RAG?
Engram is a Weaviate memory layer for per-user semantic memory — not a RAG pipeline. Use Engram for dynamic user facts and a separate RAG index for org docs. See Engram explained and memory vs RAG.