What Are the AI Agent Memory Frameworks?

What are the four classes of memory framework?

Memory API, temporal knowledge graph, virtual paging, and framework-native, separated by who decides what enters the context window. Storage engine is not the dividing line, because several classes commonly sit on the same vector database.

A memory API is called by the application: write this, retrieve what is relevant. The service handles extraction and ranking, behaviour is predictable, and adaptivity is limited to what the API exposes. Engram (Weaviate), Mem0 and Supermemory sit here, and it is the class most products should start with.

A temporal knowledge graph stores facts with the period during which they held, so a changed fact invalidates its predecessor rather than competing with it. Zep sits here. The capability is narrow and, where it applies, hard to replicate: it is the only class that answers questions about the past natively.

Virtual paging gives the agent tools to fetch and evict its own memories across tiers, which is the MemGPT design that became Letta. Maximum control, maximum ways to spend tokens, and retrieval quality that depends on the agent asking the right question.

Framework-native memory lives inside an agent framework you already run, so there is no extra service to operate. LangMem and Cognee sit here, and the trade is portability: the memory layer follows the framework.

With the classes clear, the individual tools are easier to place: every framework in this cluster.

Which AI agent memory system is best?

There is no single best, and the honest answer is a mapping from workload to class: a memory API for per-user facts, a temporal graph for facts that expire, paging for one very long conversation, and framework-native memory when you already run a framework.

The reason a ranking cannot settle it is visible in the published benchmarks. Letta reports 93.4% on Deep Memory Retrieval against 35.3% for the same model with no memory (Packer et al., arXiv:2310.08560), Zep reports 94.8% on that same benchmark (Rasmussen et al., arXiv:2501.13956), and Mem0 reports a LOCOMO judge score of 66.9 against 72.9 for a full-context baseline costing roughly fifteen times the tokens (Chhikara et al., 2025). Those are three different tests, and two of the three were reported by the team whose tool was measured.

What the numbers do establish is the category’s real value proposition. Memory systems trade a small amount of accuracy against sending everything, in exchange for large reductions in cost and latency, and for working at all once a conversation exceeds the window. Mem0’s figures put that at roughly 1,800 tokens per query instead of 26,000, with p95 latency of 1.44 seconds against 17.1 seconds.

A practical shortcut for a first build: start with a managed memory API, instrument retrieval, and only move to another class when you can point at a specific behaviour the API will not give you. By then you know what you need, which is not true at the start.

Frameworks manage memory, and how they do it is the same loop in every case: how agents manage memory.

Do you need a memory framework, or can you build it?

A first version is genuinely easy to build, and what follows is the part that justifies a framework. An embed, an insert and a search take an afternoon. Extraction that keeps the right facts, deduplication that catches paraphrases, scoping enforced at query time, ranking, consolidation and eviction are each small and together are a component with an owner.

Building makes sense in two situations. When retrieval behaviour is your differentiator, a framework’s opinions become constraints you fight. When data residency forbids a managed service, the choice is made for you, and the open-source options in this cluster are genuinely deployable rather than demo-ware.

Buying makes sense in most others, and particularly early. A managed layer sets the extraction policy, which may not match your domain, and holds your memories in someone else’s system, which is a procurement question as much as a technical one. Against that it removes an entire component from your operational surface while you are still learning what your agent needs to remember, which is usually the more valuable thing at that stage.

A reasonable middle path is common: adopt a managed API for the proof of concept, instrument retrieval quality, and build only once you can name a specific behaviour the product will not give you. By then the requirement is concrete, which it is not at the start. The hosting dimension is covered on open source versus managed memory, and the do-it-yourself route on Redis for agent memory.

The types of memory these operations act on are covered on the types of AI agent memory, and the build itself on how to add memory to an AI agent.

Frequently asked questions

The questions that follow: whether frameworks are interchangeable, and what a framework does not solve.