What Is Virtual Context? MemGPT and Memory Paging for LLMs

What is MemGPT?

MemGPT is a system that gives an LLM agent effectively unbounded context by letting the agent manage several memory tiers itself, moving information between the context window and external storage through function calls. It was introduced in the 2023 research paper “MemGPT: Towards LLMs as Operating Systems” by Charles Packer, Sarah Wooders, Kevin Lin, Vivian Fang, Shishir G. Patil, Ion Stoica and Joseph E. Gonzalez at UC Berkeley, published as arXiv:2310.08560.

What MemGPT does, stated plainly: it decides what the model sees. A conversation that outgrows the context window normally loses its earliest turns, and MemGPT keeps them reachable by writing them to external storage and retrieving them on demand.

The name now carries two meanings, and the distinction matters when you search for it. MemGPT is the research system; Letta is the open-source framework the same team built from it. Documentation, releases and installation instructions live under Letta, while the paper, the benchmark numbers and the original design vocabulary remain under MemGPT.

The technique that makes the decision possible has its own name, and it is the term the paper leads with: virtual context management.

What is an LLM operating system?

An LLM operating system is a layer that manages a model’s limited resources on its behalf, exactly as an operating system manages a computer’s: it decides what occupies the scarce fast memory, moves the rest to slower storage, and interrupts execution when something needs attention. The model becomes the processor rather than the whole machine.

Three properties of a conventional operating system carry over, and the paper uses all three.

• Virtual memory. The prompt is treated as physical memory and external stores as disk, so the agent appears to hold far more than it can physically fit.
• Interrupts. The agent can pause to run a memory operation before it answers, instead of being forced to respond in a single pass. This is what the paper means by control flow between the system and the user.
• System calls. Reading and writing memory is a function call the model issues, which is why the pattern generalises to any model with reliable tool use.

Calling the model an operating system is a claim about responsibility rather than about capability: the agent now owns a scheduling problem it did not have before. Seeing how it solves that problem means looking at the three memory tiers it schedules across.

What happens when the context window fills up?

When the message queue reaches 70% of the context window, MemGPT inserts a system message warning the agent of memory pressure, so the agent can store anything important in external memory before it is evicted. The threshold is the most concrete number in the design: it converts “the window is nearly full” from an error state into a scheduled event the agent can act on.

Once the limit is reached, evicted messages are replaced by a recursive summary that keeps the gist of what was flushed. The full messages remain in recall storage and can be searched back into the prompt, so eviction removes them from the prompt rather than from the system.

The honest cost sits in the same mechanism. Recursive summarisation preserves the gist and loses detail by construction, and the paper acknowledges the tradeoff without quantifying how much is lost. Any agent that relies on summaries rather than on retrieval inherits that loss, which is why the eviction policy matters as much as the storage. The general version of this problem is covered in how agents decide what to forget.

That machinery exists to serve two workloads in particular, which is where MemGPT is actually used.

What is the Deep Memory Retrieval benchmark, and how did MemGPT score?

Deep Memory Retrieval, usually shortened to DMR, is the task the MemGPT team introduced to test conversational consistency: the agent is asked a question that can only be answered from an earlier session, and its reply is scored against a gold answer. It runs on the Multi-Session Chat dataset from Xu et al. (2021), in which human labellers held five sessions of roughly a dozen messages each while playing a consistent persona. The MemGPT team added a sixth session containing a single question and answer pair, and scored responses with ROUGE-L recall alongside an LLM judge.

The table below gives every result from Table 2 of the paper. Each pair compares a fixed-context baseline against MemGPT running on the same underlying model.

Two details decide how to read those numbers, and most summaries omit both. First, the model names are pinned to specific endpoints: GPT-4 is gpt-4-0613 with an 8,192 token window, GPT-4 Turbo is gpt-4-1106-preview with 128,000 tokens, and GPT-3.5 Turbo is gpt-3.5-turbo-1106 with 16,385 tokens. A 92.5% result against an 8k baseline is a different claim from the same result against a 128k baseline. Second, the baselines were not blind: they received a lossy recursive summary of the five prior sessions, while MemGPT had to retrieve the full history through paginated search. The comparison is between summarisation and retrieval, not between memory and nothing.

Later systems report higher figures on the same task. Zep, a temporal knowledge graph architecture for agent memory (Rasmussen, Paliychuk, Beauvais, Ryan and Chalef, arXiv:2501.13956), reports 94.8% against MemGPT’s 93.4%. The gap is real but narrow, and it is measured on a benchmark the MemGPT team designed, which is worth stating whenever the two are compared. Benchmarks for agent memory are covered in what the LOCOMO benchmark measures and which metrics matter for agent memory.

Every figure above sits in Table 2 of the paper rather than in its abstract, which matters if you go looking for them yourself: where to read the MemGPT paper.

Is MemGPT the same thing as Letta?

No. MemGPT is the research system described in the paper, and Letta is the open-source agent framework that the same team developed from it. If you are looking for the concept, the tier vocabulary or the benchmark results, you want MemGPT. If you are looking for something to install and run, you want Letta.

The practical consequence shows up in searches for alternatives. People who ask for MemGPT alternatives almost always want a framework they can deploy, which makes the honest comparison set Engram (Weaviate), Mem0, Zep and LangMem rather than other research prototypes. That comparison is made in full on which Letta alternatives are worth using.

Before choosing any of them, most teams ask a blunter question first: whether a longer context window removes the need for paging at all.

How is virtual context different from a vector memory API?

A vector memory API retrieves into a prompt that someone else assembles, while virtual context makes the agent responsible for assembling the prompt. The distinction is about who holds the policy, not about which storage engine sits underneath, and both classes commonly use the same vector store.

The three architecture classes below cover most production systems in 2026.

None of the three is strictly better. Paging gives the agent the most control and the most ways to waste tokens; a memory API gives the application predictable behaviour and less adaptivity. See how the memory APIs compare. Choosing between them is easier once the weaknesses of the paging design are on the table, and the paper is unusually clear about where its own limits sit.

What came after MemGPT?

Three lines of work extend or replace the fixed-tier design: temporal knowledge graphs, agentic memory that organises itself, and background consolidation that runs while the agent is idle. Each one targets a limit named in the section above.

1. Temporal knowledge graphs. Zep (Rasmussen, Paliychuk, Beauvais, Ryan and Chalef, arXiv:2501.13956) stores facts and their validity over time rather than paging message history, and reports 94.8% on the same DMR benchmark against MemGPT’s 93.4%. It addresses stale facts, which paging does not handle.
2. Agentic memory. A-MEM (Xu, Liang, Mei, Gao, Tan and Zhang, arXiv:2502.12110, February 2025) lets the memory structure organise itself instead of relying on fixed tiers, and reports improvement over prior systems across six foundation models. The paper’s abstract states the improvement in general terms rather than as a single number.
3. Background consolidation. Sleep-time compute moves summarisation and reorganisation off the critical path, so the latency cost the MemGPT paper never measured is paid while the user is not waiting. See how sleep-time compute consolidates memory.

Virtual context remains the reference design that the rest are measured against, which is why its vocabulary of main context, recall and archival storage still turns up in systems that do not page at all. If you are moving from reading to building, the next step is choosing where memory writes happen in your own agent loop, set out step by step in how to add memory to an AI agent and in how to build long-term memory for an LLM agent.