Architecture · Compression
Summarizing and Compressing Agent Memory
Memory summarization compresses long conversation histories into shorter representations — fitting more effective context into the window and cutting token cost, with a trade-off in fine-grained detail.
Raw history
26,000 tokens · 40 messages
Summary
~1,800 tokens · key facts preserved
Motivation
Why summarize agent memory?
Context limits, token cost and context rot — raw history eventually exceeds what models can use effectively.
Mem0’s selective retrieval uses ~1,800 tokens per LOCOMO query vs ~26,000 full-context — over 90% fewer (Chhikara et al., 2025). Summarization is the main compression lever before selective retrieval kicks in.
Techniques
Summarization techniques
- Rolling window summary — summarize every N turns, keep only the latest summary + recent messages
- Recursive hierarchical summary — summarize summaries for multi-session histories
- Extractive — pull key sentences verbatim (lower hallucination risk)
- Abstractive — LLM rewrites in fewer tokens (higher compression, higher risk)
- Episode-level summaries — one summary per session, stored as episodic memory
Disambiguation
Summarization vs consolidation
Summarization compresses representation; consolidation promotes to LTM and merges duplicates — often used together.
Typical pipeline: summarize session transcript → extract durable facts → merge into long-term store → evict raw transcript. Consolidation without summarization still works for fact-extraction pipelines (Engram, Mem0).
Frameworks
Summarization in frameworks
- Engram — Weaviate-backed write pipeline; summarize before store in custom middleware
- Letta — built-in compaction tools for core/archival memory paging
- LangChain — ConversationSummaryMemory, ConversationSummaryBufferMemory
- Mem0 — extraction pipeline compresses to salient facts automatically
- Custom — LLM summarize-before-store in your middleware
Quality
Measuring summarization quality
Recall of key facts post-summary — run LOCOMO or LongMemEval before and after compression.
LOCOMO J scores: Mem0 66.9, Zep 66.0, LangMem 58.1 (Chhikara et al., 2025 eval). If summarization drops recall below your threshold, reduce compression ratio or switch to fact extraction instead of abstractive summary.
FAQ
Frequently asked questions
Should agents summarize memory every turn?
No — summarize at session end, memory pressure thresholds or every N turns. Per-turn summarization adds latency and can lose facts before extraction runs.
Does summarization lose accuracy?
Yes — abstractive summaries can drop fine-grained facts. Validate recall@k on LOCOMO after compression. Fact extraction (Engram, Mem0) often preserves accuracy better than free-form summary.
Summarization vs retrieval for long histories?
Use both: summarize for thread overview, retrieve specific facts from LTM for precision. Mem0 uses ~1,800 vs ~26,000 tokens per LOCOMO query (Chhikara et al., 2025).
Can Claude summarize agent memory?
Yes — any LLM can summarize transcripts before store or on session end. Same pattern for Claude, GPT-4o or open models. See persist conversation memory.
How much token savings does summarization provide?
Mem0 reports ~1,800 vs ~26,000 tokens per LOCOMO query — over 90% reduction vs full-context (Chhikara et al., 2025). Your ratio depends on compression aggressiveness.
Where do you persist summaries?
Vector store as episodic memory, archival tier (Letta), or replace raw transcript in Redis/Postgres. See storage backends.
Production summarization pattern?
Session end → LLM summarize → extract facts → write to LTM → evict raw transcript. Run LOCOMO in CI to gate recall regressions. See reduce token cost.