Architecture · Eviction
Forgetting in AI Agents: Eviction and Decay
Agents must forget — through eviction policies, TTLs and relevance decay — or memory stores grow stale, expensive and contradictory.
Forget path
Why forget
Why forgetting is necessary
Cost control, stale fact removal, contradiction resolution, privacy and retrieval precision.
- Cost — unbounded stores increase embedding, storage and retrieval costs
- Staleness — old preferences and policies poison responses
- Contradictions — conflicting facts degrade trust
- Privacy — GDPR right to erasure requires delete APIs
- Precision — too many low-relevance memories dilute top-k retrieval
Policies
Eviction policies
| Policy | How it works | Best for |
|---|---|---|
| LRU | Evict least-recently accessed | Bounded caches |
| TTL per type | Hard expiry by memory category | Session vs preference facts |
| Salience threshold | Delete below importance score | Noisy extraction cleanup |
| Explicit delete tool | Agent or user triggers forget | Memory-as-tool patterns |
| User-triggered forget | “Forget what I said about X” | Privacy compliance |
Decay
Memory decay models
Soft forget (downrank in retrieval) vs hard delete (remove from store).
Park et al. (2023) use recency decay at 0.995 per hour since last access in Generative Agents scoring. Downrank stale memories in retrieval; hard-delete when archival value is zero or policy requires erasure.
Complement
Forgetting vs summarization
Summarization compresses; forgetting removes — complementary strategies.
Typical flow: summarize session → extract facts to LTM → evict raw transcript. Summarization preserves gist; forgetting removes noise entirely.
Compliance
Framework and compliance considerations
- Engram — delete via Weaviate collection APIs; scope by tenant
- Mem0 — delete memory API per user/fact
- Zep — bi-temporal invalidation (soft forget with audit trail)
- GDPR — implement delete-all-memories for erasure requests
FAQ
Frequently asked questions
Should AI agents forget memories?
Yes — without eviction, stores grow unbounded, retrieval quality degrades and costs rise. Use TTLs, decay and relevance pruning. Park et al. (2023) decay recency at 0.995/hour.
What TTL should agent memories have?
Depends on type: session context (hours), preferences (months–years), policies (until superseded). Support bots often use 90–365 days for episodic events.
Can users delete their agent memories?
Production systems should expose forget/delete APIs for privacy compliance. Engram, Mem0 and Zep all support per-user deletion or invalidation.
Forgetting vs consolidation?
Consolidation promotes and merges facts into LTM. Forgetting removes stale or irrelevant data. Run consolidation first, then evict raw transcripts.
How does Zep handle memory invalidation?
Graphiti bi-temporally invalidates superseded facts — soft forget with audit trail, not hard delete. See Zep alternatives.
How do you prevent accidental forgetting?
Archive before delete, require confidence thresholds for auto-eviction, log all deletions and gate aggressive TTLs behind user consent for preferences.
How do you benchmark forgetting policies?
Run LOCOMO before/after eviction rules change. Ensure recall@k stays above threshold while store size shrinks. See memory metrics.