Memory Types · Hierarchy
Sensory & Buffer Memory in AI Systems
Sensory or buffer memory in AI systems is the brief holding area for raw inputs — audio chunks, image frames, tool streams — before the agent extracts meaningful facts into working or long-term memory.
Input pipeline
Definition
What is sensory / buffer memory?
The first stage in the memory hierarchy — a raw input cache with milliseconds-to-minutes retention before meaningful extraction.
Analogous to the human sensory register: information arrives, is held briefly, then either discarded or promoted to working memory. In agents, this is the streaming token buffer, image preprocessing queue or webhook payload hold — not yet structured facts.
Also called the input buffer or sensory stage. Distinct from working memory (context window) and long-term memory (Engram, Mem0, Zep external stores).
Stack
Where buffer memory sits in the agent stack
Input → sensory buffer → working/context → extraction → long-term store.
- Input — user message, audio stream, image upload, tool webhook
- Sensory buffer — raw hold; ring buffer or Redis list; seconds to minutes
- Working memory — context window + recent turns; current reasoning scratchpad
- Extraction — memory layer pulls facts from buffer/working into structured store
- Long-term store — Engram, Mem0, Zep, vector DB, knowledge graph
Implementation
Implementation patterns
Ring buffers, Redis lists, in-process deques and multimodal frame stacks.
- Ring buffers — fixed-size in-process deque; oldest input dropped when full
- Redis lists — TTL-scoped streams for multi-worker agents; see Redis for AI agent memory
- In-process deques — LangChain ConversationBuffer holding recent raw messages
- Multimodal frame stacks — image/audio frames queued before vision model processing
Promotion rule: when the buffer contains extractable facts (user stated a preference, tool returned durable data), the memory layer writes to LTM. Ephemeral inputs (greetings, transient tool noise) are discarded without promotion.
FAQ
Frequently asked questions
Do AI agents need sensory memory?
Multimodal and streaming agents yes — to hold raw inputs before extraction. Simple text chatbots often skip a dedicated buffer and go straight to working memory (context window).
Sensory buffer vs working memory?
Sensory buffer holds raw, unprocessed inputs briefly. Working memory is the active reasoning scratchpad (context window + recent turns). See short-term memory.
Multimodal agents and buffer memory?
Image frames, audio chunks and video streams queue in sensory buffers before vision/speech models process them. Then facts promote to LTM via the memory layer.
How do you tune buffer size?
Match input rate and extraction latency. Too small → dropped context. Too large → memory cost before promotion. Redis TTL lists are common for multi-worker setups.
Is this the same as human sensory memory?
Analogous but not identical. See AI memory vs human memory for where the analogy holds and breaks.