Infrastructure · Vectors
Embeddings Explained: Semantic Memory
Embeddings turn text into numerical vectors so AI agents can find memories by meaning — the foundation of semantic long-term memory and vector-based retrieval.
Text
“User prefers email over phone”
Vector
[0.12, -0.34, 0.89, …]
Definition
What are embeddings?
Text → dense numerical vector; similar meaning → nearby vectors in high-dimensional space.
Embedding models (OpenAI text-embedding-3, Cohere embed-v3, open-source e5/BGE) map sentences and facts into vectors. Cosine similarity measures semantic closeness — the basis of semantic memory.
Pipeline
How embeddings power agent memory
Write: memory text → embed → store vector. Retrieve: query → embed → nearest neighbors.
- Extract fact from conversation
- Call embedding API (or local model)
- Store vector + metadata in Weaviate, Pinecone, pgvector, etc.
- On query, embed the user message and run similarity search
- Inject top-k results into the LLM context
Engram uses Weaviate’s embedding pipeline natively. Mem0 reports median search latency 0.148 s on LOCOMO (Chhikara et al., 2025).
Models
Choosing an embedding model for memory
| Criteria | Why it matters |
|---|---|
| Dimension | Storage cost and recall quality (768–3072 typical) |
| Multilingual | Global user bases need multilingual models |
| Domain | Medical/legal may need domain-tuned models |
| Cost & latency | Per-write API calls add up at scale |
| Model change | Requires full re-embed migration |
→ Memory management (migration notes)
Hybrid
Embeddings vs keyword search
Vectors win on paraphrase and meaning; BM25 wins on exact IDs, SKUs and rare tokens.
Production memory stacks often combine both — hybrid search fuses vector and keyword scores. See hybrid search for memory retrieval.
FAQ
Frequently asked questions
Are embeddings required for AI agent memory?
Not always — keyword/BM25 and graph traversal work without vectors. Most semantic long-term memory stacks use embeddings for meaning-based retrieval. Engram, Mem0 and Zep all embed text for similarity search.
Must write and retrieve use the same embedding model?
Yes — different models produce incompatible vector spaces. Changing models requires re-embedding all stored memories.
How do you migrate when changing embedding models?
Export memories, re-embed with new model, dual-write to new collection, validate recall@k on LOCOMO, cut over. See memory management.
Do knowledge graphs need embeddings?
Graphs can query by structure alone, but hybrid systems embed node text for semantic graph traversal. Zep combines Graphiti graph + vector search.
What embeddings does Engram use?
Engram runs on Weaviate — uses Weaviate's embedding modules (OpenAI, Cohere, HuggingFace, etc.). Configurable per collection. See Engram explained.
What embedding dimension should I use?
Match your model (e.g. text-embedding-3-small = 1536). Higher dims can improve recall but increase storage and latency. Benchmark on your domain.
Should you fine-tune embeddings for agent memory?
Rarely needed early — general models work for most domains. Fine-tune when recall@k plateaus on domain-specific jargon. Re-embed entire store after fine-tuning.