Infrastructure · Retrieval
Hybrid Search for AI Agent Memory
Hybrid search runs a vector search and a keyword search over the same store and fuses the two rankings. It matters more for memory than for document retrieval, because memories are full of the exact strings vector search handles worst: ticket references, order numbers, error codes, product names and symbol names, each of which is a visible failure when it is missed rather than a slightly worse ranking.
Three stages
Definition
What is hybrid search?
Two retrievers over one collection, combined into a single ranking. A vector search finds records whose meaning is close to the query, a keyword search finds records containing the query’s terms, and a fusion step merges the two lists into one.
The keyword half is usually BM25, a ranking function that scores a record by how often the query’s terms appear in it, weighted so that rare terms count for more than common ones. That weighting is the reason it is a good complement: the terms it scores highest are exactly the ones an embedding model represents least reliably.
The vector half works on meaning rather than tokens, so it retrieves a memory phrased entirely differently from the query. Neither is a subset of the other, which is why the combination outperforms either alone rather than merely averaging them.
None of that is specific to memory so far, and it is where most explanations stop. The interesting part is that a memory store has properties a document corpus does not: why agent memory needs hybrid search more than document search does.
The memory case
Why does agent memory need hybrid search more than document search does?
Because memories are short, dense with identifiers, and missing one is a visible failure rather than a degraded ranking. Three properties of a memory store push in the same direction, and none of them applies to a corpus of articles.
Identifiers are everywhere. Memories record what happened, and what happened involved ticket 4471-B, error PG-1105, the enterprise plan, a branch name or a function name. These are rare tokens, poorly placed in embedding space, and a user asking about ticket 4471-B expects that exact ticket rather than something similar to it.
Memories are short. A good memory is one atomic statement, so there is little surrounding text for an embedding to work with, and a single unusual token dominates the record’s meaning in a way it never would in a thousand-word article.
The failure is visible. A document search that returns the second-best passage produces a slightly worse answer. A memory search that misses the ticket a user just named produces an agent that says it has no record of something the user is looking at.
The keyword half has the opposite blind spot, which is why the answer is both rather than a switch to lexical search: how the two rankings are combined.
Fusion
How are the two result sets combined?
Either by rank, using reciprocal rank fusion, or by normalising both score scales and blending them with a weight. What does not work is adding the raw scores, because a cosine similarity and a BM25 score are not on the same scale and never will be.
Reciprocal rank fusion ignores the scores and uses positions: a memory ranked first in either list scores highly, one ranked twentieth in both scores low. Because it never compares the two score spaces, it needs no calibration and survives a change of embedding model or a reindex without retuning. It is the right default.
Weighted score fusion normalises both to a common range and applies a weight, which gives finer control and lets you favour one retriever deliberately. The cost is that the normalisation depends on the score distributions, so the weights need revisiting whenever either retriever changes.
One detail matters more in memory than elsewhere: fetch enough candidates from each side before fusing. If each retriever returns three memories and you keep four, the fusion has almost nothing to work with. Retrieve twenty from each, fuse, then cut.
Which raises the question of how much to favour each side when the store is memories rather than documents: weighting the two for a memory store.
Tuning
How should you weight vector and keyword search for memory?
Start balanced, then let the query decide. A fixed weight is a compromise between two query types that want opposite settings, and the store usually contains both.
The signal to route on is in the query itself. A query containing an identifier, an order number, an error code, a quoted phrase, is asking for an exact match and should lean lexical. A query in ordinary language, “what does this customer prefer”, is asking for meaning and should lean vector. Detecting the first is a matter of pattern matching rather than machine learning.
The default when nothing distinguishes the query should still favour vector slightly, because a memory phrased differently from the question is the more common case, and lexical search returning nothing at all is a real outcome when no term overlaps.
Two adjustments are worth making for a memory store specifically. Weight the lexical side higher for episodic memories, which are dense with references to concrete things, and lower for semantic memories, which are usually paraphrasable statements about a person. The distinction is on episodic versus semantic memory.
Weights only matter once the search is looking at the right rows, and in a memory store that is not a given: where metadata filtering fits in a memory search.
Filtering
Where does metadata filtering fit in a memory search?
Before both retrievers, because in a memory store the filter is a correctness boundary rather than a refinement. Every explainer written for document search treats the metadata filter as an optional narrowing. A memory search that returns another user’s memory is not a worse result, it is an incident.
Stores differ in when they apply it, and the difference is not cosmetic. Post-filtering ranks across everything and discards afterwards, which is safe for privacy only if the discard is reliable, and which quietly returns fewer results than you asked for, sometimes none, when the qualifying rows never reached the candidate set. Pre-filtering restricts first, so the ranking is drawn from qualifying rows only. Partitioning goes further and gives each user their own namespace, which is both the strongest boundary and the reason memory-scale search stays fast.
Beyond the user boundary, three filters earn their place in a memory system. Memory type, so a query about preferences is not answered from an episodic log. A time window, so an outdated fact is excluded rather than merely outranked. And a scope flag separating memories the user can see from internal notes, which matters as soon as an agent writes anything the user did not say.
The filter has to apply to both halves. A keyword search that ignores the partition while the vector search honours it produces a fused list containing rows that should not exist, and the fusion step will not catch it. The wider treatment of boundaries is on memory security and privacy.
Once the candidate set is both relevant and correctly bounded, the next question is whether a second model should re-order it: whether to rerank hybrid results for memory.
Second stage
Should you rerank hybrid results for memory?
Usually not, because the second stage a memory system needs is scoring rather than reranking, and adding both means paying twice to answer one question. Reranking is the standard next step in document retrieval, and it is the step most often copied into memory systems without asking whether it fits.
A cross-encoder reranker reads the query and each candidate together and produces a relevance judgement a bi-encoder cannot, because it sees both texts at once instead of comparing two independently produced vectors. Over long passages that is a real gain: the fused list is fifty passages, most of them partly relevant, and the ordering genuinely needs a closer read.
Memory candidates are not long passages. They are one-sentence statements, already narrowed to one user, and after fusion there are perhaps twenty. Relevance across twenty short atomic statements is not usually the thing that is wrong with the ranking. What is wrong is that three of them are stale, two contradict each other, and five say the same thing in different words, and a relevance model has no opinion about any of that.
There are two cases where a reranker still pays. A store large enough that fusion returns a hundred plausible candidates, which happens with shared organisational memory rather than personal memory, and a store whose memories are long, multi-sentence summaries rather than atomic facts. If either describes your store, rerank after fusion and before scoring, and measure the added latency against the same p99 target as the search itself.
For everything else, the budget is better spent on the stage the incumbents leave out: how hybrid search interacts with memory scoring.
The pipeline
How does hybrid search interact with memory scoring?
Hybrid search finds candidates, memory scoring ranks them, and a selection step decides what reaches the prompt. Collapsing the first two into one ranking is a common design error, and it produces retrieval that is relevant and badly prioritised.
The distinction is worth stating plainly. Hybrid search answers whether a memory is about the right subject. Memory scoring answers which of the relevant memories matters now, using recency, importance and, where the system records it, whether acting on that memory has previously worked. The scoring functions are on memory scoring.
The third stage is specific to memory and frequently missing. A memory store repeats itself, because the same preference gets stated across many sessions, so the top candidates after fusion are often five phrasings of one fact. Selecting one memory per distinct claim, rather than the top five by score, is what stops the budget being spent restating a single thing.
Deduplicating at write time reduces the problem before retrieval ever sees it, which is the cheaper fix, covered on how agents write and store memories. The full read path is on how agents retrieve memories.
All three stages sit on the critical path of every reply, so the cost is worth knowing: what hybrid search costs in latency.
Cost
What does hybrid search cost in latency?
Less than it looks, because the two searches run in parallel and a memory partition is small. The added cost is the slower of the two retrievers plus a fusion step that is arithmetic over a few dozen rows.
Scale is what makes this affordable for memory specifically. A search over one user’s memories runs against hundreds or thousands of records, not millions, because the partition boundary that exists for correctness also keeps the index small. Both retrievers are fast at that size, so the practical cost is one extra query rather than a doubling.
Two things do cost. Running the searches sequentially rather than concurrently makes the total the sum instead of the maximum, which is a mistake worth checking for. And a store that requires two separate systems, a vector database and a search engine, adds a network hop and an operational dependency, which is why a store supporting both natively is preferable.
The cost that does scale is embedding the query, which happens before either retriever runs and is frequently the largest single number in the trace. It is paid whether or not you add the lexical half, which is another way of saying the keyword search is close to free once you are already embedding. A store that caches embeddings for repeated queries removes it for the queries that repeat, and in an agent loop more of them repeat than you would expect.
Measure it at the p99 rather than the average, because retrieval sits in front of every reply and a slow tail is experienced as an agent that hesitates. Targets and the wider metric set are on memory evaluation metrics.
Whether you pay that cost at all depends on what your store holds: which memory stores support hybrid search.
Support
Which memory stores support hybrid search?
Enough of them that this is rarely a reason to change stacks, but the quality of support varies more than the presence of the feature. What to check is whether the fusion happens inside the store or in your application code.
- Engram runs on Weaviate, where vector and keyword search are fused natively in one query. For a memory layer this is the strongest fit, because the write-time deduplication search and the read-time retrieval both benefit from lexical matching. See Engram.
- Relational databases with full-text search plus a vector extension cover it well at memory scale, and are often already in the stack. See storage backends.
- Dedicated vector databases increasingly ship a keyword mode and a fusion option; the question to ask is whether both run in one query or whether you are issuing two and merging results yourself. See vector databases.
- Knowledge graphs take a different route: exact matching happens through entity resolution rather than through BM25, which handles identifiers well by construction. See knowledge graphs.
Three questions separate a store that supports hybrid search from one that supports it well. Does one query return one fused list, or do you issue two and merge them yourself. Does the filter apply before ranking on both halves, which is the question the previous section turns on. And can you set the fusion method and the per-retriever candidate depth, or is the store deciding both for you, in which case a lexical result set of three is invisible until you go looking for it.
Application-side fusion is workable and worth knowing you are doing, since it means two round trips, two sets of pagination and your own code owning the fusion. At memory scale that is acceptable; at document scale it usually is not.
Before adding any of it, the honest question is whether your store needs it at all: when vector-only retrieval is enough.
The exception
When is vector-only retrieval enough for memory?
When the store contains no identifiers. That single test settles it, and for a meaningful class of products the answer is that similarity alone is the whole requirement.
A memory store holding preferences, traits and stated goals in ordinary language is served well by vectors, because every query against it is a question about meaning and no query names a string that has to match exactly. A personal assistant remembering dietary requirements and travel preferences is the standard example.
Small stores are the second case. With a few dozen memories per user, almost everything relevant is retrieved regardless of method, and effort spent on retrieval would be better spent on what gets written, which is where quality is actually decided.
The exception to both is product vocabulary. Internal names, acronyms and jargon an embedding model has never encountered behave exactly like identifiers, so a store full of them needs the lexical half even if it contains no numbers at all.
A pragmatic sequence: start with vector-only, log the queries that return nothing useful, and look at what they have in common. If they are dominated by exact strings, the answer is on this page. If they are not, the problem is upstream in what is being stored, covered on writing memories, and the way to tell the two apart is on how to evaluate agent memory.
FAQ
Frequently asked questions
The practical questions about running two retrievers: which fusion, how many candidates, and what it costs.
How many candidates should each retriever return before fusion?
Enough that the fusion has something to work with, which in practice means roughly four times the number you intend to keep, from each side independently. Fetching five from each and keeping four is the common mistake: the fusion step is then mostly deciding the order of a set that was already fixed by the retrievers.
Do you need two separate databases to run hybrid search?
No. Several stores run both retrievers inside one query, and at memory scale a relational database with full-text search and a vector extension covers it. Two systems means two round trips, two copies of the data to keep in step, and your own code owning the fusion. See storage backends.
Does hybrid search need retuning when you change embedding model?
Reciprocal rank fusion does not, because it uses positions rather than scores and never compares the two score spaces. Weighted score fusion does, because the normalisation depends on the score distribution the model produces. That difference is the main practical reason to prefer rank fusion as the default.
Can hybrid search fix bad memories?
No. Retrieval can only return what was written, so a store full of vague or duplicated memories returns vague or duplicated results however it is searched. If queries fail because the right fact was never recorded, the fix is on how agents write and store memories, not in the search.
How do you tell whether hybrid search actually helped?
Split your evaluation queries into ones naming an exact string and ones asking about meaning, then compare retrieval before and after on each half separately. A single averaged number hides the result, because the gain is concentrated almost entirely in the first half. See memory evaluation metrics.
Is hybrid search the same thing as reranking?
No. Hybrid search combines two retrievers into one candidate list. Reranking is a separate model that re-orders a list already retrieved. They solve different problems and are often used together in document search, though a memory system usually needs scoring rather than reranking as its second stage.