What are hybrid search and reranking?

Keyword search and vector search fail in opposite ways, which is exactly why they work well together. One misses paraphrases; the other misses exact strings.

Why is vector search alone not enough?

Vector search matches meaning, which is great for "how do I get my money back" finding a page titled "Refund policy". But embeddings compress text into a general sense of its topic, so they are weak at exact details: error code E4012, a part number, a person's surname, an API name, or a rare technical term. A passage mentioning "E4011" can look nearly identical to one mentioning "E4012". Keyword search handles these cases easily, because it matches the actual words.

What is BM25?

BM25 is the classic keyword ranking function used by search engines such as Elasticsearch, OpenSearch and Lucene. It scores a document higher when it contains the query's terms, gives more weight to rare terms than common ones, and stops rewarding a term after it has appeared a few times, while adjusting for document length. It needs no model, runs fast, and has been a strong baseline for decades.

How do you combine keyword and vector results?

The most common method is reciprocal rank fusion (RRF), introduced by Cormack, Clarke and Büttcher in 2009. Instead of trying to compare BM25 scores with cosine similarities, which are on different scales, RRF uses only each document's rank in each list. A document's fused score is the sum, over the lists, of 1 divided by (k plus its rank), with k commonly set to 60. Documents ranked well by both methods rise to the top.

1. 1
   Run both searches

   Retrieve, for example, the top 50 results from BM25 and the top 50 from vector search.
2. 2
   Fuse

   Merge the lists with reciprocal rank fusion or a weighted score.
3. 3
   Rerank

   Pass the top 20 to 100 fused candidates to a reranker.
4. 4
   Select

   Keep the best few, often 3 to 10, for the model's context.

Many search engines and vector databases now support hybrid queries directly.

What is a reranker?

A reranker is a model that scores how well each candidate passage answers the query by reading both together. Most are cross-encoders: unlike embeddings, which encode the query and passage separately, a cross-encoder reads them as one input, so it can judge relevance much more precisely. It is too slow to run over a whole collection, which is why it only reorders the short list that search produced. Early work applied BERT to reranking in 2019, and hosted and open-source rerankers are now widely available. An LLM can also be used as a reranker, at higher cost.

How much do hybrid search and reranking help?

They often help substantially, and the effect is easy to measure. In Anthropic's 2024 contextual retrieval experiments, adding BM25 to embedding search, together with contextualised chunks, reduced failed retrievals by 49% compared with embeddings alone, and adding reranking brought the reduction to 67%. Results vary by dataset, so measure on your own questions.

• ✓Do: Add BM25 when users search for names, codes, IDs or technical terms
• ✓Do: Retrieve generously, then let the reranker pick the few passages the model sees
• ✓Do: Measure recall at each stage: search, fusion and reranking
• ✕Avoid: Send 50 unranked passages to the model and hope it finds the right one
• ✕Avoid: Tune fusion weights on a handful of queries; use a proper test set
• ✕Avoid: Forget that rerankers add latency, often tens to hundreds of milliseconds