Hybrid Search and Re-Ranking: The Cheapest Quality Win
Keywords catch what vectors miss, and re-rankers fix the order.
Hybrid search runs two kinds of retrieval side by side, BM25 for exact keyword matching and vector search for meaning, then merges their results with reciprocal rank fusion. A cross-encoder re-ranker then reads the question together with each surviving candidate and puts them in a better order. Together they are usually among the cheapest relevance wins you can add to a RAG (retrieval-augmented generation) pipeline.
Many systems stop at vector search. Part 3 showed why that is not enough: asked "Which form is HR-27?", our embedding search put the vacation chunk above the one that mentions the form. This part adds keyword search, fusion and a re-ranker to the handbook assistant, and shows each one fixing a different failure.
What problem does hybrid search actually solve?
Dense vector retrieval struggles with rare identifiers, exact codes and names that the embedding model has little sense of. BM25 struggles with paraphrases, where the question and the document share no words. Both failure modes are common in real use.
The two methods tend to fail on different questions, which is why combining them pays off. The BEIR benchmark tested retrieval models on datasets from many domains without training on them, and found BM25 a strong baseline that many dense retrievers did not beat on unfamiliar domains, while re-ranking and late-interaction models did best on average, at a higher computational cost (Thakur et al. 2021, BEIR: A Heterogenous Benchmark for Zero-shot Evaluation of Information Retrieval Models).
The mechanism is a pipeline: two retrievers running in parallel, a fusion step that merges their ranked lists without needing comparable scores, and optionally a re-ranker that looks at the question and each candidate together. Each piece is simple.
How does BM25 retrieval work under the hood?
BM25 is a ranking function for keyword search over an inverted index. Each distinct word in your collection maps to a postings list: the documents that contain it, with how often it appears. When a question arrives, the system splits it into words, looks up each word's postings list, and scores every document that contains at least one of them.
The scoring formula rewards rare words heavily and stops rewarding repeated words after a point:
IDF stands for inverse document frequency: a word that appears in only three documents, like "SKU-44921", counts for far more than "the", which appears in every document and so adds almost nothing. The denominator normalizes for document length, so a long document does not win just by containing more words; the b parameter sets how strongly. The k1 parameter sets how quickly extra repetitions of a word stop adding to the score. Elasticsearch's defaults are k1 = 1.2 and b = 0.75 (Elasticsearch's similarity settings).
This makes BM25 strong for questions with distinctive words: error codes, product numbers, form numbers, names. If the handbook mentions "form HR-27" and an employee searches for exactly that, BM25 finds it. An embedding model might return something merely related.
How does dense retrieval fail on queries BM25 catches?
Dense retrieval maps questions and documents into the same vector space with a bi-encoder, a model that embeds each text separately, and then finds the nearest vectors, as in Part 3.
The weakness is in what the model learned. Rare tokens, numbers and in-house jargon appeared little or not at all in its training data, so their vectors do not capture how decisive an exact match is. The model treats "HR-27" as a vague piece of text about forms or HR, not as a precise identifier.
The complementarity works both ways. A dense retriever can connect "PTO request process" with a document titled "How to submit vacation time" even though they share no words, which BM25 cannot do without synonym lists. Employees paraphrase policy questions in their own words while the handbook uses formal language, so you need both.
How does reciprocal rank fusion combine two incompatible score distributions?
BM25 scores have no fixed upper limit, while cosine similarity stays between -1 and 1. Averaging them directly lets whichever has the bigger numbers dominate.
Reciprocal rank fusion (RRF) ignores the raw scores entirely and uses only positions. Each document receives 1 / (k + rank) from each list it appears in, where rank counts from 1 and k is a damping constant, 60 in the original paper (Cormack et al. 2009, Reciprocal Rank Fusion outperforms Condorcet and individual Rank Learning Methods). Documents that appear in both lists collect both contributions, and the final ranking sorts by the total.
It printed (rounded here):
"carryover" appears in both lists, second and first, so it collects 1/62 + 1/61 and wins. A document ranked 1st by BM25 and 15th by vector search gets 1/61 + 1/75 ≈ 0.0297. A document ranked 5th in both gets 1/65 + 1/65 ≈ 0.0308. RRF rewards documents that both retrievers think are good.
In the original paper, RRF beat each individual system and the standard Condorcet Fuse method, and the authors found k = 60 near-optimal but not critical. It needs no score normalization and no labelled training data, which is why it is a common default. The main alternative is to normalize both scores to a common range and take a weighted sum. A later study found that such a weighted combination, tuned on a small set of labelled questions, outperformed RRF, and that RRF is sensitive to its k (Bruch et al. 2022, An Analysis of Fusion Functions for Hybrid Retrieval). Start with RRF; move to a tuned weighted sum when you have labelled questions to tune it on, which Part 7 builds.
What does a cross-encoder re-ranker do that vector search cannot?
A cross-encoder takes the question and one candidate together, joined into a single input, and runs them through a transformer so that every word of the question can attend to every word of the candidate (Sentence Transformers' cross-encoder models). It can see that "not eligible" in a document answers "Am I eligible?", or that "required before" sets an order of events.
A bi-encoder, the kind behind vector search, compresses each document into one vector without seeing the question, and that compression loses detail. A cross-encoder pays a full model pass for every question and candidate pair, so it is only used on a short list, typically the top 20 to 100 candidates from the fusion step.
That cost is why search has two stages. With a bi-encoder you embed each of D documents once and each of Q questions once, then the cross-encoder scores N candidates per question: about D + Q + NQ model passes in total, instead of D × Q if the cross-encoder had to score every document for every question.
How long does re-ranking take? We measured cross-encoder/ms-marco-MiniLM-L6-v2, a small cross-encoder of about 22.7 million parameters, on a laptop's CPU (AMD Ryzen 7 7730U, PyTorch 2.14, October 2026). Re-ranking 20 candidates of 100 tokens took about 0.2 seconds; 50 candidates of 250 tokens took about 1.3 to 1.5 seconds. A GPU is much faster, so measure on your own hardware and keep the candidate list short.
The quality gain is not subtle. Two measures used here, and again in Part 7's evaluation of retrieval metrics, need a definition. MRR, mean reciprocal rank, looks at where the first relevant result appears for each question, takes one divided by that position (1 for first place, 0.5 for second), and averages over all questions; 1.0 means the right result always comes first. nDCG@k, normalized discounted cumulative gain, scores a whole ranked list, giving more credit for relevant results near the top, and divides by the best possible score, so 1.0 is a perfect order. On MS MARCO, a large benchmark of real search questions, BM25 alone scores about 0.19 MRR@10, and re-ranking BM25's top 1,000 candidates with this cross-encoder raises it to 0.39 (MS MARCO's passage ranking leaderboard, the ms-marco-MiniLM-L6-v2 model card).
Build it: hybrid search for the handbook
This is the series' fourth file, search.py. It adds BM25 keyword search to Part 3's exact vector search, fuses the two with RRF, and re-ranks the top five with the cross-encoder. Part 4's comparison table explains why it uses the small rank-bm25 library: open-source Chroma has no BM25 ranking of its own.
When we ran it (rank-bm25 0.2.2, sentence-transformers 6.1.0, October 2026), it printed:
Each method fails once, in opposite directions. For "Which form is HR-27?", vector search puts the vacation chunk (h3.2) first, while keyword search finds the home office chunk (h5.3) that names the form; fusion puts it first, and the re-ranker agrees by a wide margin. For "How do I book time off?", keyword search is fooled by the word "book", which only the travel chunk (h5.2) contains, and fusion still ranks travel first. The re-ranker reads the question against each chunk and promotes vacation (h3.2), helped by the "Time off > Vacation" heading path that Part 2 put into the chunk.
The re-ranker's scores are not probabilities, and negative values are normal. Only the order and the gaps matter, and Part 6 shows how to turn them into a calibrated "is anything here relevant?" check.
How do you choose a re-ranking model and deployment pattern?
Two families dominate. The first is small cross-encoders such as cross-encoder/ms-marco-MiniLM-L6-v2, 6 transformer layers and about 22.7 million parameters, trained on MS MARCO, a large set of English web search questions (its model card). It dates from 2021. Newer re-rankers often beat it, and several handle many languages: for example BAAI's bge-reranker-v2-m3, a lightweight multilingual re-ranker, and Qwen3-Reranker-0.6B, which supports over 100 languages and inputs of 32K tokens, both under the Apache 2.0 license (bge-reranker-v2-m3, Qwen3-Reranker-0.6B). Several API providers also offer hosted re-ranking. If your users do not write in English, choose a multilingual model.
The second family is late-interaction models, notably ColBERT. ColBERT encodes the question and each document into one vector per token, and scores relevance by matching each question token with its most similar document token and summing those best matches (Khattab and Zaharia 2020, ColBERT: Efficient and Effective Passage Search via Contextualized Late Interaction over BERT). Unlike a cross-encoder, the document token vectors can be computed ahead of time and stored, so each search costs less, while keeping a token-level match. ColBERT sits between bi-encoders and cross-encoders in both quality and cost, at the price of storing many vectors per document.
For the handbook assistant, a small cross-encoder re-ranking the top handful of hybrid results is the pragmatic choice. It needs nothing beyond a Python process running the model. If re-ranking becomes your bottleneck, re-rank fewer or shorter candidates, use a GPU, or move to ColBERT-style late interaction.
Quick Reference
| Property | Value |
|---|---|
| Default fusion algorithm | Reciprocal Rank Fusion (RRF) |
| RRF constant k | 60 in the original paper, near-optimal but not critical |
| Alternative fusion | Normalized, weighted sum of scores, tuned on labelled questions |
| Typical re-rank depth | The top 20 to 100 candidates |
| BM25 defaults in Elasticsearch | k1 = 1.2, b = 0.75 |
| Small cross-encoder | cross-encoder/ms-marco-MiniLM-L6-v2 (about 22.7M parameters) |
| Its re-ranking time on a laptop CPU | About 0.2 s for 20 candidates of 100 tokens, 1.3 to 1.5 s for 50 of 250 (our measurement) |
| MS MARCO MRR@10 | 0.187 for BM25 alone, 0.39 after this cross-encoder |
| Gain from hybrid and re-ranking on your data | Varies; measure it (Part 7) |
Frequently Asked Questions
Q: Why not just use a larger embedding model instead of hybrid search?
Larger embedding models improve recall on paraphrases but do not reliably solve exact matching. An embedding model has no special sense that "HR-27" or "SKU-44921" is a precise identifier unless it saw such patterns often in training. Rare codes, new product names and in-house acronyms stay hard. BM25 gives you exact matching cheaply.
Q: Does RRF work with more than two retrieval methods?
Yes. RRF sums contributions over any number of ranked lists. You can add a third retriever, such as a learned sparse model like SPLADE, and fuse all three. The only requirement is that all retrievers use the same document IDs, so the contributions add up correctly.
Q: When should I skip re-ranking entirely?
When your latency budget is very tight and your measurements show the first stage already puts the right chunk on top. Re-ranking helps most when the candidate list mixes relevant and irrelevant documents that need finer judgment. Measure both with and without it on your labelled questions before deciding.
Q: Can I use an LLM (large language model) as a re-ranker instead of a cross-encoder?
Yes. Some systems give an LLM the question and a list of candidates and ask it to reorder them. This is slower and more expensive than a cross-encoder, but it can apply more complex relevance rules. Start with a cross-encoder; move to an LLM re-ranker only if the cross-encoder keeps misjudging cases you care about and you can afford the latency.
Q: How do I evaluate whether hybrid search and re-ranking are working?
Track recall@k and nDCG@k on a representative set of labelled questions, with k matching what your pipeline passes on (often 5 or 10). Compare four configurations: BM25 only, dense only, hybrid with RRF, and hybrid plus re-ranker. If the last two do not clearly beat the first two, your questions may not exercise the cases where the methods differ, or your re-ranking depth is too shallow. Part 7 builds exactly this comparison.
Test yourself
Your handbook chatbot receives the query "Can I carry over unused PTO to next year?" The top three BM25 results are: (1) a page titled "PTO Policy" with a table of accrual rates, (2) a page titled "Benefits Overview" listing all benefits, (3) a page titled "PTO Carryover Rules." The vector search top three are: (1) "PTO Carryover Rules," (2) "Unused Vacation Time Policy," (3) "Annual Leave and Rollover." After RRF fusion, the "PTO Carryover Rules" page ranks first. When the cross-encoder re-ranks, it demotes "PTO Carryover Rules" to third and promotes "Annual Leave and Rollover" to first. Is the cross-encoder broken?
Answer: Probably not. RRF put "PTO Carryover Rules" first because it appeared in both lists, third and first, which earns more than any page found by only one retriever. The cross-encoder judges something different: it reads the question and each page together. "PTO Carryover Rules" may match the words but spend most of its text on exceptions and disqualifications, while "Annual Leave and Rollover" may use other terms yet describe exactly what the employee asks about: unused days moving to the next year. That is the behavior a re-ranker is for. The right next step is to read both pages against this question to confirm the judgment, then check whether the pattern holds across other PTO questions in your labelled set. If the re-ranker is wrong on many of them, try a different re-ranking model.
Next: Grounding and Citations: Answers You Can Check. It builds the prompt from these chunks with citation IDs, and checks every citation the model gives back.
If you want breakdowns like this every week, how real retrieval systems work at the mechanical level, not just which library to import, subscribe to Internals Decoded at internalsdecoded.com.
Sources
- BEIR: A Heterogenous Benchmark for Zero-shot Evaluation of Information Retrieval Models (Thakur et al. 2021)
- Similarity settings (Elasticsearch, read October 2026)
- Reciprocal Rank Fusion outperforms Condorcet and individual Rank Learning Methods (Cormack, Clarke and Büttcher, SIGIR 2009)
- An Analysis of Fusion Functions for Hybrid Retrieval (Bruch et al. 2022)
- Pretrained cross-encoder models (Sentence Transformers)
- ms-marco-MiniLM-L6-v2 model card (Hugging Face)
- MS MARCO passage ranking leaderboard (Microsoft)
- bge-reranker-v2-m3 model card (Hugging Face)
- Qwen3-Reranker-0.6B model card (Hugging Face)
- ColBERT: Efficient and Effective Passage Search via Contextualized Late Interaction over BERT (Khattab and Zaharia 2020)
- ColBERT GitHub repository
