Instead of picking the top-k highest-scoring chunks outright, MMR iteratively picks chunks that are both relevant to the query and different from chunks already selected — going candidate by candidate, always choosing whichever remaining one best balances "still relevant" against "not too similar to what's already picked." Formally, at each step MMR selects the candidate maximizing λ · sim(candidate, query) − (1 − λ) · max(sim(candidate, already_selected)) — relevance minus a penalty for resembling what's already in the result set.

fetch_k is how many candidates to pull before diversifying down to the final k. lambda_mult is a dial from 0 to 1: 1.0 is pure relevance (behaves like vanilla retrieval), 0.0 is pure diversity, and 0.5 is a common balanced default. Use it when the document set has overlapping or duplicate-ish content and the LLM needs a variety of angles, not five versions of the same fact.

An LLM generates several different phrasings of the original question. Each rephrased version runs through semantic search separately, and all the results are merged and de-duplicated — increasing the chance of matching however the source documents happen to be worded.

MQR does not use lambda_mult — that parameter is MMR-specific. Its real controls are which LLM generates the rephrasings and how many variants get generated. It costs one extra LLM call per user query, so it adds latency and cost — worth it for query-heavy ambiguity, not worth it when queries are already clear and well-structured.

The fast bi-encoder search used for initial retrieval narrows millions of documents down to a reasonable shortlist quickly, but it isn't the most accurate way to judge fine-grained relevance. Reranking retrieves a wider initial set (e.g., top-20) with the fast bi-encoder, then uses a slower but far more accurate cross-encoder to re-score and re-order that shortlist, keeping only the true top few (e.g., top-5) to actually send to the LLM.

In practice, teams reach for either an open-weight cross-encoder (the bge-reranker family, or Sentence-Transformers cross-encoders, self-hosted) or a managed reranking API (Cohere's Rerank endpoint is a common choice) rather than training one from scratch. A middle-ground alternative is ColBERT, which uses "late interaction": instead of one vector per chunk (bi-encoder) or one joint pass per query-document pair (cross-encoder), it keeps a vector per token and computes fine-grained token-to-token similarity at query time — recovering much of a cross-encoder's precision while staying far cheaper to run over a large candidate set, since document token vectors can still be pre-computed. Use full reranking whenever retrieval precision matters a lot and the extra latency and cost of a second pass is affordable — a common quality-boost layer in production RAG.

Users naturally mix semantic intent and structured filters in one sentence — "insurance claims from 2023 above $10,000" — but a vanilla retriever can't automatically separate "insurance claims" (semantic) from "year = 2023 AND amount > 10000" (a structured filter). A self-query retriever has an LLM parse the natural-language query and automatically extract both the semantic search string and the structured metadata filters, then runs a filtered similarity search using both.

This only works as well as the metadata schema it has to parse against (see Document Loaders) — a self-query retriever can't extract a filter on a field that was never populated as metadata in the first place.

Small chunks are great for precise matching — a focused chunk embeds more distinctly — but too small to give the LLM enough context to actually answer well. Large chunks give more context but embed less precisely. This retriever embeds and indexes small chunks for precise search matching, but when a small chunk is retrieved, returns its larger "parent" chunk or document — with more surrounding context — to the LLM instead.

This is a special case of a more general "small-to-big" idea: retrieve on a fine-grained signal (a sentence, a proposition, a small chunk) but generate on a coarser one (the paragraph or section it came from) — the same trade-off contextual retrieval and propositional chunking approach from the indexing side instead.

Questions and their answers are often phrased very differently — a casual user question might not closely match the formal, technical style of the actual answer text in a document, hurting semantic search accuracy. HyDE first asks the LLM to generate a hypothetical (possibly not even fully correct) answer to the question, then embeds and searches with that hypothetical answer instead of the raw question. Because the hypothetical answer is written in a style closer to how real answers are written, it tends to retrieve better matches than the raw question would.

The trade-off is direct: an extra LLM call before every search, and a risk that a badly hallucinated hypothetical answer actively steers retrieval toward the wrong part of the corpus rather than the right one — worth validating on real queries before trusting it as a default rather than a targeted fix for known phrasing-mismatch cases.