A retriever is a runnable component whose job is: given a query, return the most relevant chunks
Retrieval quality is usually the real bottleneck in a RAG system — generation can only be as good as the context it's handed. Before reaching for any advanced technique, it's worth knowing exactly what the simplest possible retriever does, what its two configuration knobs actually control, and where it breaks down.
A retriever exposes a single invoke(query) method that takes a query and returns chunks, no matter what strategy runs underneath. That uniform interface is what makes it possible to swap the retrieval strategy — everything covered in Advanced Retrieval and Search Types — without touching the rest of the pipeline.
A retriever is a runnable component — it exposes a standard invoke() call, so it can be plugged into a chain interchangeably regardless of which underlying strategy it uses (plain similarity search, MMR, hybrid search, or anything else). This abstraction is what lets a team change the retrieval implementation entirely — from a plain vector store to a hybrid search pipeline — without rewriting the augmentation or generation code that consumes its output.
Embed the query, run cosine similarity search against the vector database, return the top-k closest matches. Nothing more.
Limitations:
These three limitations are exactly what Advanced Retrieval and Search Types exist to fix — each technique there targets one of them specifically.
The vanilla retriever above already ranks results — that's what "return the top-k closest matches" means: every candidate chunk gets a cosine similarity score against the query vector, and the retriever sorts by that score before returning the top k. This first ranking pass is cheap specifically because it's a bi-encoder comparison — the query is embedded once, and it's compared against millions of already-precomputed chunk vectors with simple vector math, which is why it can run over an entire large corpus in milliseconds (see Vector Databases for how HNSW/IVF make this scale).
Re-ranking (covered in full in Advanced Retrieval) is a deliberately separate, second sorting pass that only runs over the small shortlist the first ranking already produced — typically the top 10-20 candidates, never the whole corpus. It uses a different, far more expensive signal (a cross-encoder that reads the query and each candidate document together, rather than comparing two independently-computed vectors) specifically because that signal is too slow to run over millions of chunks but perfectly affordable over twenty.
The reason this two-stage design exists at all, rather than just always using the more accurate cross-encoder signal: a cross-encoder has no way to pre-compute anything ahead of time, because its score depends on the query and the document being read together — there's no "document vector" to store in advance the way a bi-encoder produces. Ranking (bi-encoder, cheap, corpus-wide) and re-ranking (cross-encoder, expensive, shortlist-only) are therefore not competing techniques — they're two stages of the same pipeline, each doing the part the other can't do efficiently.
k — how many chunks to return — is a real trade-off, not a default to leave alone. A k that's too small risks missing a genuinely relevant chunk that scored just outside the cutoff, especially when the answer to a question is spread across more than one chunk. A k that's too large drags in marginally-relevant chunks that dilute the prompt, increase cost, and reintroduce the "lost in the middle" risk discussed in Generation Failure Modes. Common values in production sit between 3 and 10, tuned empirically against an evaluation set (see Evaluation) — specifically by tracking context precision (are the returned chunks relevant?) and context recall (did we get everything needed?) as k changes.
Instead of (or in addition to) a fixed k, a retriever can filter out any result below a minimum similarity score — discarding chunks that are technically the "closest available" but still not actually relevant. This sounds simple but has a subtle trap: cosine similarity scores are not calibrated the same way across different queries. A very specific, narrow query might have its best-matching chunk score 0.92, while a broad, ambiguous query's best match might only score 0.71 — even when both retrievals are working correctly. A single global threshold tuned on one kind of query can silently reject good results for another kind. In practice, thresholds need validation against a representative range of real query types, not a single test case, and are often combined with a floor on k (e.g., "always return at least 1 result, even below threshold, but flag it as low-confidence") rather than used as a hard cutoff alone.
A retriever's search almost always runs alongside metadata filtering rather than instead of it — restricting the candidate set to a specific tenant's documents, a date range, or an access-permitted subset, and only then ranking by similarity within that restricted set (see Vector Databases for pre-filter vs. post-filter semantics). This combination — filter first, rank second — is what makes retrieval usable in any multi-tenant or access-controlled system, and it's worth testing explicitly: a retriever that works in a single-tenant demo can behave very differently once real filters are added, because a narrow filter can shrink the effective candidate pool enough to change which chunks even show up in the top-k.