RAG can still fail even though it reduces hallucination overall. Common failure modes: retrieval failure (the right document exists but isn't fetched), bad chunking, irrelevant-but-similar context, the model ignoring a correctly retrieved chunk (lost in the middle), conflicting documents, hallucinated synthesis across retrieved facts, citations that don't actually support the claim, a stale index, and permission leakage where the retriever surfaces documents the user shouldn't see.

RAG fails not only at generation time but at every stage of the pipeline. Retrieval failure happens when poor embeddings, a bad query, wrong chunking, missing metadata, or weak vector search cause the correct document to exist but never get fetched. Bad chunking cuts both ways: chunks too small lose context, chunks too large add noise. Irrelevant context is the retriever returning something semantically similar but not actually answer-relevant — e.g. querying for the enterprise refund policy and getting the general free-tier policy back. Lost-in-the-middle is the model ignoring the right chunk anyway because it's buried in a long or competing context. Conflicting documents (an old policy saying 15 days, a new one saying 30) leave the model no way to know which is current without metadata like recency. Hallucinated synthesis is the model combining multiple retrieved facts incorrectly. No citation grounding means the answer looks correct but isn't traceable to a source, or worse, cites the wrong document while still being right. Stale index means the vector database holds outdated documents. Permission leakage is the retriever fetching documents the user shouldn't have access to — a security failure, not just a quality one.